WIP: Aggregation push-down
Started by Antonin Houskaalmost 9 years ago58 messages
This is a new version of the patch I presented in [1]/messages/by-id/29111.1483984605@localhost. A new thread seems
appropriate because the current version can aggregate both base relations and
joins, so the original subject would no longer match.
There's still work to do but I'd consider the patch complete in terms of
concept. A few things worth attention for those who want to look into the
code:
* I've abandoned the concept of aggmultifn proposed in [1]/messages/by-id/29111.1483984605@localhost, as it doesn't
appear to be very useful. That implies that a "grouped join" can be formed
in 2 ways: 1) join a grouped relation to a "plain" (i.e. non-grouped) one,
2) join 2 plain relations and aggregate the result. However, w/o the
aggmultifn we can't join 2 grouped relations.
* GroupedVar type is used to propagate the result of partial aggregation from
to the top-level join. It's conceptually very similar to PlaceHolderVar.
* Although I intended to use the "unique join" feature [2]https://commitfest.postgresql.org/13/859/, I postponed it so
far. The point is that [2]https://commitfest.postgresql.org/13/859/ does conflict with my patch and thus I'd have to
rebase the patch more often. Anyway, the impact of [2]https://commitfest.postgresql.org/13/859/ on aggregation
finalization (i.e. possible avoidance of the "finalize aggregate node"
setup) is not really specific to my patch.
* Scan of base relation or join result can be partially aggregated for 2
reasons: 1) it makes the whole plan cheaper because the aggregation takes
place on remote node and thus the amount of data to be transferred via
network is significanlty reduced, 2) aggregate functions are rather
expensive so it makes sense to evaluate them by multiple parallel workers.
The patch contains both of these features as they are hard to
separate from each other.
While 1) needs additional work on postgres_fdw, scripts to simulate 2) are
attached. Planner settings are such that cost of expression evaluation is
significant, so that it's worth to engage multiple parallel workers.
In my environment it yields the following output:
Parallel Finalize HashAggregate
Group Key: a.i
-> Gather Merge
Workers Planned: 4
-> Merge Join
Merge Cond: (b.parent = a.i)
-> Sort
Sort Key: b.parent
-> Parallel Partial HashAggregate
Group Key: b.parent
-> Hash Join
Hash Cond: ((b.parent = c.parent) AND (b.j = c.k))
-> Parallel Seq Scan on b
-> Hash
-> Seq Scan on c
-> Sort
Sort Key: a.i
-> Seq Scan on a
Feedback is appreciated.
[1]: /messages/by-id/29111.1483984605@localhost
[2]: https://commitfest.postgresql.org/13/859/
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de, http://www.cybertec.at
Attachments:
agg_pushdown.difftext/x-diffDownload
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
new file mode 100644
index 5a84742..d7afcea
*** a/src/backend/executor/execExpr.c
--- b/src/backend/executor/execExpr.c
*************** ExecInitExprRec(Expr *node, PlanState *p
*** 717,722 ****
--- 717,749 ----
break;
}
+ case T_GroupedVar:
+ /*
+ * GroupedVar is treated as an aggregate if it appears in the
+ * targetlist of Agg node, but as a normal variable elsewhere.
+ */
+ if (parent && (IsA(parent, AggState)))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ /*
+ * Currently GroupedVar can only represent partial aggregate.
+ */
+ Assert(gvar->agg_partial != NULL);
+
+ ExecInitExprRec((Expr *) gvar->agg_partial, parent, state,
+ resv, resnull);
+ break;
+ }
+ else
+ {
+ /*
+ * set_plan_refs should have replaced GroupedVar in the
+ * targetlist with an ordinary Var.
+ */
+ elog(ERROR, "parent of GroupedVar is not Agg node");
+ }
+
case T_GroupingFunc:
{
GroupingFunc *grp_node = (GroupingFunc *) node;
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
new file mode 100644
index ef35da6..70d2367
*** a/src/backend/executor/nodeAgg.c
--- b/src/backend/executor/nodeAgg.c
*************** find_unaggregated_cols_walker(Node *node
*** 1826,1831 ****
--- 1826,1842 ----
/* do not descend into aggregate exprs */
return false;
}
+ if (IsA(node, GroupedVar))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ /*
+ * GroupedVar is currently used only for partial aggregation, so treat
+ * it like an Aggref above.
+ */
+ Assert(gvar->agg_partial != NULL);
+ return false;
+ }
return expression_tree_walker(node, find_unaggregated_cols_walker,
(void *) colnos);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
new file mode 100644
index 61bc502..5f5bb4f
*** a/src/backend/nodes/copyfuncs.c
--- b/src/backend/nodes/copyfuncs.c
*************** _copyPlaceHolderVar(const PlaceHolderVar
*** 2189,2194 ****
--- 2189,2209 ----
}
/*
+ * _copyGroupedVar
+ */
+ static GroupedVar *
+ _copyGroupedVar(const GroupedVar *from)
+ {
+ GroupedVar *newnode = makeNode(GroupedVar);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(gvid);
+
+ return newnode;
+ }
+
+ /*
* _copySpecialJoinInfo
*/
static SpecialJoinInfo *
*************** copyObjectImpl(const void *from)
*** 4958,4963 ****
--- 4973,4981 ----
case T_PlaceHolderVar:
retval = _copyPlaceHolderVar(from);
break;
+ case T_GroupedVar:
+ retval = _copyGroupedVar(from);
+ break;
case T_SpecialJoinInfo:
retval = _copySpecialJoinInfo(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
new file mode 100644
index 5941b7a..4fe3aa8
*** a/src/backend/nodes/equalfuncs.c
--- b/src/backend/nodes/equalfuncs.c
*************** _equalPlaceHolderVar(const PlaceHolderVa
*** 865,870 ****
--- 865,878 ----
}
static bool
+ _equalGroupedVar(const GroupedVar *a, const GroupedVar *b)
+ {
+ COMPARE_SCALAR_FIELD(gvid);
+
+ return true;
+ }
+
+ static bool
_equalSpecialJoinInfo(const SpecialJoinInfo *a, const SpecialJoinInfo *b)
{
COMPARE_BITMAPSET_FIELD(min_lefthand);
*************** equal(const void *a, const void *b)
*** 3130,3135 ****
--- 3138,3146 ----
case T_PlaceHolderVar:
retval = _equalPlaceHolderVar(a, b);
break;
+ case T_GroupedVar:
+ retval = _equalGroupedVar(a, b);
+ break;
case T_SpecialJoinInfo:
retval = _equalSpecialJoinInfo(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
new file mode 100644
index d5293a1..da517ce
*** a/src/backend/nodes/nodeFuncs.c
--- b/src/backend/nodes/nodeFuncs.c
*************** exprType(const Node *expr)
*** 256,261 ****
--- 256,264 ----
case T_PlaceHolderVar:
type = exprType((Node *) ((const PlaceHolderVar *) expr)->phexpr);
break;
+ case T_GroupedVar:
+ type = exprType((Node *) ((const GroupedVar *) expr)->agg_partial);
+ break;
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(expr));
type = InvalidOid; /* keep compiler quiet */
*************** exprCollation(const Node *expr)
*** 925,930 ****
--- 928,936 ----
case T_PlaceHolderVar:
coll = exprCollation((Node *) ((const PlaceHolderVar *) expr)->phexpr);
break;
+ case T_GroupedVar:
+ coll = exprCollation((Node *) ((const GroupedVar *) expr)->gvexpr);
+ break;
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(expr));
coll = InvalidOid; /* keep compiler quiet */
*************** expression_tree_walker(Node *node,
*** 2188,2193 ****
--- 2194,2201 ----
break;
case T_PlaceHolderVar:
return walker(((PlaceHolderVar *) node)->phexpr, context);
+ case T_GroupedVar:
+ return walker(((GroupedVar *) node)->gvexpr, context);
case T_InferenceElem:
return walker(((InferenceElem *) node)->expr, context);
case T_AppendRelInfo:
*************** expression_tree_mutator(Node *node,
*** 2978,2983 ****
--- 2986,3001 ----
return (Node *) newnode;
}
break;
+ case T_GroupedVar:
+ {
+ GroupedVar *gv = (GroupedVar *) node;
+ GroupedVar *newnode;
+
+ FLATCOPY(newnode, gv, GroupedVar);
+ MUTATE(newnode->gvexpr, gv->gvexpr, Expr *);
+ MUTATE(newnode->agg_partial, gv->agg_partial, Aggref *);
+ return (Node *) newnode;
+ }
case T_InferenceElem:
{
InferenceElem *inferenceelemdexpr = (InferenceElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
new file mode 100644
index 766ca49..81da091
*** a/src/backend/nodes/outfuncs.c
--- b/src/backend/nodes/outfuncs.c
*************** _outPlannerInfo(StringInfo str, const Pl
*** 2181,2186 ****
--- 2181,2187 ----
WRITE_NODE_FIELD(pcinfo_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
*************** _outParamPathInfo(StringInfo str, const
*** 2401,2406 ****
--- 2402,2417 ----
}
static void
+ _outGroupedPathInfo(StringInfo str, const GroupedPathInfo *node)
+ {
+ WRITE_NODE_TYPE("GROUPEDPATHINFO");
+
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(pathlist);
+ WRITE_NODE_FIELD(partial_pathlist);
+ }
+
+ static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
WRITE_NODE_TYPE("RESTRICTINFO");
*************** _outPlaceHolderVar(StringInfo str, const
*** 2444,2449 ****
--- 2455,2470 ----
}
static void
+ _outGroupedVar(StringInfo str, const GroupedVar *node)
+ {
+ WRITE_NODE_TYPE("GROUPEDVAR");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(gvid);
+ }
+
+ static void
_outSpecialJoinInfo(StringInfo str, const SpecialJoinInfo *node)
{
WRITE_NODE_TYPE("SPECIALJOININFO");
*************** outNode(StringInfo str, const void *obj)
*** 3980,3991 ****
--- 4001,4018 ----
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_GroupedPathInfo:
+ _outGroupedPathInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
case T_PlaceHolderVar:
_outPlaceHolderVar(str, obj);
break;
+ case T_GroupedVar:
+ _outGroupedVar(str, obj);
+ break;
case T_SpecialJoinInfo:
_outSpecialJoinInfo(str, obj);
break;
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
new file mode 100644
index 766f2d8..fdf4edd
*** a/src/backend/nodes/readfuncs.c
--- b/src/backend/nodes/readfuncs.c
*************** _readVar(void)
*** 521,526 ****
--- 521,541 ----
}
/*
+ * _readGroupedVar
+ */
+ static GroupedVar *
+ _readGroupedVar(void)
+ {
+ READ_LOCALS(GroupedVar);
+
+ READ_NODE_FIELD(gvexpr);
+ READ_NODE_FIELD(agg_partial);
+ READ_UINT_FIELD(gvid);
+
+ READ_DONE();
+ }
+
+ /*
* _readConst
*/
static Const *
*************** parseNodeString(void)
*** 2436,2441 ****
--- 2451,2458 ----
return_value = _readTableFunc();
else if (MATCH("VAR", 3))
return_value = _readVar();
+ else if (MATCH("GROUPEDVAR", 10))
+ return_value = _readGroupedVar();
else if (MATCH("CONST", 5))
return_value = _readConst();
else if (MATCH("PARAM", 5))
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
new file mode 100644
index b5cab0c..f89406d
*** a/src/backend/optimizer/geqo/geqo_eval.c
--- b/src/backend/optimizer/geqo/geqo_eval.c
*************** merge_clump(PlannerInfo *root, List *clu
*** 265,271 ****
if (joinrel)
{
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, joinrel);
/* Find and save the cheapest paths for this joinrel */
set_cheapest(joinrel);
--- 265,271 ----
if (joinrel)
{
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, joinrel, false);
/* Find and save the cheapest paths for this joinrel */
set_cheapest(joinrel);
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
new file mode 100644
index 343b35a..fca4727
*** a/src/backend/optimizer/path/allpaths.c
--- b/src/backend/optimizer/path/allpaths.c
*************** set_rel_pathlist(PlannerInfo *root, RelO
*** 486,492 ****
* we'll consider gathering partial paths for the parent appendrel.)
*/
if (rel->reloptkind == RELOPT_BASEREL)
! generate_gather_paths(root, rel);
/*
* Allow a plugin to editorialize on the set of Paths for this base
--- 486,495 ----
* we'll consider gathering partial paths for the parent appendrel.)
*/
if (rel->reloptkind == RELOPT_BASEREL)
! {
! generate_gather_paths(root, rel, false);
! generate_gather_paths(root, rel, true);
! }
/*
* Allow a plugin to editorialize on the set of Paths for this base
*************** static void
*** 687,692 ****
--- 690,696 ----
set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
{
Relids required_outer;
+ Path *seq_path;
/*
* We don't support pushing join clauses into the quals of a seqscan, but
*************** set_plain_rel_pathlist(PlannerInfo *root
*** 695,709 ****
*/
required_outer = rel->lateral_relids;
! /* Consider sequential scan */
! add_path(rel, create_seqscan_path(root, rel, required_outer, 0));
! /* If appropriate, consider parallel sequential scan */
if (rel->consider_parallel && required_outer == NULL)
create_plain_partial_paths(root, rel);
/* Consider index scans */
! create_index_paths(root, rel);
/* Consider TID scans */
create_tidscan_paths(root, rel);
--- 699,726 ----
*/
required_outer = rel->lateral_relids;
! /* Consider sequential scan, both plain and grouped. */
! seq_path = create_seqscan_path(root, rel, required_outer, 0);
! add_path(rel, seq_path, false);
! if (rel->gpi != NULL && required_outer == NULL)
! create_grouped_path(root, rel, seq_path, false, false, AGG_HASHED);
! /* If appropriate, consider parallel sequential scan (plain or grouped) */
if (rel->consider_parallel && required_outer == NULL)
create_plain_partial_paths(root, rel);
/* Consider index scans */
! create_index_paths(root, rel, false);
! if (rel->gpi != NULL)
! {
! /*
! * TODO Instead of calling the whole clause-matching machinery twice
! * (there should be no difference between plain and grouped paths from
! * this point of view), consider returning a separate list of paths
! * usable as grouped ones.
! */
! create_index_paths(root, rel, true);
! }
/* Consider TID scans */
create_tidscan_paths(root, rel);
*************** static void
*** 717,722 ****
--- 734,740 ----
create_plain_partial_paths(PlannerInfo *root, RelOptInfo *rel)
{
int parallel_workers;
+ Path *path;
parallel_workers = compute_parallel_worker(rel, rel->pages, -1);
*************** create_plain_partial_paths(PlannerInfo *
*** 725,731 ****
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! add_partial_path(rel, create_seqscan_path(root, rel, NULL, parallel_workers));
}
/*
--- 743,850 ----
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! path = create_seqscan_path(root, rel, NULL, parallel_workers);
! add_partial_path(rel, path, false);
!
! /*
! * Do partial aggregation at base relation level if the relation is
! * eligible for it.
! */
! if (rel->gpi != NULL)
! create_grouped_path(root, rel, path, false, true, AGG_HASHED);
! }
!
! /*
! * Apply partial aggregation to a subpath and add the AggPath to the
! * appropriate pathlist.
! *
! * "precheck" tells whether the aggregation path should first be checked using
! * add_path_precheck().
! *
! * If "partial" is true, the resulting path is considered partial in terms of
! * parallel execution.
! *
! * The path we create here shouldn't be parameterized because of supposedly
! * high startup cost of aggregation (whether due to build of hash table for
! * AGG_HASHED strategy or due to explicit sort for AGG_SORTED).
! *
! * XXX IndexPath as an input for AGG_SORTED might seem to be an exception, but
! * aggregation of its output is only beneficial if it's performed by multiple
! * workers, i.e. the resulting path is partial (Besides parallel aggregation,
! * the other use case of aggregation push-down is aggregation performed on
! * remote database, but that has nothing to do with IndexScan). And partial
! * path cannot be parameterized because it's semantically wrong to use it on
! * the inner side of NL join.
! */
! void
! create_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
! bool precheck, bool partial, AggStrategy aggstrategy)
! {
! List *group_clauses = NIL;
! List *group_exprs = NIL;
! List *agg_exprs = NIL;
! Path *agg_path;
!
! /*
! * If the AggPath should be partial, the subpath must be too, and
! * therefore the subpath is essentially parallel_safe.
! */
! Assert(subpath->parallel_safe || !partial);
!
! /*
! * Grouped path should never be parameterized, so we're not supposed to
! * receive parameterized subpath.
! */
! Assert(subpath->param_info == NULL);
!
! /*
! * Note that "partial" in the following function names refers to 2-stage
! * aggregation, not to parallel processing.
! */
! if (aggstrategy == AGG_HASHED)
! agg_path = (Path *) create_partial_agg_hashed_path(root, subpath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! subpath->rows);
! else if (aggstrategy == AGG_SORTED)
! agg_path = (Path *) create_partial_agg_sorted_path(root, subpath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! subpath->rows);
! else
! elog(ERROR, "unexpected strategy %d", aggstrategy);
!
! /* Add the grouped path to the list of grouped base paths. */
! if (agg_path != NULL)
! {
! if (precheck)
! {
! List *pathkeys;
!
! /* AGG_HASH is not supposed to generate sorted output. */
! pathkeys = aggstrategy == AGG_SORTED ? subpath->pathkeys : NIL;
!
! if (!partial &&
! !add_path_precheck(rel, agg_path->startup_cost,
! agg_path->total_cost, pathkeys, NULL,
! true))
! return;
!
! if (partial &&
! !add_partial_path_precheck(rel, agg_path->total_cost, pathkeys,
! true))
! return;
! }
!
! if (!partial)
! add_path(rel, (Path *) agg_path, true);
! else
! add_partial_path(rel, (Path *) agg_path, true);
! }
}
/*
*************** set_tablesample_rel_pathlist(PlannerInfo
*** 811,817 ****
path = (Path *) create_material_path(rel, path);
}
! add_path(rel, path);
/* For the moment, at least, there are no other paths to consider */
}
--- 930,936 ----
path = (Path *) create_material_path(rel, path);
}
! add_path(rel, path, false);
/* For the moment, at least, there are no other paths to consider */
}
*************** set_append_rel_size(PlannerInfo *root, R
*** 1066,1071 ****
--- 1185,1233 ----
appinfo);
/*
+ * If grouping is applicable to the parent relation, it should be
+ * applicable to the children too. Make sure the child rel has valid
+ * sortgrouprefs.
+ *
+ * TODO Consider if this is really needed --- child rel is not joined
+ * to grouped rel itself, so it might not participate on creation of
+ * the grouped path target that upper joins will see.
+ */
+ if (rel->reltarget->sortgrouprefs)
+ {
+ Assert(childrel->reltarget->sortgrouprefs == NULL);
+ childrel->reltarget->sortgrouprefs = rel->reltarget->sortgrouprefs;
+ }
+
+ /*
+ * Also the grouped target needs to be adjusted, if one exists.
+ */
+ if (rel->gpi != NULL)
+ {
+ PathTarget *target = rel->gpi->target;
+
+ Assert(target->sortgrouprefs != NULL);
+
+ Assert(childrel->gpi == NULL);
+ childrel->gpi = makeNode(GroupedPathInfo);
+ memcpy(childrel->gpi, rel->gpi, sizeof(GroupedPathInfo));
+
+ /*
+ * add_grouping_info_to_base_rels was not sure if grouping makes
+ * sense for the parent rel, so create a separate copy of the
+ * target now.
+ */
+ childrel->gpi->target = copy_pathtarget(childrel->gpi->target);
+
+ /* Translate vars of the grouping target. */
+ Assert(childrel->gpi->target->exprs != NIL);
+ childrel->gpi->target->exprs = (List *)
+ adjust_appendrel_attrs(root,
+ (Node *) childrel->gpi->target->exprs,
+ appinfo);
+ }
+
+ /*
* We have to make child entries in the EquivalenceClass data
* structures as well. This is needed either if the parent
* participates in some eclass joins (because we will want to consider
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1280,1285 ****
--- 1442,1449 ----
bool subpaths_valid = true;
List *partial_subpaths = NIL;
bool partial_subpaths_valid = true;
+ List *grouped_subpaths = NIL;
+ bool grouped_subpaths_valid = true;
List *all_child_pathkeys = NIL;
List *all_child_outers = NIL;
ListCell *l;
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1323,1328 ****
--- 1487,1515 ----
partial_subpaths_valid = false;
/*
+ * For grouped paths, use only the unparameterized subpaths.
+ *
+ * XXX Consider if the parameterized subpaths should be processed
+ * below. It's probably not useful for sequential scans (due to
+ * repeated aggregation), but might be worthwhile for other child
+ * nodes.
+ */
+ if (childrel->gpi != NULL && childrel->gpi->pathlist != NIL)
+ {
+ Path *path;
+
+ path = (Path *) linitial(childrel->gpi->pathlist);
+ if (path->param_info == NULL)
+ grouped_subpaths = accumulate_append_subpath(grouped_subpaths,
+ path);
+ else
+ grouped_subpaths_valid = false;
+ }
+ else
+ grouped_subpaths_valid = false;
+
+
+ /*
* Collect lists of all the available path orderings and
* parameterizations for all the children. We use these as a
* heuristic to indicate which sort orderings and parameterizations we
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1394,1400 ****
*/
if (subpaths_valid)
add_path(rel, (Path *) create_append_path(rel, subpaths, NULL, 0,
! partitioned_rels));
/*
* Consider an append of partial unordered, unparameterized partial paths.
--- 1581,1588 ----
*/
if (subpaths_valid)
add_path(rel, (Path *) create_append_path(rel, subpaths, NULL, 0,
! partitioned_rels),
! false);
/*
* Consider an append of partial unordered, unparameterized partial paths.
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1421,1428 ****
/* Generate a partial append path. */
appendpath = create_append_path(rel, partial_subpaths, NULL,
! parallel_workers, partitioned_rels);
! add_partial_path(rel, (Path *) appendpath);
}
/*
--- 1609,1629 ----
/* Generate a partial append path. */
appendpath = create_append_path(rel, partial_subpaths, NULL,
! parallel_workers,
! partitioned_rels);
! add_partial_path(rel, (Path *) appendpath, false);
! }
!
! /* TODO Also partial grouped paths? */
! if (grouped_subpaths_valid)
! {
! Path *path;
!
! path = (Path *) create_append_path(rel, grouped_subpaths, NULL, 0,
! partitioned_rels);
! /* pathtarget will produce the grouped relation.. */
! path->pathtarget = rel->gpi->target;
! add_path(rel, path, true);
}
/*
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1475,1481 ****
if (subpaths_valid)
add_path(rel, (Path *)
create_append_path(rel, subpaths, required_outer, 0,
! partitioned_rels));
}
}
--- 1676,1683 ----
if (subpaths_valid)
add_path(rel, (Path *)
create_append_path(rel, subpaths, required_outer, 0,
! partitioned_rels),
! false);
}
}
*************** generate_mergeappend_paths(PlannerInfo *
*** 1571,1584 ****
startup_subpaths,
pathkeys,
NULL,
! partitioned_rels));
if (startup_neq_total)
add_path(rel, (Path *) create_merge_append_path(root,
rel,
total_subpaths,
pathkeys,
NULL,
! partitioned_rels));
}
}
--- 1773,1788 ----
startup_subpaths,
pathkeys,
NULL,
! partitioned_rels),
! false);
if (startup_neq_total)
add_path(rel, (Path *) create_merge_append_path(root,
rel,
total_subpaths,
pathkeys,
NULL,
! partitioned_rels),
! false);
}
}
*************** set_dummy_rel_pathlist(RelOptInfo *rel)
*** 1711,1717 ****
rel->pathlist = NIL;
rel->partial_pathlist = NIL;
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL));
/*
* We set the cheapest path immediately, to ensure that IS_DUMMY_REL()
--- 1915,1921 ----
rel->pathlist = NIL;
rel->partial_pathlist = NIL;
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL), false);
/*
* We set the cheapest path immediately, to ensure that IS_DUMMY_REL()
*************** set_subquery_pathlist(PlannerInfo *root,
*** 1925,1931 ****
/* Generate outer path using this subpath */
add_path(rel, (Path *)
create_subqueryscan_path(root, rel, subpath,
! pathkeys, required_outer));
}
}
--- 2129,2135 ----
/* Generate outer path using this subpath */
add_path(rel, (Path *)
create_subqueryscan_path(root, rel, subpath,
! pathkeys, required_outer), false);
}
}
*************** set_function_pathlist(PlannerInfo *root,
*** 1994,2000 ****
/* Generate appropriate path */
add_path(rel, create_functionscan_path(root, rel,
! pathkeys, required_outer));
}
/*
--- 2198,2204 ----
/* Generate appropriate path */
add_path(rel, create_functionscan_path(root, rel,
! pathkeys, required_outer), false);
}
/*
*************** set_values_pathlist(PlannerInfo *root, R
*** 2014,2020 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_valuesscan_path(root, rel, required_outer));
}
/*
--- 2218,2224 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_valuesscan_path(root, rel, required_outer), false);
}
/*
*************** set_tablefunc_pathlist(PlannerInfo *root
*** 2035,2041 ****
/* Generate appropriate path */
add_path(rel, create_tablefuncscan_path(root, rel,
! required_outer));
}
/*
--- 2239,2245 ----
/* Generate appropriate path */
add_path(rel, create_tablefuncscan_path(root, rel,
! required_outer), false);
}
/*
*************** set_cte_pathlist(PlannerInfo *root, RelO
*** 2101,2107 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_ctescan_path(root, rel, required_outer));
}
/*
--- 2305,2311 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_ctescan_path(root, rel, required_outer), false);
}
/*
*************** set_namedtuplestore_pathlist(PlannerInfo
*** 2128,2134 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_namedtuplestorescan_path(root, rel, required_outer));
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(rel);
--- 2332,2339 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_namedtuplestorescan_path(root, rel, required_outer),
! false);
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(rel);
*************** set_worktable_pathlist(PlannerInfo *root
*** 2181,2187 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_worktablescan_path(root, rel, required_outer));
}
/*
--- 2386,2393 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_worktablescan_path(root, rel, required_outer),
! false);
}
/*
*************** set_worktable_pathlist(PlannerInfo *root
*** 2194,2207 ****
* path that some GatherPath or GatherMergePath has a reference to.)
*/
void
! generate_gather_paths(PlannerInfo *root, RelOptInfo *rel)
{
Path *cheapest_partial_path;
Path *simple_gather_path;
ListCell *lc;
/* If there are no partial paths, there's nothing to do here. */
! if (rel->partial_pathlist == NIL)
return;
/*
--- 2400,2420 ----
* path that some GatherPath or GatherMergePath has a reference to.)
*/
void
! generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool grouped)
{
Path *cheapest_partial_path;
Path *simple_gather_path;
+ List *pathlist = NIL;
+ PathTarget *partial_target;
ListCell *lc;
+ if (!grouped)
+ pathlist = rel->partial_pathlist;
+ else if (rel->gpi != NULL)
+ pathlist = rel->gpi->partial_pathlist;
+
/* If there are no partial paths, there's nothing to do here. */
! if (pathlist == NIL)
return;
/*
*************** generate_gather_paths(PlannerInfo *root,
*** 2209,2225 ****
* path of interest: the cheapest one. That will be the one at the front
* of partial_pathlist because of the way add_partial_path works.
*/
! cheapest_partial_path = linitial(rel->partial_pathlist);
simple_gather_path = (Path *)
! create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
NULL, NULL);
! add_path(rel, simple_gather_path);
/*
* For each useful ordering, we can consider an order-preserving Gather
* Merge.
*/
! foreach (lc, rel->partial_pathlist)
{
Path *subpath = (Path *) lfirst(lc);
GatherMergePath *path;
--- 2422,2444 ----
* path of interest: the cheapest one. That will be the one at the front
* of partial_pathlist because of the way add_partial_path works.
*/
! cheapest_partial_path = linitial(pathlist);
!
! if (!grouped)
! partial_target = rel->reltarget;
! else if (rel->gpi != NULL)
! partial_target = rel->gpi->target;
!
simple_gather_path = (Path *)
! create_gather_path(root, rel, cheapest_partial_path, partial_target,
NULL, NULL);
! add_path(rel, simple_gather_path, grouped);
/*
* For each useful ordering, we can consider an order-preserving Gather
* Merge.
*/
! foreach (lc, pathlist)
{
Path *subpath = (Path *) lfirst(lc);
GatherMergePath *path;
*************** generate_gather_paths(PlannerInfo *root,
*** 2227,2235 ****
if (subpath->pathkeys == NIL)
continue;
! path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
subpath->pathkeys, NULL, NULL);
! add_path(rel, &path->path);
}
}
--- 2446,2454 ----
if (subpath->pathkeys == NIL)
continue;
! path = create_gather_merge_path(root, rel, subpath, partial_target,
subpath->pathkeys, NULL, NULL);
! add_path(rel, &path->path, grouped);
}
}
*************** standard_join_search(PlannerInfo *root,
*** 2395,2401 ****
rel = (RelOptInfo *) lfirst(lc);
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, rel);
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
--- 2614,2621 ----
rel = (RelOptInfo *) lfirst(lc);
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, rel, false);
! generate_gather_paths(root, rel, true);
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
*************** create_partial_bitmap_paths(PlannerInfo
*** 3046,3052 ****
return;
add_partial_path(rel, (Path *) create_bitmap_heap_path(root, rel,
! bitmapqual, rel->lateral_relids, 1.0, parallel_workers));
}
/*
--- 3266,3272 ----
return;
add_partial_path(rel, (Path *) create_bitmap_heap_path(root, rel,
! bitmapqual, rel->lateral_relids, 1.0, parallel_workers), false);
}
/*
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
new file mode 100644
index a5d19f9..89f4308
*** a/src/backend/optimizer/path/indxpath.c
--- b/src/backend/optimizer/path/indxpath.c
***************
*** 32,37 ****
--- 32,38 ----
#include "optimizer/predtest.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "utils/builtins.h"
#include "utils/bytea.h"
*************** static bool eclass_already_used(Equivale
*** 107,119 ****
static bool bms_equal_any(Relids relids, List *relids_list);
static void get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths);
static List *build_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop);
static List *build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
List *clauses, List *other_clauses);
static List *generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
--- 108,121 ----
static bool bms_equal_any(Relids relids, List *relids_list);
static void get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths, bool grouped);
static List *build_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop,
! bool grouped);
static List *build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
List *clauses, List *other_clauses);
static List *generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
*************** static Const *string_to_const(const char
*** 229,235 ****
* as meaning "unparameterized so far as the indexquals are concerned".
*/
void
! create_index_paths(PlannerInfo *root, RelOptInfo *rel)
{
List *indexpaths;
List *bitindexpaths;
--- 231,237 ----
* as meaning "unparameterized so far as the indexquals are concerned".
*/
void
! create_index_paths(PlannerInfo *root, RelOptInfo *rel, bool grouped)
{
List *indexpaths;
List *bitindexpaths;
*************** create_index_paths(PlannerInfo *root, Re
*** 274,281 ****
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
! get_index_paths(root, rel, index, &rclauseset,
! &bitindexpaths);
/*
* Identify the join clauses that can match the index. For the moment
--- 276,283 ----
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
! get_index_paths(root, rel, index, &rclauseset, &bitindexpaths,
! grouped);
/*
* Identify the join clauses that can match the index. For the moment
*************** create_index_paths(PlannerInfo *root, Re
*** 338,344 ****
bitmapqual = choose_bitmap_and(root, rel, bitindexpaths);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
rel->lateral_relids, 1.0, 0);
! add_path(rel, (Path *) bpath);
/* create a partial bitmap heap path */
if (rel->consider_parallel && rel->lateral_relids == NULL)
--- 340,346 ----
bitmapqual = choose_bitmap_and(root, rel, bitindexpaths);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
rel->lateral_relids, 1.0, 0);
! add_path(rel, (Path *) bpath, false);
/* create a partial bitmap heap path */
if (rel->consider_parallel && rel->lateral_relids == NULL)
*************** create_index_paths(PlannerInfo *root, Re
*** 415,421 ****
loop_count = get_loop_count(root, rel->relid, required_outer);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
required_outer, loop_count, 0);
! add_path(rel, (Path *) bpath);
}
}
}
--- 417,423 ----
loop_count = get_loop_count(root, rel->relid, required_outer);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
required_outer, loop_count, 0);
! add_path(rel, (Path *) bpath, false);
}
}
}
*************** get_join_index_paths(PlannerInfo *root,
*** 667,673 ****
Assert(clauseset.nonempty);
/* Build index path(s) using the collected set of clauses */
! get_index_paths(root, rel, index, &clauseset, bitindexpaths);
/*
* Remember we considered paths for this set of relids. We use lcons not
--- 669,675 ----
Assert(clauseset.nonempty);
/* Build index path(s) using the collected set of clauses */
! get_index_paths(root, rel, index, &clauseset, bitindexpaths, false);
/*
* Remember we considered paths for this set of relids. We use lcons not
*************** bms_equal_any(Relids relids, List *relid
*** 736,742 ****
static void
get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths)
{
List *indexpaths;
bool skip_nonnative_saop = false;
--- 738,744 ----
static void
get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths, bool grouped)
{
List *indexpaths;
bool skip_nonnative_saop = false;
*************** get_index_paths(PlannerInfo *root, RelOp
*** 754,760 ****
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! &skip_lower_saop);
/*
* If we skipped any lower-order ScalarArrayOpExprs on an index with an AM
--- 756,762 ----
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! &skip_lower_saop, grouped);
/*
* If we skipped any lower-order ScalarArrayOpExprs on an index with an AM
*************** get_index_paths(PlannerInfo *root, RelOp
*** 769,775 ****
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! NULL));
}
/*
--- 771,777 ----
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! NULL, grouped));
}
/*
*************** get_index_paths(PlannerInfo *root, RelOp
*** 789,797 ****
IndexPath *ipath = (IndexPath *) lfirst(lc);
if (index->amhasgettuple)
! add_path(rel, (Path *) ipath);
! if (index->amhasgetbitmap &&
(ipath->path.pathkeys == NIL ||
ipath->indexselectivity < 1.0))
*bitindexpaths = lappend(*bitindexpaths, ipath);
--- 791,799 ----
IndexPath *ipath = (IndexPath *) lfirst(lc);
if (index->amhasgettuple)
! add_path(rel, (Path *) ipath, grouped);
! if (!grouped && index->amhasgetbitmap &&
(ipath->path.pathkeys == NIL ||
ipath->indexselectivity < 1.0))
*bitindexpaths = lappend(*bitindexpaths, ipath);
*************** get_index_paths(PlannerInfo *root, RelOp
*** 802,815 ****
* natively, generate bitmap scan paths relying on executor-managed
* ScalarArrayOpExpr.
*/
! if (skip_nonnative_saop)
{
indexpaths = build_index_paths(root, rel,
index, clauses,
false,
ST_BITMAPSCAN,
NULL,
! NULL);
*bitindexpaths = list_concat(*bitindexpaths, indexpaths);
}
}
--- 804,818 ----
* natively, generate bitmap scan paths relying on executor-managed
* ScalarArrayOpExpr.
*/
! if (!grouped && skip_nonnative_saop)
{
indexpaths = build_index_paths(root, rel,
index, clauses,
false,
ST_BITMAPSCAN,
NULL,
! NULL,
! false);
*bitindexpaths = list_concat(*bitindexpaths, indexpaths);
}
}
*************** build_index_paths(PlannerInfo *root, Rel
*** 861,867 ****
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop)
{
List *result = NIL;
IndexPath *ipath;
--- 864,870 ----
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop, bool grouped)
{
List *result = NIL;
IndexPath *ipath;
*************** build_index_paths(PlannerInfo *root, Rel
*** 878,883 ****
--- 881,890 ----
bool index_is_ordered;
bool index_only_scan;
int indexcol;
+ bool can_agg_sorted;
+ List *group_clauses, *group_exprs, *agg_exprs;
+ AggPath *agg_path;
+ double agg_input_rows;
/*
* Check that index supports the desired scan type(s)
*************** build_index_paths(PlannerInfo *root, Rel
*** 891,896 ****
--- 898,906 ----
case ST_BITMAPSCAN:
if (!index->amhasgetbitmap)
return NIL;
+
+ if (grouped)
+ return NIL;
break;
case ST_ANYSCAN:
/* either or both are OK */
*************** build_index_paths(PlannerInfo *root, Rel
*** 1032,1037 ****
--- 1042,1051 ----
* later merging or final output ordering, OR the index has a useful
* predicate, OR an index-only scan is possible.
*/
+ can_agg_sorted = true;
+ group_clauses = NIL;
+ group_exprs = NIL;
+ agg_exprs = NIL;
if (index_clauses != NIL || useful_pathkeys != NIL || useful_predicate ||
index_only_scan)
{
*************** build_index_paths(PlannerInfo *root, Rel
*** 1048,1054 ****
outer_relids,
loop_count,
false);
! result = lappend(result, ipath);
/*
* If appropriate, consider parallel index scan. We don't allow
--- 1062,1086 ----
outer_relids,
loop_count,
false);
! if (!grouped)
! result = lappend(result, ipath);
! else
! {
! /* TODO Double-check if this is the correct input value. */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root, (Path *) ipath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
!
! if (agg_path != NULL)
! result = lappend(result, agg_path);
! else
! can_agg_sorted = false;
! }
/*
* If appropriate, consider parallel index scan. We don't allow
*************** build_index_paths(PlannerInfo *root, Rel
*** 1077,1083 ****
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath);
else
pfree(ipath);
}
--- 1109,1139 ----
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! {
! if (!grouped)
! add_partial_path(rel, (Path *) ipath, grouped);
! else if (can_agg_sorted && outer_relids == NULL)
! {
! /* TODO Double-check if this is the correct input value. */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! false,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
!
! /*
! * If create_agg_sorted_path succeeded once, it should
! * always do.
! */
! Assert(agg_path != NULL);
!
! add_partial_path(rel, (Path *) agg_path, grouped);
! }
! }
else
pfree(ipath);
}
*************** build_index_paths(PlannerInfo *root, Rel
*** 1105,1111 ****
outer_relids,
loop_count,
false);
! result = lappend(result, ipath);
/* If appropriate, consider parallel index scan */
if (index->amcanparallel &&
--- 1161,1185 ----
outer_relids,
loop_count,
false);
!
! if (!grouped)
! result = lappend(result, ipath);
! else if (can_agg_sorted)
! {
! /* TODO Double-check if this is the correct input value. */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
!
! Assert(agg_path != NULL);
! result = lappend(result, agg_path);
! }
/* If appropriate, consider parallel index scan */
if (index->amcanparallel &&
*************** build_index_paths(PlannerInfo *root, Rel
*** 1129,1135 ****
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath);
else
pfree(ipath);
}
--- 1203,1227 ----
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! {
! if (!grouped)
! add_partial_path(rel, (Path *) ipath, grouped);
! else if (can_agg_sorted && outer_relids == NULL)
! {
! /* TODO Double-check if this is the correct input value. */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! false,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
! Assert(agg_path != NULL);
! add_partial_path(rel, (Path *) agg_path, grouped);
! }
! }
else
pfree(ipath);
}
*************** build_paths_for_OR(PlannerInfo *root, Re
*** 1244,1250 ****
useful_predicate,
ST_BITMAPSCAN,
NULL,
! NULL);
result = list_concat(result, indexpaths);
}
--- 1336,1343 ----
useful_predicate,
ST_BITMAPSCAN,
NULL,
! NULL,
! false);
result = list_concat(result, indexpaths);
}
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
new file mode 100644
index de7044d..212b40c
*** a/src/backend/optimizer/path/joinpath.c
--- b/src/backend/optimizer/path/joinpath.c
***************
*** 21,26 ****
--- 21,27 ----
#include "optimizer/cost.h"
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
+ #include "optimizer/tlist.h"
/* Hook for plugins to get control in add_paths_to_joinrel() */
set_join_pathlist_hook_type set_join_pathlist_hook = NULL;
*************** static void try_partial_mergejoin_path(P
*** 37,65 ****
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra);
static void sort_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra);
static void match_unsorted_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra);
static void consider_parallel_nestloop(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra);
static void consider_parallel_mergejoin(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total);
static void hash_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra);
static List *select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
--- 38,86 ----
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate);
static void sort_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra,
! bool grouped);
! static void sort_inner_and_outer_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! RelOptInfo *outerrel,
! RelOptInfo *innerrel,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate);
static void match_unsorted_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra,
! bool grouped);
static void consider_parallel_nestloop(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped, bool do_aggregate);
static void consider_parallel_mergejoin(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total,
! bool grouped);
static void hash_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra,
! bool grouped);
! static bool is_grouped_join_target_complete(PlannerInfo *root,
! PathTarget *jointarget,
! Path *outer_path,
! Path *inner_path);
static List *select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
*************** static void generate_mergejoin_paths(Pla
*** 76,82 ****
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial);
/*
--- 97,106 ----
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate);
/*
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 197,204 ****
* sorted. Skip this if we can't mergejoin.
*/
if (mergejoin_allowed)
sort_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra);
/*
* 2. Consider paths where the outer relation need not be explicitly
--- 221,232 ----
* sorted. Skip this if we can't mergejoin.
*/
if (mergejoin_allowed)
+ {
sort_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, false);
! sort_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, true);
! }
/*
* 2. Consider paths where the outer relation need not be explicitly
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 208,215 ****
* joins at all, so it wouldn't work in the prohibited cases either.)
*/
if (mergejoin_allowed)
match_unsorted_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra);
#ifdef NOT_USED
--- 236,247 ----
* joins at all, so it wouldn't work in the prohibited cases either.)
*/
if (mergejoin_allowed)
+ {
match_unsorted_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, false);
! match_unsorted_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, true);
! }
#ifdef NOT_USED
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 235,242 ****
* joins, because there may be no other alternative.
*/
if (enable_hashjoin || jointype == JOIN_FULL)
hash_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra);
/*
* 5. If inner and outer relations are foreign tables (or joins) belonging
--- 267,278 ----
* joins, because there may be no other alternative.
*/
if (enable_hashjoin || jointype == JOIN_FULL)
+ {
hash_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, false);
! hash_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, true);
! }
/*
* 5. If inner and outer relations are foreign tables (or joins) belonging
*************** try_nestloop_path(PlannerInfo *root,
*** 300,309 ****
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
Relids required_outer;
JoinCostWorkspace workspace;
/*
* Check to see if proposed path is still parameterized, and reject if the
--- 336,355 ----
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* Caller should not request aggregation w/o grouped output. */
+ Assert(!do_aggregate || grouped);
+
+ /* GroupedPathInfo is necessary for us to produce a grouped set. */
+ Assert(joinrel->gpi != NULL || !grouped);
/*
* Check to see if proposed path is still parameterized, and reject if the
*************** try_nestloop_path(PlannerInfo *root,
*** 311,329 ****
* says to allow it anyway. Also, we must reject if have_dangerous_phv
* doesn't like the look of it, which could only happen if the nestloop is
* still parameterized.
*/
! required_outer = calc_nestloop_required_outer(outer_path,
! inner_path);
! if (required_outer &&
! ((!bms_overlap(required_outer, extra->param_source_rels) &&
! !allow_star_schema_join(root, outer_path, inner_path)) ||
! have_dangerous_phv(root,
! outer_path->parent->relids,
! PATH_REQ_OUTER(inner_path))))
{
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
}
/*
--- 357,379 ----
* says to allow it anyway. Also, we must reject if have_dangerous_phv
* doesn't like the look of it, which could only happen if the nestloop is
* still parameterized.
+ *
+ * Grouped path should never be parameterized.
*/
! required_outer = calc_nestloop_required_outer(outer_path, inner_path);
! if (required_outer)
{
! if (grouped ||
! (!bms_overlap(required_outer, extra->param_source_rels) &&
! !allow_star_schema_join(root, outer_path, inner_path)) ||
! have_dangerous_phv(root,
! outer_path->parent->relids,
! PATH_REQ_OUTER(inner_path)))
! {
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
! }
}
/*
*************** try_nestloop_path(PlannerInfo *root,
*** 339,360 ****
outer_path, inner_path,
extra->sjinfo, &extra->semifactors);
! if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer))
{
! add_path(joinrel, (Path *)
! create_nestloop_path(root,
! joinrel,
! jointype,
! &workspace,
! extra->sjinfo,
! &extra->semifactors,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer));
}
else
{
--- 389,425 ----
outer_path, inner_path,
extra->sjinfo, &extra->semifactors);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! join_target = joinrel->gpi->target;
!
! join_path = (Path *) create_nestloop_path(root, joinrel, jointype,
! &workspace, extra->sjinfo,
! &extra->semifactors,
! outer_path, inner_path,
! extra->restrictlist, pathkeys,
! required_outer, join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate && required_outer == NULL)
! {
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_SORTED);
! }
! else if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, grouped))
{
! add_path(joinrel, join_path, grouped);
}
else
{
*************** try_partial_nestloop_path(PlannerInfo *r
*** 375,383 ****
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* If the inner path is parameterized, the parameterization must be fully
--- 440,456 ----
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* The same checks we do in try_nestloop_path. */
+ Assert(!do_aggregate || grouped);
+ Assert(joinrel->gpi != NULL || !grouped);
/*
* If the inner path is parameterized, the parameterization must be fully
*************** try_partial_nestloop_path(PlannerInfo *r
*** 401,422 ****
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path,
extra->sjinfo, &extra->semifactors);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys))
return;
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_nestloop_path(root,
! joinrel,
! jointype,
! &workspace,
! extra->sjinfo,
! &extra->semifactors,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL));
}
/*
--- 474,555 ----
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path,
extra->sjinfo, &extra->semifactors);
!
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! {
! Assert(joinrel->gpi != NULL);
! join_target = joinrel->gpi->target;
! }
!
! join_path = (Path *) create_nestloop_path(root, joinrel, jointype,
! &workspace, extra->sjinfo,
! &extra->semifactors,
! outer_path, inner_path,
! extra->restrictlist, pathkeys,
! NULL, join_target);
!
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, true, AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, true, AGG_SORTED);
! }
! else if (add_partial_path_precheck(joinrel, workspace.total_cost,
! pathkeys, grouped))
! {
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *) join_path, grouped);
! }
! }
!
! static void
! try_grouped_nestloop_path(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *pathkeys,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate,
! bool partial)
! {
! /*
! * Missing GroupedPathInfo indicates that we should not try to create a
! * grouped join.
! */
! if (joinrel->gpi == NULL)
return;
! /*
! * Reject the path if we're supposed to combine grouped and plain relation
! * but the grouped one does not evaluate all the relevant aggregates.
! */
! if (!do_aggregate &&
! !is_grouped_join_target_complete(root, joinrel->gpi->target,
! outer_path, inner_path))
! return;
!
! /*
! * As repeated aggregation doesn't seem to be attractive, make sure that
! * the resulting grouped relation is not parameterized.
! */
! if (outer_path->param_info != NULL || inner_path->param_info != NULL)
! return;
!
! if (!partial)
! try_nestloop_path(root, joinrel, outer_path, inner_path, pathkeys,
! jointype, extra, true, do_aggregate);
! else
! try_partial_nestloop_path(root, joinrel, outer_path, inner_path,
! pathkeys, jointype, extra, true,
! do_aggregate);
}
/*
*************** try_mergejoin_path(PlannerInfo *root,
*** 435,444 ****
List *innersortkeys,
JoinType jointype,
JoinPathExtraData *extra,
! bool is_partial)
{
Relids required_outer;
JoinCostWorkspace workspace;
if (is_partial)
{
--- 568,587 ----
List *innersortkeys,
JoinType jointype,
JoinPathExtraData *extra,
! bool is_partial,
! bool grouped,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* Caller should not request aggregation w/o grouped output. */
+ Assert(!do_aggregate || grouped);
+
+ /* GroupedPathInfo is necessary for us to produce a grouped set. */
+ Assert(joinrel->gpi != NULL || !grouped);
if (is_partial)
{
*************** try_mergejoin_path(PlannerInfo *root,
*** 451,472 ****
outersortkeys,
innersortkeys,
jointype,
! extra);
return;
}
/*
! * Check to see if proposed path is still parameterized, and reject if the
! * parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path,
! inner_path);
! if (required_outer &&
! !bms_overlap(required_outer, extra->param_source_rels))
{
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
}
/*
--- 594,618 ----
outersortkeys,
innersortkeys,
jointype,
! extra,
! grouped,
! do_aggregate);
return;
}
/*
! * Check to see if proposed path is still parameterized, and reject if
! * it's grouped or if the parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path, inner_path);
! if (required_outer)
{
! if (grouped || !bms_overlap(required_outer, extra->param_source_rels))
! {
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
! }
}
/*
*************** try_mergejoin_path(PlannerInfo *root,
*** 488,511 ****
outersortkeys, innersortkeys,
extra->sjinfo);
! if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer))
{
! add_path(joinrel, (Path *)
! create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra->sjinfo,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer,
! mergeclauses,
! outersortkeys,
! innersortkeys));
}
else
{
--- 634,679 ----
outersortkeys, innersortkeys,
extra->sjinfo);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! join_target = joinrel->gpi->target;
!
!
! join_path = (Path *) create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra->sjinfo,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_SORTED);
! }
! else if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, grouped))
{
! add_path(joinrel, (Path *) join_path, grouped);
}
else
{
*************** try_partial_mergejoin_path(PlannerInfo *
*** 529,537 ****
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* See comments in try_partial_hashjoin_path().
--- 697,713 ----
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* The same checks we do in try_mergejoin_path. */
+ Assert(!do_aggregate || grouped);
+ Assert(joinrel->gpi != NULL || !grouped);
/*
* See comments in try_partial_hashjoin_path().
*************** try_partial_mergejoin_path(PlannerInfo *
*** 564,587 ****
outersortkeys, innersortkeys,
extra->sjinfo);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys))
return;
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra->sjinfo,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL,
! mergeclauses,
! outersortkeys,
! innersortkeys));
}
/*
--- 740,910 ----
outersortkeys, innersortkeys,
extra->sjinfo);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! {
! Assert(joinrel->gpi != NULL);
! join_target = joinrel->gpi->target;
! }
!
! join_path = (Path *) create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra->sjinfo,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! join_target);
!
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, true, AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, true, AGG_SORTED);
! }
! else if (add_partial_path_precheck(joinrel, workspace.total_cost,
! pathkeys, grouped))
! {
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *) join_path, grouped);
! }
! }
!
! static void
! try_grouped_mergejoin_path(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *pathkeys,
! List *mergeclauses,
! List *outersortkeys,
! List *innersortkeys,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool do_aggregate)
! {
! /*
! * Missing GroupedPathInfo indicates that we should not try to create a
! * grouped join.
! */
! if (joinrel->gpi == NULL)
return;
! /*
! * Reject the path if we're supposed to combine grouped and plain relation
! * but the grouped one does not evaluate all the relevant aggregates.
! */
! if (!do_aggregate &&
! !is_grouped_join_target_complete(root, joinrel->gpi->target,
! outer_path, inner_path))
! return;
!
! /*
! * As repeated aggregation doesn't seem to be attractive, make sure that
! * the resulting grouped relation is not parameterized.
! */
! if (outer_path->param_info != NULL || inner_path->param_info != NULL)
! return;
!
! if (!partial)
! try_mergejoin_path(root, joinrel, outer_path, inner_path, pathkeys,
! mergeclauses, outersortkeys, innersortkeys,
! jointype, extra, false, true, do_aggregate);
! else
! try_partial_mergejoin_path(root, joinrel, outer_path, inner_path,
! pathkeys,
! mergeclauses, outersortkeys, innersortkeys,
! jointype, extra, true, do_aggregate);
! }
!
! static void
! try_mergejoin_path_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *pathkeys,
! List *mergeclauses,
! List *outersortkeys,
! List *innersortkeys,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
! {
! bool grouped_join;
!
! grouped_join = grouped_outer || grouped_inner || do_aggregate;
!
! /* Join of two grouped paths is not supported. */
! Assert(!(grouped_outer && grouped_inner));
!
! if (!grouped_join)
! {
! /* Only join plain paths. */
! try_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! jointype,
! extra,
! partial,
! false, false);
! }
! else if (grouped_outer || grouped_inner)
! {
! Assert(!do_aggregate);
!
! /*
! * Exactly one of the input paths is grouped, so create a grouped join
! * path.
! */
! try_grouped_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! jointype,
! extra,
! partial,
! false);
! }
! /* Preform explicit aggregation only if suitable target exists. */
! else if (joinrel->gpi != NULL)
! {
! try_grouped_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! jointype,
! extra,
! partial, true);
! }
}
/*
*************** try_hashjoin_path(PlannerInfo *root,
*** 596,644 ****
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra)
{
Relids required_outer;
JoinCostWorkspace workspace;
/*
! * Check to see if proposed path is still parameterized, and reject if the
! * parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path,
! inner_path);
! if (required_outer &&
! !bms_overlap(required_outer, extra->param_source_rels))
{
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
}
/*
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
! outer_path, inner_path,
! extra->sjinfo, &extra->semifactors);
! if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! NIL, required_outer))
{
! add_path(joinrel, (Path *)
! create_hashjoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra->sjinfo,
! &extra->semifactors,
! outer_path,
! inner_path,
! extra->restrictlist,
! required_outer,
! hashclauses));
}
else
{
--- 919,994 ----
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* Caller should not request aggregation w/o grouped output. */
+ Assert(!do_aggregate || grouped);
+
+ /* GroupedPathInfo is necessary for us to produce a grouped set. */
+ Assert(joinrel->gpi != NULL || !grouped);
/*
! * Check to see if proposed path is still parameterized, and reject if
! * it's grouped or if the parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path, inner_path);
! if (required_outer)
{
! if (grouped || !bms_overlap(required_outer, extra->param_source_rels))
! {
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
! }
}
/*
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
+ *
+ * TODO Need to consider aggregation here?
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
! outer_path, inner_path, extra->sjinfo, &extra->semifactors);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! join_target = joinrel->gpi->target;
!
! join_path = (Path *) create_hashjoin_path(root, joinrel, jointype,
! &workspace,
! extra->sjinfo,
! &extra->semifactors,
! outer_path, inner_path,
! extra->restrictlist,
! required_outer, hashclauses,
! join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_HASHED);
! }
! else if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! NIL, required_outer, grouped))
{
! add_path(joinrel, (Path *) join_path, grouped);
}
else
{
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 659,667 ****
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* If the inner path is parameterized, the parameterization must be fully
--- 1009,1025 ----
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* The same checks we do in try_hashjoin_path. */
+ Assert(!do_aggregate || grouped);
+ Assert(joinrel->gpi != NULL || !grouped);
/*
* If the inner path is parameterized, the parameterization must be fully
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 685,706 ****
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path,
extra->sjinfo, &extra->semifactors);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, NIL))
return;
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_hashjoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra->sjinfo,
! &extra->semifactors,
! outer_path,
! inner_path,
! extra->restrictlist,
! NULL,
! hashclauses));
}
/*
--- 1043,1139 ----
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path,
extra->sjinfo, &extra->semifactors);
!
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! {
! Assert(joinrel->gpi != NULL);
! join_target = joinrel->gpi->target;
! }
!
! join_path = (Path *) create_hashjoin_path(root, joinrel, jointype,
! &workspace,
! extra->sjinfo,
! &extra->semifactors,
! outer_path, inner_path,
! extra->restrictlist, NULL,
! hashclauses, join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, true, AGG_HASHED);
! }
! else if (add_partial_path_precheck(joinrel, workspace.total_cost,
! NIL, grouped))
! {
! add_partial_path(joinrel, (Path *) join_path , grouped);
! }
! }
!
! /*
! * Create a new grouped hash join path by joining a grouped path to plain
! * (non-grouped) one, or by joining 2 plain relations and applying grouping on
! * the result.
! *
! * Joining of 2 grouped paths is not supported. If a grouped relation A was
! * joined to grouped relation B, then the grouping of B reduces the number of
! * times each group of A is appears in the join output. This makes difference
! * for some aggregates, e.g. sum().
! *
! * If do_aggregate is true, neither input rel is grouped so we need to
! * aggregate the join result explicitly.
! *
! * partial argument tells whether the join path should be considered partial.
! */
! static void
! try_grouped_hashjoin_path(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *hashclauses,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate,
! bool partial)
! {
! /*
! * Missing GroupedPathInfo indicates that we should not try to create a
! * grouped join.
! */
! if (joinrel->gpi == NULL)
return;
! /*
! * Reject the path if we're supposed to combine grouped and plain relation
! * but the grouped one does not evaluate all the relevant aggregates.
! */
! if (!do_aggregate &&
! !is_grouped_join_target_complete(root, joinrel->gpi->target,
! outer_path, inner_path))
! return;
!
! /*
! * As repeated aggregation doesn't seem to be attractive, make sure that
! * the resulting grouped relation is not parameterized.
! */
! if (outer_path->param_info != NULL || inner_path->param_info != NULL)
! return;
!
! if (!partial)
! try_hashjoin_path(root, joinrel, outer_path, inner_path, hashclauses,
! jointype, extra, true, do_aggregate);
! else
! try_partial_hashjoin_path(root, joinrel, outer_path, inner_path,
! hashclauses, jointype, extra, true,
! do_aggregate);
}
/*
*************** sort_inner_and_outer(PlannerInfo *root,
*** 751,757 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
Path *outer_path;
--- 1184,1223 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped)
! {
! if (!grouped)
! {
! sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, false, false, false);
! }
! else
! {
! /* Use all the supported strategies to generate grouped join. */
! sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, true, false, false);
! sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, false, true, false);
! sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, false, false, true);
! }
! }
!
! /*
! * TODO As merge_pathkeys shouldn't differ across execution, use a separate
! * function to derive them and pass them here in a list.
! */
! static void
! sort_inner_and_outer_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! RelOptInfo *outerrel,
! RelOptInfo *innerrel,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
{
JoinType save_jointype = jointype;
Path *outer_path;
*************** sort_inner_and_outer(PlannerInfo *root,
*** 760,765 ****
--- 1226,1232 ----
Path *cheapest_safe_inner = NULL;
List *all_pathkeys;
ListCell *l;
+ bool grouped_result;
/*
* We only consider the cheapest-total-cost input paths, since we are
*************** sort_inner_and_outer(PlannerInfo *root,
*** 774,781 ****
* against mergejoins with parameterized inputs; see comments in
* src/backend/optimizer/README.
*/
! outer_path = outerrel->cheapest_total_path;
! inner_path = innerrel->cheapest_total_path;
/*
* If either cheapest-total path is parameterized by the other rel, we
--- 1241,1267 ----
* against mergejoins with parameterized inputs; see comments in
* src/backend/optimizer/README.
*/
! if (grouped_outer)
! {
! if (outerrel->gpi != NULL && outerrel->gpi->pathlist != NIL)
! outer_path = linitial(outerrel->gpi->pathlist);
! else
! return;
! }
! else
! outer_path = outerrel->cheapest_total_path;
!
! if (grouped_inner)
! {
! if (innerrel->gpi != NULL && innerrel->gpi->pathlist != NIL)
! inner_path = linitial(innerrel->gpi->pathlist);
! else
! return;
! }
! else
! inner_path = innerrel->cheapest_total_path;
!
! grouped_result = grouped_outer || grouped_inner || do_aggregate;
/*
* If either cheapest-total path is parameterized by the other rel, we
*************** sort_inner_and_outer(PlannerInfo *root,
*** 821,833 ****
outerrel->partial_pathlist != NIL &&
bms_is_empty(joinrel->lateral_relids))
{
! cheapest_partial_outer = (Path *) linitial(outerrel->partial_pathlist);
if (inner_path->parallel_safe)
cheapest_safe_inner = inner_path;
else if (save_jointype != JOIN_UNIQUE_INNER)
cheapest_safe_inner =
! get_cheapest_parallel_safe_total_inner(innerrel->pathlist);
}
/*
--- 1307,1356 ----
outerrel->partial_pathlist != NIL &&
bms_is_empty(joinrel->lateral_relids))
{
! if (grouped_outer)
! {
! if (outerrel->gpi != NULL && outerrel->gpi->partial_pathlist != NIL)
! cheapest_partial_outer = (Path *)
! linitial(outerrel->gpi->partial_pathlist);
! else
! return;
! }
! else
! cheapest_partial_outer = (Path *)
! linitial(outerrel->partial_pathlist);
!
! if (grouped_inner)
! {
! if (innerrel->gpi != NULL && innerrel->gpi->pathlist != NIL)
! inner_path = linitial(innerrel->gpi->pathlist);
! else
! return;
! }
! else
! inner_path = innerrel->cheapest_total_path;
if (inner_path->parallel_safe)
cheapest_safe_inner = inner_path;
else if (save_jointype != JOIN_UNIQUE_INNER)
+ {
+ List *inner_pathlist;
+
+ if (!grouped_inner)
+ inner_pathlist = innerrel->pathlist;
+ else
+ {
+ Assert(innerrel->gpi != NULL);
+ inner_pathlist = innerrel->gpi->pathlist;
+ }
+
+ /*
+ * All the grouped paths should be unparameterized, so the
+ * function is overly stringent in the grouped_inner case, but
+ * still useful.
+ */
cheapest_safe_inner =
! get_cheapest_parallel_safe_total_inner(inner_pathlist);
! }
}
/*
*************** sort_inner_and_outer(PlannerInfo *root,
*** 903,935 ****
* properly. try_mergejoin_path will detect that case and suppress an
* explicit sort step, so we needn't do so here.
*/
! try_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! merge_pathkeys,
! cur_mergeclauses,
! outerkeys,
! innerkeys,
! jointype,
! extra,
! false);
/*
* If we have partial outer and parallel safe inner path then try
* partial mergejoin path.
*/
if (cheapest_partial_outer && cheapest_safe_inner)
! try_partial_mergejoin_path(root,
! joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! merge_pathkeys,
! cur_mergeclauses,
! outerkeys,
! innerkeys,
! jointype,
! extra);
}
}
--- 1426,1484 ----
* properly. try_mergejoin_path will detect that case and suppress an
* explicit sort step, so we needn't do so here.
*/
! if (!grouped_result)
! try_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! merge_pathkeys,
! cur_mergeclauses,
! outerkeys,
! innerkeys,
! jointype,
! extra,
! false, false, false);
! else
! {
! try_mergejoin_path_common(root, joinrel, outer_path, inner_path,
! merge_pathkeys, cur_mergeclauses,
! outerkeys, innerkeys, jointype, extra,
! false,
! grouped_outer, grouped_inner,
! do_aggregate);
! }
/*
* If we have partial outer and parallel safe inner path then try
* partial mergejoin path.
*/
if (cheapest_partial_outer && cheapest_safe_inner)
! {
! if (!grouped_result)
! {
! try_partial_mergejoin_path(root,
! joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! merge_pathkeys,
! cur_mergeclauses,
! outerkeys,
! innerkeys,
! jointype,
! extra, false, false);
! }
! else
! {
! try_mergejoin_path_common(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! merge_pathkeys, cur_mergeclauses,
! outerkeys, innerkeys, jointype, extra,
! true,
! grouped_outer, grouped_inner,
! do_aggregate);
! }
! }
}
}
*************** sort_inner_and_outer(PlannerInfo *root,
*** 946,951 ****
--- 1495,1508 ----
* some sort key requirements). So, we consider truncations of the
* mergeclause list as well as the full list. (Ideally we'd consider all
* subsets of the mergeclause list, but that seems way too expensive.)
+ *
+ * grouped_outer - is outerpath grouped?
+ * grouped_inner - use grouped paths of innerrel?
+ * do_aggregate - apply (partial) aggregation to the output?
+ *
+ * TODO If subsequent calls often differ only by the 3 arguments above,
+ * consider a workspace structure to share useful info (eg merge clauses)
+ * across calls.
*/
static void
generate_mergejoin_paths(PlannerInfo *root,
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 957,963 ****
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial)
{
List *mergeclauses;
List *innersortkeys;
--- 1514,1523 ----
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
{
List *mergeclauses;
List *innersortkeys;
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1008,1024 ****
* try_mergejoin_path will do the right thing if inner_cheapest_total is
* already correctly sorted.)
*/
! try_mergejoin_path(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! mergeclauses,
! NIL,
! innersortkeys,
! jointype,
! extra,
! is_partial);
/* Can't do anything else if inner path needs to be unique'd */
if (save_jointype == JOIN_UNIQUE_INNER)
--- 1568,1585 ----
* try_mergejoin_path will do the right thing if inner_cheapest_total is
* already correctly sorted.)
*/
! try_mergejoin_path_common(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! mergeclauses,
! NIL,
! innersortkeys,
! jointype,
! extra,
! is_partial,
! grouped_outer, grouped_inner, do_aggregate);
/* Can't do anything else if inner path needs to be unique'd */
if (save_jointype == JOIN_UNIQUE_INNER)
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1074,1089 ****
for (sortkeycnt = num_sortkeys; sortkeycnt > 0; sortkeycnt--)
{
Path *innerpath;
List *newclauses = NIL;
/*
* Look for an inner path ordered well enough for the first
* 'sortkeycnt' innersortkeys. NB: trialsortkeys list is modified
* destructively, which is why we made a copy...
*/
trialsortkeys = list_truncate(trialsortkeys, sortkeycnt);
! innerpath = get_cheapest_path_for_pathkeys(innerrel->pathlist,
trialsortkeys,
NULL,
TOTAL_COST,
--- 1635,1656 ----
for (sortkeycnt = num_sortkeys; sortkeycnt > 0; sortkeycnt--)
{
+ List *inner_pathlist = NIL;
Path *innerpath;
List *newclauses = NIL;
+ if (!grouped_inner)
+ inner_pathlist = innerrel->pathlist;
+ else if (innerrel->gpi != NULL)
+ inner_pathlist = innerrel->gpi->pathlist;
+
/*
* Look for an inner path ordered well enough for the first
* 'sortkeycnt' innersortkeys. NB: trialsortkeys list is modified
* destructively, which is why we made a copy...
*/
trialsortkeys = list_truncate(trialsortkeys, sortkeycnt);
! innerpath = get_cheapest_path_for_pathkeys(inner_pathlist,
trialsortkeys,
NULL,
TOTAL_COST,
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1106,1126 ****
}
else
newclauses = mergeclauses;
! try_mergejoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial);
cheapest_total_inner = innerpath;
}
/* Same on the basis of cheapest startup cost ... */
! innerpath = get_cheapest_path_for_pathkeys(innerrel->pathlist,
trialsortkeys,
NULL,
STARTUP_COST,
--- 1673,1697 ----
}
else
newclauses = mergeclauses;
!
! try_mergejoin_path_common(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial,
! grouped_outer, grouped_inner,
! do_aggregate);
!
cheapest_total_inner = innerpath;
}
/* Same on the basis of cheapest startup cost ... */
! innerpath = get_cheapest_path_for_pathkeys(inner_pathlist,
trialsortkeys,
NULL,
STARTUP_COST,
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1151,1167 ****
else
newclauses = mergeclauses;
}
! try_mergejoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial);
}
cheapest_startup_inner = innerpath;
}
--- 1722,1740 ----
else
newclauses = mergeclauses;
}
! try_mergejoin_path_common(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial,
! grouped_outer, grouped_inner,
! do_aggregate);
}
cheapest_startup_inner = innerpath;
}
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1196,1201 ****
--- 1769,1776 ----
* 'innerrel' is the inner join relation
* 'jointype' is the type of join to do
* 'extra' contains additional input values
+ * 'grouped' indicates that the at least one relation in the join has been
+ * aggregated.
*/
static void
match_unsorted_outer(PlannerInfo *root,
*************** match_unsorted_outer(PlannerInfo *root,
*** 1203,1209 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
bool nestjoinOK;
--- 1778,1785 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped)
{
JoinType save_jointype = jointype;
bool nestjoinOK;
*************** match_unsorted_outer(PlannerInfo *root,
*** 1213,1218 ****
--- 1789,1816 ----
ListCell *lc1;
/*
+ * If grouped join path is requested, we ignore cases where either input
+ * path needs to be unique. For each side we should expect either grouped
+ * or plain relation, which differ quite a bit.
+ *
+ * XXX Although unique-ification of grouped path might result in too
+ * expensive input path (note that grouped input relation is not
+ * necessarily unique, regardless the grouping keys --- one or more plain
+ * relation could already have been joined to it), we might want to
+ * unique-ify the input relation in the future at least in the case it's a
+ * plain relation.
+ *
+ * (Materialization is not involved in grouped paths for similar reasons.)
+ */
+ if (grouped &&
+ (jointype == JOIN_UNIQUE_OUTER || jointype == JOIN_UNIQUE_INNER))
+ return;
+
+ /* No grouped join w/o grouped target. */
+ if (grouped && joinrel->gpi == NULL)
+ return;
+
+ /*
* Nestloop only supports inner, left, semi, and anti joins. Also, if we
* are doing a right or full mergejoin, we must use *all* the mergeclauses
* as join clauses, else we will not have a valid plan. (Although these
*************** match_unsorted_outer(PlannerInfo *root,
*** 1268,1274 ****
create_unique_path(root, innerrel, inner_cheapest_total, extra->sjinfo);
Assert(inner_cheapest_total);
}
! else if (nestjoinOK)
{
/*
* Consider materializing the cheapest inner path, unless
--- 1866,1872 ----
create_unique_path(root, innerrel, inner_cheapest_total, extra->sjinfo);
Assert(inner_cheapest_total);
}
! else if (nestjoinOK && !grouped)
{
/*
* Consider materializing the cheapest inner path, unless
*************** match_unsorted_outer(PlannerInfo *root,
*** 1299,1304 ****
--- 1897,1904 ----
*/
if (save_jointype == JOIN_UNIQUE_OUTER)
{
+ Assert(!grouped);
+
if (outerpath != outerrel->cheapest_total_path)
continue;
outerpath = (Path *) create_unique_path(root, outerrel,
*************** match_unsorted_outer(PlannerInfo *root,
*** 1326,1332 ****
inner_cheapest_total,
merge_pathkeys,
jointype,
! extra);
}
else if (nestjoinOK)
{
--- 1926,1933 ----
inner_cheapest_total,
merge_pathkeys,
jointype,
! extra,
! false, false);
}
else if (nestjoinOK)
{
*************** match_unsorted_outer(PlannerInfo *root,
*** 1342,1365 ****
{
Path *innerpath = (Path *) lfirst(lc2);
! try_nestloop_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! jointype,
! extra);
}
! /* Also consider materialized form of the cheapest inner path */
! if (matpath != NULL)
try_nestloop_path(root,
joinrel,
outerpath,
matpath,
merge_pathkeys,
jointype,
! extra);
}
/* Can't do anything else if outer path needs to be unique'd */
--- 1943,1988 ----
{
Path *innerpath = (Path *) lfirst(lc2);
! if (!grouped)
! try_nestloop_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! jointype,
! extra, false, false);
! else
! {
! /*
! * Since both input paths are plain, request explicit
! * aggregation.
! */
! try_grouped_nestloop_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! jointype,
! extra,
! true,
! false);
! }
}
! /*
! * Also consider materialized form of the cheapest inner path.
! *
! * (There's no matpath for grouped join.)
! */
! if (matpath != NULL && !grouped)
try_nestloop_path(root,
joinrel,
outerpath,
matpath,
merge_pathkeys,
jointype,
! extra,
! false, false);
}
/* Can't do anything else if outer path needs to be unique'd */
*************** match_unsorted_outer(PlannerInfo *root,
*** 1374,1380 ****
generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
save_jointype, extra, useallclauses,
inner_cheapest_total, merge_pathkeys,
! false);
}
/*
--- 1997,2073 ----
generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
save_jointype, extra, useallclauses,
inner_cheapest_total, merge_pathkeys,
! false, false, false, grouped);
!
! /* Try to join the plain outer relation to grouped inner. */
! if (grouped && nestjoinOK &&
! save_jointype != JOIN_UNIQUE_OUTER &&
! save_jointype != JOIN_UNIQUE_INNER &&
! innerrel->gpi != NULL && outerrel->gpi == NULL)
! {
! Path *inner_cheapest_grouped = (Path *) linitial(innerrel->gpi->pathlist);
!
! if (PATH_PARAM_BY_REL(inner_cheapest_grouped, outerrel))
! continue;
!
! /* XXX Shouldn't Assert() be used here instead? */
! if (PATH_PARAM_BY_REL(outerpath, innerrel))
! continue;
!
! /*
! * Only outer grouped path is interesting in this case: grouped
! * path on the inner side of NL join would imply repeated
! * aggregation somewhere in the inner path.
! */
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! save_jointype, extra, useallclauses,
! inner_cheapest_grouped, merge_pathkeys,
! false, false, true, false);
! }
! }
!
! /*
! * Combine grouped outer and plain inner paths.
! */
! if (grouped && nestjoinOK &&
! save_jointype != JOIN_UNIQUE_OUTER &&
! save_jointype != JOIN_UNIQUE_INNER)
! {
! /*
! * If the inner rel had a grouped target, its plain paths should be
! * ignored. Otherwise we could create grouped paths with different
! * targets.
! */
! if (outerrel->gpi != NULL && innerrel->gpi == NULL &&
! inner_cheapest_total != NULL)
! {
! /* Nested loop paths. */
! foreach(lc1, outerrel->gpi->pathlist)
! {
! Path *outerpath = (Path *) lfirst(lc1);
! List *merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
! outerpath->pathkeys);
!
! if (PATH_PARAM_BY_REL(outerpath, innerrel))
! continue;
!
! try_grouped_nestloop_path(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! jointype,
! extra,
! false,
! false);
!
! /* Merge join paths. */
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! save_jointype, extra, useallclauses,
! inner_cheapest_total, merge_pathkeys,
! false, true, false, false);
! }
! }
}
/*
*************** match_unsorted_outer(PlannerInfo *root,
*** 1394,1401 ****
bms_is_empty(joinrel->lateral_relids))
{
if (nestjoinOK)
! consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra);
/*
* If inner_cheapest_total is NULL or non parallel-safe then find the
--- 2087,2107 ----
bms_is_empty(joinrel->lateral_relids))
{
if (nestjoinOK)
! {
! if (!grouped)
! /* Plain partial paths. */
! consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra, false, false);
! else
! {
! /* Grouped partial paths with explicit aggregation. */
! consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra, true, true);
! /* Grouped partial paths w/o explicit aggregation. */
! consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra, true, false);
! }
! }
/*
* If inner_cheapest_total is NULL or non parallel-safe then find the
*************** match_unsorted_outer(PlannerInfo *root,
*** 1415,1421 ****
if (inner_cheapest_total)
consider_parallel_mergejoin(root, joinrel, outerrel, innerrel,
save_jointype, extra,
! inner_cheapest_total);
}
}
--- 2121,2127 ----
if (inner_cheapest_total)
consider_parallel_mergejoin(root, joinrel, outerrel, innerrel,
save_jointype, extra,
! inner_cheapest_total, grouped);
}
}
*************** consider_parallel_mergejoin(PlannerInfo
*** 1438,1447 ****
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total)
{
ListCell *lc1;
/* generate merge join path for each partial outer path */
foreach(lc1, outerrel->partial_pathlist)
{
--- 2144,2162 ----
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total,
! bool grouped)
{
ListCell *lc1;
+ if (grouped)
+ {
+ /* TODO Consider if these types should be supported. */
+ if (jointype == JOIN_UNIQUE_OUTER ||
+ jointype == JOIN_UNIQUE_INNER)
+ return;
+ }
+
/* generate merge join path for each partial outer path */
foreach(lc1, outerrel->partial_pathlist)
{
*************** consider_parallel_mergejoin(PlannerInfo
*** 1454,1462 ****
merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
outerpath->pathkeys);
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath, jointype,
! extra, false, inner_cheapest_total,
! merge_pathkeys, true);
}
}
--- 2169,2224 ----
merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
outerpath->pathkeys);
! if (!grouped)
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! jointype, extra, false,
! inner_cheapest_total, merge_pathkeys,
! true,
! false, false, false);
! else
! {
! /*
! * Create grouped join by joining plain rels and aggregating the
! * result.
! */
! Assert(joinrel->gpi != NULL);
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! jointype, extra, false,
! inner_cheapest_total, merge_pathkeys,
! true, false, false, true);
!
! /* Combine the plain outer with grouped inner one(s). */
! if (outerrel->gpi == NULL && innerrel->gpi != NULL)
! {
! Path *inner_cheapest_grouped = (Path *)
! linitial(innerrel->gpi->pathlist);
!
! if (inner_cheapest_grouped != NULL &&
! inner_cheapest_grouped->parallel_safe)
! generate_mergejoin_paths(root, joinrel, innerrel,
! outerpath, jointype, extra,
! false, inner_cheapest_grouped,
! merge_pathkeys,
! true, false, true, false);
! }
! }
! }
!
! /* In addition, try to join grouped outer to plain inner one(s). */
! if (grouped && outerrel->gpi != NULL && innerrel->gpi == NULL)
! {
! foreach(lc1, outerrel->gpi->partial_pathlist)
! {
! Path *outerpath = (Path *) lfirst(lc1);
! List *merge_pathkeys;
!
! merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
! outerpath->pathkeys);
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! jointype, extra, false,
! inner_cheapest_total, merge_pathkeys,
! true, true, false, false);
! }
}
}
*************** consider_parallel_nestloop(PlannerInfo *
*** 1477,1491 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
ListCell *lc1;
if (jointype == JOIN_UNIQUE_INNER)
jointype = JOIN_INNER;
! foreach(lc1, outerrel->partial_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *pathkeys;
--- 2239,2283 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped, bool do_aggregate)
{
JoinType save_jointype = jointype;
+ List *outer_pathlist;
ListCell *lc1;
+ if (grouped)
+ {
+ /* TODO Consider if these types should be supported. */
+ if (save_jointype == JOIN_UNIQUE_OUTER ||
+ save_jointype == JOIN_UNIQUE_INNER)
+ return;
+ }
+
if (jointype == JOIN_UNIQUE_INNER)
jointype = JOIN_INNER;
! if (!grouped || do_aggregate)
! {
! /*
! * If creating grouped paths by explicit aggregation, the input paths
! * must be plain.
! */
! outer_pathlist = outerrel->partial_pathlist;
! }
! else if (outerrel->gpi != NULL)
! {
! /*
! * Only the outer paths are accepted as grouped when we try to combine
! * grouped and plain ones. Grouped inner path implies repeated
! * aggregation, which doesn't sound as a good idea.
! */
! outer_pathlist = outerrel->gpi->partial_pathlist;
! }
! else
! return;
!
! foreach(lc1, outer_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *pathkeys;
*************** consider_parallel_nestloop(PlannerInfo *
*** 1516,1522 ****
* inner paths, but right now create_unique_path is not on board
* with that.)
*/
! if (save_jointype == JOIN_UNIQUE_INNER)
{
if (innerpath != innerrel->cheapest_total_path)
continue;
--- 2308,2314 ----
* inner paths, but right now create_unique_path is not on board
* with that.)
*/
! if (save_jointype == JOIN_UNIQUE_INNER && !grouped)
{
if (innerpath != innerrel->cheapest_total_path)
continue;
*************** consider_parallel_nestloop(PlannerInfo *
*** 1526,1533 ****
Assert(innerpath);
}
! try_partial_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra);
}
}
}
--- 2318,2343 ----
Assert(innerpath);
}
! if (!grouped)
! try_partial_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra,
! false, false);
! else if (do_aggregate)
! {
! /* Request aggregation as both input rels are plain. */
! try_grouped_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra,
! true, true);
! }
! /*
! * Only combine the grouped outer path with the plain inner if the
! * inner relation cannot produce grouped paths. Otherwise we could
! * generate grouped paths with different targets.
! */
! else if (innerrel->gpi == NULL)
! try_grouped_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra,
! false, true);
}
}
}
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1549,1561 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
bool isouterjoin = IS_OUTER_JOIN(jointype);
List *hashclauses;
ListCell *l;
/*
* We need to build only one hashclauses list for any given pair of outer
* and inner relations; all of the hashable clauses will be used as keys.
--- 2359,2376 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped)
{
JoinType save_jointype = jointype;
bool isouterjoin = IS_OUTER_JOIN(jointype);
List *hashclauses;
ListCell *l;
+ /* No grouped join w/o grouped target. */
+ if (grouped && joinrel->gpi == NULL)
+ return;
+
/*
* We need to build only one hashclauses list for any given pair of outer
* and inner relations; all of the hashable clauses will be used as keys.
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1605,1610 ****
--- 2420,2428 ----
* can't use a hashjoin. (There's no use looking for alternative
* input paths, since these should already be the least-parameterized
* available paths.)
+ *
+ * (The same check should work for grouped paths, as these don't
+ * differ in parameterization.)
*/
if (PATH_PARAM_BY_REL(cheapest_total_outer, innerrel) ||
PATH_PARAM_BY_REL(cheapest_total_inner, outerrel))
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1624,1630 ****
cheapest_total_inner,
hashclauses,
jointype,
! extra);
/* no possibility of cheap startup here */
}
else if (jointype == JOIN_UNIQUE_INNER)
--- 2442,2449 ----
cheapest_total_inner,
hashclauses,
jointype,
! extra,
! false, false);
/* no possibility of cheap startup here */
}
else if (jointype == JOIN_UNIQUE_INNER)
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1640,1646 ****
cheapest_total_inner,
hashclauses,
jointype,
! extra);
if (cheapest_startup_outer != NULL &&
cheapest_startup_outer != cheapest_total_outer)
try_hashjoin_path(root,
--- 2459,2466 ----
cheapest_total_inner,
hashclauses,
jointype,
! extra,
! false, false);
if (cheapest_startup_outer != NULL &&
cheapest_startup_outer != cheapest_total_outer)
try_hashjoin_path(root,
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1649,1711 ****
cheapest_total_inner,
hashclauses,
jointype,
! extra);
}
else
{
! /*
! * For other jointypes, we consider the cheapest startup outer
! * together with the cheapest total inner, and then consider
! * pairings of cheapest-total paths including parameterized ones.
! * There is no use in generating parameterized paths on the basis
! * of possibly cheap startup cost, so this is sufficient.
! */
! ListCell *lc1;
! ListCell *lc2;
!
! if (cheapest_startup_outer != NULL)
! try_hashjoin_path(root,
! joinrel,
! cheapest_startup_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra);
!
! foreach(lc1, outerrel->cheapest_parameterized_paths)
{
- Path *outerpath = (Path *) lfirst(lc1);
-
/*
! * We cannot use an outer path that is parameterized by the
! * inner rel.
*/
! if (PATH_PARAM_BY_REL(outerpath, innerrel))
! continue;
! foreach(lc2, innerrel->cheapest_parameterized_paths)
{
! Path *innerpath = (Path *) lfirst(lc2);
/*
! * We cannot use an inner path that is parameterized by
! * the outer rel, either.
*/
! if (PATH_PARAM_BY_REL(innerpath, outerrel))
continue;
! if (outerpath == cheapest_startup_outer &&
! innerpath == cheapest_total_inner)
! continue; /* already tried it */
! try_hashjoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! hashclauses,
! jointype,
! extra);
}
}
}
--- 2469,2622 ----
cheapest_total_inner,
hashclauses,
jointype,
! extra,
! false, false);
}
else
{
! if (!grouped)
{
/*
! * For other jointypes, we consider the cheapest startup outer
! * together with the cheapest total inner, and then consider
! * pairings of cheapest-total paths including parameterized
! * ones. There is no use in generating parameterized paths on
! * the basis of possibly cheap startup cost, so this is
! * sufficient.
*/
! ListCell *lc1;
! if (cheapest_startup_outer != NULL)
! try_hashjoin_path(root,
! joinrel,
! cheapest_startup_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra,
! false, false);
!
! foreach(lc1, outerrel->cheapest_parameterized_paths)
{
! Path *outerpath = (Path *) lfirst(lc1);
! ListCell *lc2;
/*
! * We cannot use an outer path that is parameterized by the
! * inner rel.
*/
! if (PATH_PARAM_BY_REL(outerpath, innerrel))
continue;
! foreach(lc2, innerrel->cheapest_parameterized_paths)
! {
! Path *innerpath = (Path *) lfirst(lc2);
! /*
! * We cannot use an inner path that is parameterized by
! * the outer rel, either.
! */
! if (PATH_PARAM_BY_REL(innerpath, outerrel))
! continue;
!
! if (outerpath == cheapest_startup_outer &&
! innerpath == cheapest_total_inner)
! continue; /* already tried it */
!
! try_hashjoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! hashclauses,
! jointype,
! extra,
! false, false);
! }
! }
! }
! else
! {
! /* Create grouped paths if possible. */
! /*
! * TODO
! *
! * Consider processing JOIN_UNIQUE_INNER and JOIN_UNIQUE_OUTER
! * join types, ie perform grouping of the inner / outer rel if
! * it's not unique yet and if the grouping is legal.
! */
! if (jointype == JOIN_UNIQUE_OUTER ||
! jointype == JOIN_UNIQUE_INNER)
! return;
!
! /*
! * Join grouped relation to non-grouped one.
! *
! * Do not use plain path of the input rel whose target does
! * have GroupedPahtInfo. For example (assuming that join of
! * two grouped rels is not supported), the only way to
! * evaluate SELECT sum(a.x), sum(b.y) ... is to join "a" and
! * "b" and aggregate the result. Otherwise the path target
! * wouldn't match joinrel->gpi->target. TODO Move this comment
! * elsewhere as it seems common to all join kinds.
! */
! /*
! * TODO Allow outer join if the grouped rel is on the
! * non-nullable side.
! */
! if (jointype == JOIN_INNER)
! {
! Path *grouped_path, *plain_path;
!
! if (outerrel->gpi != NULL &&
! outerrel->gpi->pathlist != NIL &&
! innerrel->gpi == NULL)
! {
! grouped_path = (Path *)
! linitial(outerrel->gpi->pathlist);
! plain_path = cheapest_total_inner;
! try_grouped_hashjoin_path(root, joinrel,
! grouped_path, plain_path,
! hashclauses, jointype,
! extra, false, false);
! }
! else if (innerrel->gpi != NULL &&
! innerrel->gpi->pathlist != NIL &&
! outerrel->gpi == NULL)
! {
! grouped_path = (Path *)
! linitial(innerrel->gpi->pathlist);
! plain_path = cheapest_total_outer;
! try_grouped_hashjoin_path(root, joinrel, plain_path,
! grouped_path, hashclauses,
! jointype, extra,
! false, false);
!
! if (cheapest_startup_outer != NULL &&
! cheapest_startup_outer != cheapest_total_outer)
! {
! plain_path = cheapest_startup_outer;
! try_grouped_hashjoin_path(root, joinrel,
! plain_path,
! grouped_path,
! hashclauses,
! jointype, extra,
! false, false);
! }
! }
}
+
+ /*
+ * Try to join plain relations and make a grouped rel out of
+ * the join.
+ *
+ * Since aggregation needs the whole relation, we are only
+ * interested in total costs.
+ */
+ try_grouped_hashjoin_path(root, joinrel,
+ cheapest_total_outer,
+ cheapest_total_inner,
+ hashclauses,
+ jointype, extra, true, false);
}
}
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1743,1758 ****
cheapest_safe_inner =
get_cheapest_parallel_safe_total_inner(innerrel->pathlist);
! if (cheapest_safe_inner != NULL)
! try_partial_hashjoin_path(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! hashclauses, jointype, extra);
}
}
}
/*
* select_mergejoin_clauses
* Select mergejoin clauses that are usable for a particular join.
* Returns a list of RestrictInfo nodes for those clauses.
--- 2654,2880 ----
cheapest_safe_inner =
get_cheapest_parallel_safe_total_inner(innerrel->pathlist);
! if (!grouped)
! {
! if (cheapest_safe_inner != NULL)
! try_partial_hashjoin_path(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! hashclauses, jointype, extra,
! false, false);
! }
! else if (joinrel->gpi != NULL)
! {
! /*
! * Grouped partial path.
! *
! * 1. Apply aggregation to the plain partial join path.
! */
! if (cheapest_safe_inner != NULL)
! try_grouped_hashjoin_path(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! hashclauses,
! jointype, extra, true, true);
!
! /*
! * 2. Join the cheapest partial grouped outer path (if one
! * exists) to cheapest_safe_inner (there's no reason to look
! * for another inner path than what we used for non-grouped
! * partial join path).
! */
! if (outerrel->gpi != NULL &&
! outerrel->gpi->partial_pathlist != NIL &&
! innerrel->gpi == NULL &&
! cheapest_safe_inner != NULL)
! {
! Path *outer_path;
!
! outer_path = (Path *)
! linitial(outerrel->gpi->partial_pathlist);
!
! try_grouped_hashjoin_path(root, joinrel, outer_path,
! cheapest_safe_inner,
! hashclauses,
! jointype, extra, false, true);
! }
!
! /*
! * 3. Join the cheapest_partial_outer path (again, no reason
! * to use different outer path than the one we used for plain
! * partial join) to the cheapest grouped inner path if the
! * latter exists and is parallel-safe.
! */
! if (innerrel->gpi != NULL &&
! innerrel->gpi->pathlist != NIL &&
! outerrel->gpi == NULL)
! {
! Path *inner_path;
!
! inner_path = (Path *) linitial(innerrel->gpi->pathlist);
!
! if (inner_path->parallel_safe)
! try_grouped_hashjoin_path(root, joinrel,
! cheapest_partial_outer,
! inner_path,
! hashclauses,
! jointype, extra,
! false, true);
! }
!
! /*
! * Other combinations seem impossible because: 1. At most 1
! * input relation of the join can be grouped, 2. the inner
! * path must not be partial.
! */
! }
}
}
}
/*
+ * Do the input paths emit all the aggregates contained in the grouped target
+ * of the join?
+ *
+ * The point is that one input relation might be unable to evaluate some
+ * aggregate(s), so it'll only generate plain paths. It's wrong to combine
+ * such plain paths with grouped ones that the other input rel might be able
+ * to generate because the result would miss the aggregate(s) the first
+ * relation failed to evaluate.
+ *
+ * TODO For better efficiency, consider storing Bitmapset of
+ * GroupedVarInfo.gvid in GroupedPathInfo.
+ */
+ static bool
+ is_grouped_join_target_complete(PlannerInfo *root, PathTarget *jointarget,
+ Path *outer_path, Path *inner_path)
+ {
+ RelOptInfo *outer_rel = outer_path->parent;
+ RelOptInfo *inner_rel = inner_path->parent;
+ ListCell *l1;
+
+ /*
+ * Join of two grouped relations is not supported.
+ *
+ * This actually isn't check of target completeness --- can it be located
+ * elsewhere?
+ */
+ if (outer_rel->gpi != NULL && inner_rel->gpi != NULL)
+ return false;
+
+ foreach(l1, jointarget->exprs)
+ {
+ Expr *expr = (Expr *) lfirst(l1);
+ GroupedVar *gvar;
+ GroupedVarInfo *gvi = NULL;
+ ListCell *l2;
+ bool found = false;
+
+ /* Only interested in aggregates. */
+ if (!IsA(expr, GroupedVar))
+ continue;
+
+ gvar = castNode(GroupedVar, expr);
+
+ /* Find the corresponding GroupedVarInfo. */
+ foreach(l2, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi_tmp = castNode(GroupedVarInfo, lfirst(l2));
+
+ if (gvi_tmp->gvid == gvar->gvid)
+ {
+ gvi = gvi_tmp;
+ break;
+ }
+ }
+ Assert(gvi != NULL);
+
+ /*
+ * If any aggregate references both input relations, something went
+ * wrong during construction of one of the input targets: one input
+ * rel is grouped, but no grouping target should have been created for
+ * it if some aggregate required more than that input rel.
+ */
+ Assert(gvi->gv_eval_at == NULL ||
+ !(bms_overlap(gvi->gv_eval_at, outer_rel->relids) &&
+ bms_overlap(gvi->gv_eval_at, inner_rel->relids)));
+
+ /*
+ * If the aggregate belongs to the plain relation, it probably
+ * means that non-grouping expression made aggregation of that
+ * input relation impossible. Since that expression is not
+ * necessarily emitted by the current join, aggregation might be
+ * possible here. On the other hand, aggregation of a join which
+ * already contains a grouped relation does not seem too
+ * beneficial.
+ *
+ * XXX The condition below is also met if the query contains both
+ * "star aggregate" and a normal one. Since the earlier can be
+ * added to any base relation, and since we don't support join of
+ * 2 grouped relations, join of arbitrary 2 relations will always
+ * result in a plain relation.
+ *
+ * XXX If we conclude that aggregation is worth, only consider
+ * this test failed if target usable for aggregation cannot be
+ * created (i.e. the non-grouping expression is in the output of
+ * the current join).
+ */
+ if ((outer_rel->gpi == NULL &&
+ bms_overlap(gvi->gv_eval_at, outer_rel->relids))
+ || (inner_rel->gpi == NULL &&
+ bms_overlap(gvi->gv_eval_at, inner_rel->relids)))
+ return false;
+
+ /* Look for the aggregate in the input targets. */
+ if (outer_rel->gpi != NULL)
+ {
+ /* No more than one input path should be grouped. */
+ Assert(inner_rel->gpi == NULL);
+
+ foreach(l2, outer_path->pathtarget->exprs)
+ {
+ expr = (Expr *) lfirst(l2);
+
+ if (!IsA(expr, GroupedVar))
+ continue;
+
+ gvar = castNode(GroupedVar, expr);
+ if (gvar->gvid == gvi->gvid)
+ {
+ found = true;
+ break;
+ }
+ }
+ }
+ else if (!found && inner_rel->gpi != NULL)
+ {
+ Assert(outer_rel->gpi == NULL);
+
+ foreach(l2, inner_path->pathtarget->exprs)
+ {
+ expr = (Expr *) lfirst(l2);
+
+ if (!IsA(expr, GroupedVar))
+ continue;
+
+ gvar = castNode(GroupedVar, expr);
+ if (gvar->gvid == gvi->gvid)
+ {
+ found = true;
+ break;
+ }
+ }
+ }
+
+ /* Even a single missing aggregate causes the whole test to fail. */
+ if (!found)
+ return false;
+ }
+
+ return true;
+ }
+
+ /*
* select_mergejoin_clauses
* Select mergejoin clauses that are usable for a particular join.
* Returns a list of RestrictInfo nodes for those clauses.
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
new file mode 100644
index 6a0c67b..58aea01
*** a/src/backend/optimizer/path/joinrels.c
--- b/src/backend/optimizer/path/joinrels.c
*************** mark_dummy_rel(RelOptInfo *rel)
*** 1217,1223 ****
rel->partial_pathlist = NIL;
/* Set up the dummy path */
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL));
/* Set or update cheapest_total_path and related fields */
set_cheapest(rel);
--- 1217,1223 ----
rel->partial_pathlist = NIL;
/* Set up the dummy path */
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL), false);
/* Set or update cheapest_total_path and related fields */
set_cheapest(rel);
diff --git a/src/backend/optimizer/path/tidpath.c b/src/backend/optimizer/path/tidpath.c
new file mode 100644
index a2fe661..91d855c
*** a/src/backend/optimizer/path/tidpath.c
--- b/src/backend/optimizer/path/tidpath.c
*************** create_tidscan_paths(PlannerInfo *root,
*** 266,270 ****
if (tidquals)
add_path(rel, (Path *) create_tidscan_path(root, rel, tidquals,
! required_outer));
}
--- 266,270 ----
if (tidquals)
add_path(rel, (Path *) create_tidscan_path(root, rel, tidquals,
! required_outer), false);
}
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
new file mode 100644
index 53aefbd..f19e18c
*** a/src/backend/optimizer/plan/initsplan.c
--- b/src/backend/optimizer/plan/initsplan.c
***************
*** 14,19 ****
--- 14,20 ----
*/
#include "postgres.h"
+ #include "access/sysattr.h"
#include "catalog/pg_type.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/clauses.h"
***************
*** 26,31 ****
--- 27,33 ----
#include "optimizer/planner.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "parser/analyze.h"
#include "rewrite/rewriteManip.h"
*************** typedef struct PostponedQual
*** 45,50 ****
--- 47,53 ----
} PostponedQual;
+ static void create_grouped_var_infos(PlannerInfo *root);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
*************** add_vars_to_targetlist(PlannerInfo *root
*** 240,245 ****
--- 243,532 ----
}
}
+ /*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate and setup
+ * GroupedPathInfo for each base relation that can product grouped paths.
+ *
+ * XXX In the future we might want to create GroupedVarInfo for grouping
+ * expressions too, so that grouping key is not limited to plain Var if the
+ * grouping takes place below the top-level join.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+ extern void
+ add_grouping_info_to_base_rels(PlannerInfo *root)
+ {
+ int i;
+
+ /* No grouping in the query? */
+ if (!root->parse->groupClause || root->group_pathkeys == NIL)
+ return;
+
+ /* TODO This is just for PoC. Relax the limitation later. */
+ if (root->parse->havingQual)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_grouped_var_infos(root);
+
+ /* Is no grouping is possible below the top-level join? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Process the individual base relations. */
+ for (i = 1; i < root->simple_rel_array_size; i++)
+ {
+ RelOptInfo *rel = root->simple_rel_array[i];
+
+ /*
+ * "other rels" will have their targets built later, by translation of
+ * the target of the parent rel - see set_append_rel_size. If we
+ * wanted to prepare the child rels here, we'd need another iteration
+ * of simple_rel_array_size.
+ */
+ if (rel != NULL && rel->reloptkind == RELOPT_BASEREL)
+ prepare_rel_for_grouping(root, rel);
+ }
+ }
+
+ /*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ *
+ * TODO Include aggregates from HAVING clause.
+ */
+ static void
+ create_grouped_var_infos(PlannerInfo *root)
+ {
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ /*
+ * TODO Check if processed_tlist contains the HAVING aggregates. If not,
+ * get them elsewhere.
+ */
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES);
+ if (tlist_exprs == NIL)
+ return;
+
+ /* tlist_exprs may also contain Vars, but we only need Aggrefs. */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ if (IsA(expr, Var))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct || aggref->aggfilter)
+ {
+ /*
+ * Partial aggregation is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ Expr *expr = (Expr *) lfirst(lc2);
+
+ gvi = castNode(GroupedVarInfo, expr);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ /* TODO Initialize gv_width. */
+ gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvid = list_length(root->grouped_var_list);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+ gvi->agg_partial = copyObject(aggref);
+ mark_partial_aggref(gvi->agg_partial, AGGSPLIT_INITIAL_SERIAL);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ {
+ Assert(aggref->aggstar);
+ gvi->gv_eval_at = NULL;
+ }
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+ }
+
+ /*
+ * Check if all the expressions of rel->reltarget can be used as grouping
+ * expressions and create target for grouped paths.
+ *
+ * If we succeed to create the grouping target, also replace rel->reltarget
+ * with a new one that has sortgrouprefs initialized -- this is necessary for
+ * create_agg_plan to match the grouping clauses against the input target
+ * expressions.
+ *
+ * rel_agg_attrs is a set attributes of the relation referenced by aggregate
+ * arguments. These can exist in the (plain) target without being grouping
+ * expressions.
+ *
+ * rel_agg_vars should be passed instead if rel is a join.
+ *
+ * TODO How about PHVs?
+ *
+ * TODO Make sure cost / width of both "result" and "plain" are correct.
+ */
+ PathTarget *
+ create_grouped_target(PlannerInfo *root, RelOptInfo *rel,
+ Relids rel_agg_attrs, List *rel_agg_vars)
+ {
+ PathTarget *result, *plain;
+ ListCell *lc;
+
+ /* The plan to be returned. */
+ result = create_empty_pathtarget();
+ /* The one to replace rel->reltarget. */
+ plain = create_empty_pathtarget();
+
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *texpr;
+ Index sortgroupref;
+ bool agg_arg_only = false;
+
+ texpr = (Expr *) lfirst(lc);
+
+ sortgroupref = get_expr_sortgroupref(root, texpr);
+ if (sortgroupref > 0)
+ {
+ /* It's o.k. to use the target expression for grouping. */
+ add_column_to_pathtarget(result, texpr, sortgroupref);
+
+ /*
+ * As for the plain target, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(plain, texpr, sortgroupref);
+
+ /* Process the next expression. */
+ continue;
+ }
+
+ /*
+ * It may still be o.k. if the expression is only contained in Aggref
+ * - then it's not expected in the grouped output.
+ *
+ * TODO Try to handle generic expression, not only Var. That might
+ * require us to create rel->reltarget of the grouping rel in
+ * parallel to that of the plain rel, and adding whole expressions
+ * instead of individual vars.
+ */
+ if (IsA(texpr, Var))
+ {
+ Var *arg_var = castNode(Var, texpr);
+
+ if (rel->relid > 0)
+ {
+ AttrNumber varattno;
+
+ /*
+ * For a single relation we only need to check attribute
+ * number.
+ *
+ * Apply the same offset that pull_varattnos() did.
+ */
+ varattno = arg_var->varattno - FirstLowInvalidHeapAttributeNumber;
+
+ if (bms_is_member(varattno, rel_agg_attrs))
+ agg_arg_only = true;
+ }
+ else
+ {
+ ListCell *lc2;
+
+ /* Join case. */
+ foreach(lc2, rel_agg_vars)
+ {
+ Var *var = castNode(Var, lfirst(lc2));
+
+ if (var->varno == arg_var->varno &&
+ var->varattno == arg_var->varattno)
+ {
+ agg_arg_only = true;
+ break;
+ }
+ }
+ }
+
+ if (agg_arg_only)
+ {
+ /*
+ * This expression is not suitable for grouping, but the
+ * aggregation input target ought to stay complete.
+ */
+ add_column_to_pathtarget(plain, texpr, 0);
+ }
+ }
+
+ /*
+ * A single mismatched expression makes the whole relation useless
+ * for grouping.
+ */
+ if (!agg_arg_only)
+ {
+ /*
+ * TODO This seems possible to happen multiple times per relation,
+ * so result might be worth freeing. Implement free_pathtarget()?
+ * Or mark the relation as inappropriate for grouping?
+ */
+ /* TODO Free both result and plain. */
+ return NULL;
+ }
+ }
+
+ if (list_length(result->exprs) == 0)
+ {
+ /* TODO free_pathtarget(result); free_pathtarget(plain) */
+ result = NULL;
+ }
+
+ /* Apply the adjusted input target as the replacement is complete now.q */
+ rel->reltarget = plain;
+
+ return result;
+ }
+
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c
new file mode 100644
index 5565736..058af2c
*** a/src/backend/optimizer/plan/planagg.c
--- b/src/backend/optimizer/plan/planagg.c
*************** preprocess_minmax_aggregates(PlannerInfo
*** 223,229 ****
create_minmaxagg_path(root, grouped_rel,
create_pathtarget(root, tlist),
aggs_list,
! (List *) parse->havingQual));
}
/*
--- 223,229 ----
create_minmaxagg_path(root, grouped_rel,
create_pathtarget(root, tlist),
aggs_list,
! (List *) parse->havingQual), false);
}
/*
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
new file mode 100644
index 3c58d05..5db7dec
*** a/src/backend/optimizer/plan/planmain.c
--- b/src/backend/optimizer/plan/planmain.c
*************** query_planner(PlannerInfo *root, List *t
*** 83,89 ****
add_path(final_rel, (Path *)
create_result_path(root, final_rel,
final_rel->reltarget,
! (List *) parse->jointree->quals));
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(final_rel);
--- 83,89 ----
add_path(final_rel, (Path *)
create_result_path(root, final_rel,
final_rel->reltarget,
! (List *) parse->jointree->quals), false);
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(final_rel);
*************** query_planner(PlannerInfo *root, List *t
*** 114,119 ****
--- 114,120 ----
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
*************** query_planner(PlannerInfo *root, List *t
*** 177,182 ****
--- 178,191 ----
(*qp_callback) (root, qp_extra);
/*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, Initialize GroupedPathInfo
+ * of base relations capable to do the grouping and setup
+ * root->grouped_var_list.
+ */
+ add_grouping_info_to_base_rels(root);
+
+ /*
* Examine any "placeholder" expressions generated during subquery pullup.
* Make sure that the Vars they need are marked as needed at the relevant
* join level. This must be done before join removal because it might
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
new file mode 100644
index f99257b..8245ce0
*** a/src/backend/optimizer/plan/planner.c
--- b/src/backend/optimizer/plan/planner.c
*************** static void standard_qp_callback(Planner
*** 130,138 ****
static double get_number_of_groups(PlannerInfo *root,
double path_rows,
grouping_sets_data *gd);
- static Size estimate_hashagg_tablesize(Path *path,
- const AggClauseCosts *agg_costs,
- double dNumGroups);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *target,
--- 130,135 ----
*************** inheritance_planner(PlannerInfo *root)
*** 1420,1426 ****
returningLists,
rowMarks,
NULL,
! SS_assign_special_param(root)));
}
/*--------------------
--- 1417,1423 ----
returningLists,
rowMarks,
NULL,
! SS_assign_special_param(root)), false);
}
/*--------------------
*************** grouping_planner(PlannerInfo *root, bool
*** 2041,2047 ****
}
/* And shove it into final_rel */
! add_path(final_rel, path);
}
/*
--- 2038,2044 ----
}
/* And shove it into final_rel */
! add_path(final_rel, path, false);
}
/*
*************** get_number_of_groups(PlannerInfo *root,
*** 3445,3484 ****
}
/*
- * estimate_hashagg_tablesize
- * estimate the number of bytes that a hash aggregate hashtable will
- * require based on the agg_costs, path width and dNumGroups.
- *
- * XXX this may be over-estimating the size now that hashagg knows to omit
- * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
- * grouping columns not in the hashed set are counted here even though hashagg
- * won't store them. Is this a problem?
- */
- static Size
- estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
- double dNumGroups)
- {
- Size hashentrysize;
-
- /* Estimate per-hash-entry space at tuple width... */
- hashentrysize = MAXALIGN(path->pathtarget->width) +
- MAXALIGN(SizeofMinimalTupleHeader);
-
- /* plus space for pass-by-ref transition values... */
- hashentrysize += agg_costs->transitionSpace;
- /* plus the per-hash-entry overhead */
- hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
-
- /*
- * Note that this disregards the effect of fill-factor and growth policy
- * of the hash-table. That's probably ok, given default the default
- * fill-factor is relatively high. It'd be hard to meaningfully factor in
- * "double-in-size" growth policies here.
- */
- return hashentrysize * dNumGroups;
- }
-
- /*
* create_grouping_paths
*
* Build a new upperrel containing Paths for grouping and/or aggregation.
--- 3442,3447 ----
*************** create_grouping_paths(PlannerInfo *root,
*** 3599,3605 ****
(List *) parse->havingQual);
}
! add_path(grouped_rel, path);
/* No need to consider any other alternatives. */
set_cheapest(grouped_rel);
--- 3562,3568 ----
(List *) parse->havingQual);
}
! add_path(grouped_rel, path, false);
/* No need to consider any other alternatives. */
set_cheapest(grouped_rel);
*************** create_grouping_paths(PlannerInfo *root,
*** 3776,3782 ****
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups));
else
add_partial_path(grouped_rel, (Path *)
create_group_path(root,
--- 3739,3746 ----
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups),
! false);
else
add_partial_path(grouped_rel, (Path *)
create_group_path(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 3785,3791 ****
partial_grouping_target,
parse->groupClause,
NIL,
! dNumPartialGroups));
}
}
}
--- 3749,3756 ----
partial_grouping_target,
parse->groupClause,
NIL,
! dNumPartialGroups),
! false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 3816,3822 ****
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups));
}
}
}
--- 3781,3788 ----
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups),
! false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 3868,3874 ****
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups));
}
else if (parse->groupClause)
{
--- 3834,3840 ----
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups), false);
}
else if (parse->groupClause)
{
*************** create_grouping_paths(PlannerInfo *root,
*** 3883,3889 ****
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups));
}
else
{
--- 3849,3855 ----
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups), false);
}
else
{
*************** create_grouping_paths(PlannerInfo *root,
*** 3932,3938 ****
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups));
else
add_path(grouped_rel, (Path *)
create_group_path(root,
--- 3898,3904 ----
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups), false);
else
add_path(grouped_rel, (Path *)
create_group_path(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 3941,3947 ****
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups));
/*
* The point of using Gather Merge rather than Gather is that it
--- 3907,3913 ----
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups), false);
/*
* The point of using Gather Merge rather than Gather is that it
*************** create_grouping_paths(PlannerInfo *root,
*** 3994,4000 ****
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups));
else
add_path(grouped_rel, (Path *)
create_group_path(root,
--- 3960,3966 ----
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups), false);
else
add_path(grouped_rel, (Path *)
create_group_path(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 4003,4009 ****
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups));
}
}
}
--- 3969,3975 ----
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups), false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 4048,4054 ****
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups));
}
}
--- 4014,4020 ----
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups), false);
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 4086,4094 ****
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups));
}
}
}
/* Give a helpful error if we failed to find any implementation */
--- 4052,4128 ----
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups), false);
}
}
+
+ /*
+ * If input_rel has partially aggregated partial paths, gather them
+ * and perform the final aggregation.
+ *
+ * TODO Allow havingQual - currently not supported at base relation
+ * level.
+ */
+ if (input_rel->gpi != NULL &&
+ input_rel->gpi->partial_pathlist != NIL &&
+ !parse->havingQual)
+ {
+ Path *path = (Path *) linitial(input_rel->gpi->partial_pathlist);
+ double total_groups = path->rows * path->parallel_workers;
+
+ path = (Path *) create_gather_path(root,
+ input_rel,
+ path,
+ path->pathtarget,
+ NULL,
+ &total_groups);
+
+ /*
+ * The input path is partially aggregated and the final
+ * aggregation - if the path wins - will be done below. So we're
+ * done with it for now.
+ *
+ * The top-level grouped_rel needs to receive the path into
+ * regular pathlist, as opposed grouped_rel->gpi->pathlist.
+ */
+
+ add_path(input_rel, path, false);
+ }
+
+ /*
+ * If input_rel has partially aggregated paths, perform the final
+ * aggregation.
+ *
+ * TODO Allow havingQual - currently not supported at base relation
+ * level.
+ */
+ if (input_rel->gpi != NULL && input_rel->gpi->pathlist != NIL &&
+ !parse->havingQual)
+ {
+ Path *pre_agg = (Path *) linitial(input_rel->gpi->pathlist);
+
+ dNumGroups = get_number_of_groups(root, pre_agg->rows, gd);
+
+ MemSet(&agg_final_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, (Node *) target->exprs,
+ AGGSPLIT_FINAL_DESERIAL,
+ &agg_final_costs);
+ get_agg_clause_costs(root, parse->havingQual,
+ AGGSPLIT_FINAL_DESERIAL,
+ &agg_final_costs);
+
+ add_path(grouped_rel,
+ (Path *) create_agg_path(root, grouped_rel,
+ pre_agg,
+ target,
+ AGG_HASHED,
+ AGGSPLIT_FINAL_DESERIAL,
+ parse->groupClause,
+ (List *) parse->havingQual,
+ &agg_final_costs,
+ dNumGroups),
+ false);
+ }
}
/* Give a helpful error if we failed to find any implementation */
*************** consider_groupingsets_paths(PlannerInfo
*** 4288,4294 ****
strat,
new_rollups,
agg_costs,
! dNumGroups));
return;
}
--- 4322,4328 ----
strat,
new_rollups,
agg_costs,
! dNumGroups), false);
return;
}
*************** consider_groupingsets_paths(PlannerInfo
*** 4446,4452 ****
AGG_MIXED,
rollups,
agg_costs,
! dNumGroups));
}
}
--- 4480,4486 ----
AGG_MIXED,
rollups,
agg_costs,
! dNumGroups), false);
}
}
*************** consider_groupingsets_paths(PlannerInfo
*** 4463,4469 ****
AGG_SORTED,
gd->rollups,
agg_costs,
! dNumGroups));
}
/*
--- 4497,4503 ----
AGG_SORTED,
gd->rollups,
agg_costs,
! dNumGroups), false);
}
/*
*************** create_one_window_path(PlannerInfo *root
*** 4648,4654 ****
window_pathkeys);
}
! add_path(window_rel, path);
}
/*
--- 4682,4688 ----
window_pathkeys);
}
! add_path(window_rel, path, false);
}
/*
*************** create_distinct_paths(PlannerInfo *root,
*** 4754,4760 ****
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows));
}
}
--- 4788,4794 ----
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows), false);
}
}
*************** create_distinct_paths(PlannerInfo *root,
*** 4781,4787 ****
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows));
}
/*
--- 4815,4821 ----
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows), false);
}
/*
*************** create_distinct_paths(PlannerInfo *root,
*** 4828,4834 ****
parse->distinctClause,
NIL,
NULL,
! numDistinctRows));
}
/* Give a helpful error if we failed to find any implementation */
--- 4862,4868 ----
parse->distinctClause,
NIL,
NULL,
! numDistinctRows), false);
}
/* Give a helpful error if we failed to find any implementation */
*************** create_ordered_paths(PlannerInfo *root,
*** 4926,4932 ****
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path);
}
}
--- 4960,4966 ----
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path, false);
}
}
*************** create_ordered_paths(PlannerInfo *root,
*** 4976,4982 ****
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path);
}
}
--- 5010,5016 ----
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path, false);
}
}
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
new file mode 100644
index cdb8e95..fca3c0b
*** a/src/backend/optimizer/plan/setrefs.c
--- b/src/backend/optimizer/plan/setrefs.c
*************** typedef struct
*** 40,45 ****
--- 40,46 ----
List *tlist; /* underlying target list */
int num_vars; /* number of plain Var tlist entries */
bool has_ph_vars; /* are there PlaceHolderVar entries? */
+ bool has_grp_vars; /* are there GroupedVar entries? */
bool has_non_vars; /* are there other entries? */
tlist_vinfo vars[FLEXIBLE_ARRAY_MEMBER]; /* has num_vars entries */
} indexed_tlist;
*************** set_upper_references(PlannerInfo *root,
*** 1725,1733 ****
--- 1726,1777 ----
indexed_tlist *subplan_itlist;
List *output_targetlist;
ListCell *l;
+ List *sub_tlist_save = NIL;
+
+ if (root->grouped_var_list != NIL)
+ {
+ if (IsA(plan, Agg))
+ {
+ Agg *agg = (Agg *) plan;
+
+ if (agg->aggsplit == AGGSPLIT_FINAL_DESERIAL)
+ {
+ /*
+ * convert_combining_aggrefs could have replaced some vars
+ * with Aggref expressions representing the partial
+ * aggregation. We need to restore the same Aggrefs in the
+ * subplan targetlist, but this would break the subplan if
+ * it's something else than the partial aggregation (i.e. the
+ * partial aggregation takes place lower in the plan tree). So
+ * we'll eventually need to restore the original list.
+ */
+ if (!IsA(subplan, Agg))
+ sub_tlist_save = subplan->targetlist;
+ #ifdef USE_ASSERT_CHECKING
+ else
+ Assert(((Agg *) subplan)->aggsplit == AGGSPLIT_INITIAL_SERIAL);
+ #endif /* USE_ASSERT_CHECKING */
+
+ /*
+ * Restore the aggregate expressions that we might have
+ * removed when planning for aggregation at base relation
+ * level.
+ */
+ subplan->targetlist =
+ restore_grouping_expressions(root, subplan->targetlist);
+ }
+ }
+ }
subplan_itlist = build_tlist_index(subplan->targetlist);
+ /*
+ * The replacement of GroupVars by Aggrefs was only needed for the index
+ * build.
+ */
+ if (sub_tlist_save != NIL)
+ subplan->targetlist = sub_tlist_save;
+
output_targetlist = NIL;
foreach(l, plan->targetlist)
{
*************** build_tlist_index(List *tlist)
*** 1937,1942 ****
--- 1981,1987 ----
itlist->tlist = tlist;
itlist->has_ph_vars = false;
+ itlist->has_grp_vars = false;
itlist->has_non_vars = false;
/* Find the Vars and fill in the index array */
*************** build_tlist_index(List *tlist)
*** 1956,1961 ****
--- 2001,2008 ----
}
else if (tle->expr && IsA(tle->expr, PlaceHolderVar))
itlist->has_ph_vars = true;
+ else if (tle->expr && IsA(tle->expr, GroupedVar))
+ itlist->has_grp_vars = true;
else
itlist->has_non_vars = true;
}
*************** fix_join_expr_mutator(Node *node, fix_jo
*** 2233,2238 ****
--- 2280,2310 ----
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, GroupedVar))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ /* See if the GroupedVar has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_grp_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) gvar,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_grp_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) gvar,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+
+ /* No referent found for GroupedVar */
+ elog(ERROR, "grouped variable not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
*************** fix_upper_expr_mutator(Node *node, fix_u
*** 2389,2395 ****
/* If no match, just fall through to process it normally */
}
/* Try matching more complex expressions too, if tlist has any */
! if (context->subplan_itlist->has_non_vars)
{
newvar = search_indexed_tlist_for_non_var((Expr *) node,
context->subplan_itlist,
--- 2461,2468 ----
/* If no match, just fall through to process it normally */
}
/* Try matching more complex expressions too, if tlist has any */
! if (context->subplan_itlist->has_grp_vars ||
! context->subplan_itlist->has_non_vars)
{
newvar = search_indexed_tlist_for_non_var((Expr *) node,
context->subplan_itlist,
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
new file mode 100644
index e327e66..e90d72f
*** a/src/backend/optimizer/prep/prepunion.c
--- b/src/backend/optimizer/prep/prepunion.c
*************** plan_set_operations(PlannerInfo *root)
*** 207,213 ****
root->processed_tlist = top_tlist;
/* Add only the final path to the SETOP upperrel. */
! add_path(setop_rel, path);
/* Let extensions possibly add some more paths */
if (create_upper_paths_hook)
--- 207,213 ----
root->processed_tlist = top_tlist;
/* Add only the final path to the SETOP upperrel. */
! add_path(setop_rel, path, false);
/* Let extensions possibly add some more paths */
if (create_upper_paths_hook)
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
new file mode 100644
index 8536212..39813b8
*** a/src/backend/optimizer/util/pathnode.c
--- b/src/backend/optimizer/util/pathnode.c
***************
*** 24,29 ****
--- 24,31 ----
#include "optimizer/paths.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+ /* TODO Remove this if get_grouping_expressions ends up in another module. */
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "parser/parsetree.h"
#include "utils/lsyscache.h"
*************** set_cheapest(RelOptInfo *parent_rel)
*** 409,416 ****
* Returns nothing, but modifies parent_rel->pathlist.
*/
void
! add_path(RelOptInfo *parent_rel, Path *new_path)
{
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
List *new_path_pathkeys;
--- 411,419 ----
* Returns nothing, but modifies parent_rel->pathlist.
*/
void
! add_path(RelOptInfo *parent_rel, Path *new_path, bool grouped)
{
+ List *pathlist;
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
List *new_path_pathkeys;
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 427,432 ****
--- 430,443 ----
/* Pretend parameterized paths have no pathkeys, per comment above */
new_path_pathkeys = new_path->param_info ? NIL : new_path->pathkeys;
+ if (!grouped)
+ pathlist = parent_rel->pathlist;
+ else
+ {
+ Assert(parent_rel->gpi != NULL);
+ pathlist = parent_rel->gpi->pathlist;
+ }
+
/*
* Loop to check proposed new path against old paths. Note it is possible
* for more than one old path to be tossed out because new_path dominates
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 436,442 ****
* list cell.
*/
p1_prev = NULL;
! for (p1 = list_head(parent_rel->pathlist); p1 != NULL; p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
bool remove_old = false; /* unless new proves superior */
--- 447,453 ----
* list cell.
*/
p1_prev = NULL;
! for (p1 = list_head(pathlist); p1 != NULL; p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
bool remove_old = false; /* unless new proves superior */
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 582,589 ****
*/
if (remove_old)
{
! parent_rel->pathlist = list_delete_cell(parent_rel->pathlist,
! p1, p1_prev);
/*
* Delete the data pointed-to by the deleted cell, if possible
--- 593,599 ----
*/
if (remove_old)
{
! pathlist = list_delete_cell(pathlist, p1, p1_prev);
/*
* Delete the data pointed-to by the deleted cell, if possible
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 614,622 ****
{
/* Accept the new path: insert it at proper place in pathlist */
if (insert_after)
! lappend_cell(parent_rel->pathlist, insert_after, new_path);
else
! parent_rel->pathlist = lcons(new_path, parent_rel->pathlist);
}
else
{
--- 624,637 ----
{
/* Accept the new path: insert it at proper place in pathlist */
if (insert_after)
! lappend_cell(pathlist, insert_after, new_path);
else
! pathlist = lcons(new_path, pathlist);
!
! if (!grouped)
! parent_rel->pathlist = pathlist;
! else
! parent_rel->gpi->pathlist = pathlist;
}
else
{
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 646,653 ****
bool
add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer)
{
List *new_path_pathkeys;
bool consider_startup;
ListCell *p1;
--- 661,669 ----
bool
add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer, bool grouped)
{
+ List *pathlist;
List *new_path_pathkeys;
bool consider_startup;
ListCell *p1;
*************** add_path_precheck(RelOptInfo *parent_rel
*** 656,664 ****
new_path_pathkeys = required_outer ? NIL : pathkeys;
/* Decide whether new path's startup cost is interesting */
! consider_startup = required_outer ? parent_rel->consider_param_startup : parent_rel->consider_startup;
! foreach(p1, parent_rel->pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
--- 672,689 ----
new_path_pathkeys = required_outer ? NIL : pathkeys;
/* Decide whether new path's startup cost is interesting */
! consider_startup = required_outer ? parent_rel->consider_param_startup :
! parent_rel->consider_startup;
! if (!grouped)
! pathlist = parent_rel->pathlist;
! else
! {
! Assert(parent_rel->gpi != NULL);
! pathlist = parent_rel->gpi->pathlist;
! }
!
! foreach(p1, pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
*************** add_path_precheck(RelOptInfo *parent_rel
*** 749,771 ****
* referenced by partial BitmapHeapPaths.
*/
void
! add_partial_path(RelOptInfo *parent_rel, Path *new_path)
{
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
ListCell *p1;
ListCell *p1_prev;
ListCell *p1_next;
/* Check for query cancel. */
CHECK_FOR_INTERRUPTS();
/*
* As in add_path, throw out any paths which are dominated by the new
* path, but throw out the new path if some existing path dominates it.
*/
p1_prev = NULL;
! for (p1 = list_head(parent_rel->partial_pathlist); p1 != NULL;
p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
--- 774,805 ----
* referenced by partial BitmapHeapPaths.
*/
void
! add_partial_path(RelOptInfo *parent_rel, Path *new_path, bool grouped)
{
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
ListCell *p1;
ListCell *p1_prev;
ListCell *p1_next;
+ List *pathlist;
/* Check for query cancel. */
CHECK_FOR_INTERRUPTS();
+ if (!grouped)
+ pathlist = parent_rel->partial_pathlist;
+ else
+ {
+ Assert(parent_rel->gpi != NULL);
+ pathlist = parent_rel->gpi->partial_pathlist;
+ }
+
/*
* As in add_path, throw out any paths which are dominated by the new
* path, but throw out the new path if some existing path dominates it.
*/
p1_prev = NULL;
! for (p1 = list_head(pathlist); p1 != NULL;
p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
*************** add_partial_path(RelOptInfo *parent_rel,
*** 819,830 ****
}
/*
! * Remove current element from partial_pathlist if dominated by new.
*/
if (remove_old)
{
! parent_rel->partial_pathlist =
! list_delete_cell(parent_rel->partial_pathlist, p1, p1_prev);
pfree(old_path);
/* p1_prev does not advance */
}
--- 853,863 ----
}
/*
! * Remove current element from pathlist if dominated by new.
*/
if (remove_old)
{
! pathlist = list_delete_cell(pathlist, p1, p1_prev);
pfree(old_path);
/* p1_prev does not advance */
}
*************** add_partial_path(RelOptInfo *parent_rel,
*** 839,845 ****
/*
* If we found an old path that dominates new_path, we can quit
! * scanning the partial_pathlist; we will not add new_path, and we
* assume new_path cannot dominate any later path.
*/
if (!accept_new)
--- 872,878 ----
/*
* If we found an old path that dominates new_path, we can quit
! * scanning the pathlist; we will not add new_path, and we
* assume new_path cannot dominate any later path.
*/
if (!accept_new)
*************** add_partial_path(RelOptInfo *parent_rel,
*** 850,859 ****
{
/* Accept the new path: insert it at proper place */
if (insert_after)
! lappend_cell(parent_rel->partial_pathlist, insert_after, new_path);
else
! parent_rel->partial_pathlist =
! lcons(new_path, parent_rel->partial_pathlist);
}
else
{
--- 883,896 ----
{
/* Accept the new path: insert it at proper place */
if (insert_after)
! lappend_cell(pathlist, insert_after, new_path);
else
! pathlist = lcons(new_path, pathlist);
!
! if (!grouped)
! parent_rel->partial_pathlist = pathlist;
! else
! parent_rel->gpi->partial_pathlist = pathlist;
}
else
{
*************** add_partial_path(RelOptInfo *parent_rel,
*** 874,882 ****
*/
bool
add_partial_path_precheck(RelOptInfo *parent_rel, Cost total_cost,
! List *pathkeys)
{
ListCell *p1;
/*
* Our goal here is twofold. First, we want to find out whether this path
--- 911,928 ----
*/
bool
add_partial_path_precheck(RelOptInfo *parent_rel, Cost total_cost,
! List *pathkeys, bool grouped)
{
ListCell *p1;
+ List *pathlist;
+
+ if (!grouped)
+ pathlist = parent_rel->partial_pathlist;
+ else
+ {
+ Assert(parent_rel->gpi != NULL);
+ pathlist = parent_rel->gpi->partial_pathlist;
+ }
/*
* Our goal here is twofold. First, we want to find out whether this path
*************** add_partial_path_precheck(RelOptInfo *pa
*** 886,895 ****
* final cost computations. If so, we definitely want to consider it.
*
* Unlike add_path(), we always compare pathkeys here. This is because we
! * expect partial_pathlist to be very short, and getting a definitive
! * answer at this stage avoids the need to call add_path_precheck.
*/
! foreach(p1, parent_rel->partial_pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
--- 932,942 ----
* final cost computations. If so, we definitely want to consider it.
*
* Unlike add_path(), we always compare pathkeys here. This is because we
! * expect partial_pathlist / grouped_pathlist to be very short, and
! * getting a definitive answer at this stage avoids the need to call
! * add_path_precheck.
*/
! foreach(p1, pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
*************** add_partial_path_precheck(RelOptInfo *pa
*** 918,924 ****
* completion.
*/
if (!add_path_precheck(parent_rel, total_cost, total_cost, pathkeys,
! NULL))
return false;
return true;
--- 965,971 ----
* completion.
*/
if (!add_path_precheck(parent_rel, total_cost, total_cost, pathkeys,
! NULL, grouped))
return false;
return true;
*************** calc_non_nestloop_required_outer(Path *o
*** 2056,2061 ****
--- 2103,2109 ----
* 'restrict_clauses' are the RestrictInfo nodes to apply at the join
* 'pathkeys' are the path keys of the new join path
* 'required_outer' is the set of required outer rels
+ * 'target' can be passed to override that of joinrel.
*
* Returns the resulting path node.
*/
*************** create_nestloop_path(PlannerInfo *root,
*** 2070,2076 ****
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer)
{
NestPath *pathnode = makeNode(NestPath);
Relids inner_req_outer = PATH_REQ_OUTER(inner_path);
--- 2118,2125 ----
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer,
! PathTarget *target)
{
NestPath *pathnode = makeNode(NestPath);
Relids inner_req_outer = PATH_REQ_OUTER(inner_path);
*************** create_nestloop_path(PlannerInfo *root,
*** 2103,2109 ****
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
! pathnode->path.pathtarget = joinrel->reltarget;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2152,2158 ----
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
! pathnode->path.pathtarget = target == NULL ? joinrel->reltarget : target;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_mergejoin_path(PlannerInfo *root,
*** 2160,2172 ****
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys)
{
MergePath *pathnode = makeNode(MergePath);
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = joinrel->reltarget;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2209,2223 ----
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys,
! PathTarget *target)
{
MergePath *pathnode = makeNode(MergePath);
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = target == NULL ? joinrel->reltarget :
! target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_mergejoin_path(PlannerInfo *root,
*** 2210,2215 ****
--- 2261,2267 ----
* 'required_outer' is the set of required outer rels
* 'hashclauses' are the RestrictInfo nodes to use as hash clauses
* (this should be a subset of the restrict_clauses list)
+ * 'target' can be passed to override that of joinrel.
*/
HashPath *
create_hashjoin_path(PlannerInfo *root,
*************** create_hashjoin_path(PlannerInfo *root,
*** 2222,2234 ****
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses)
{
HashPath *pathnode = makeNode(HashPath);
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = joinrel->reltarget;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2274,2288 ----
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses,
! PathTarget *target)
{
HashPath *pathnode = makeNode(HashPath);
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = target == NULL ? joinrel->reltarget :
! target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_agg_path(PlannerInfo *root,
*** 2682,2688 ****
pathnode->path.pathtarget = target;
/* For now, assume we are above any joins, so no parameterization */
pathnode->path.param_info = NULL;
! pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
pathnode->path.parallel_workers = subpath->parallel_workers;
--- 2736,2742 ----
pathnode->path.pathtarget = target;
/* For now, assume we are above any joins, so no parameterization */
pathnode->path.param_info = NULL;
! pathnode->path.parallel_aware = true;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
pathnode->path.parallel_workers = subpath->parallel_workers;
*************** create_agg_path(PlannerInfo *root,
*** 2713,2718 ****
--- 2767,2942 ----
}
/*
+ * Apply partial AGG_SORTED aggregation path to subpath if it's suitably
+ * sorted.
+ *
+ * first_call indicates whether the function is being called first time for
+ * given index --- since the target should not change, we can skip the check
+ * of sorting during subsequent calls.
+ *
+ * group_clauses, group_exprs and agg_exprs are pointers to lists we populate
+ * when called first time for particular index, and that user passes for
+ * subsequent calls.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+ AggPath *
+ create_partial_agg_sorted_path(PlannerInfo *root, Path *subpath,
+ bool first_call,
+ List **group_clauses, List **group_exprs,
+ List **agg_exprs, double input_rows)
+ {
+ RelOptInfo *rel;
+ AggClauseCosts agg_costs;
+ double dNumGroups;
+ AggPath *result = NULL;
+
+ rel = subpath->parent;
+ Assert(rel->gpi != NULL);
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ if (first_call)
+ {
+ ListCell *lc1;
+ List *key_subset = NIL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+ }
+
+ if (first_call)
+ get_grouping_expressions(root, rel->gpi->target, group_clauses,
+ group_exprs, agg_exprs);
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ Assert(*agg_exprs != NIL);
+ get_agg_clause_costs(root, (Node *) *agg_exprs, AGGSPLIT_INITIAL_SERIAL,
+ &agg_costs);
+
+ Assert(*group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, *group_exprs, input_rows, NULL);
+
+ /* TODO HAVING qual. */
+ Assert(*group_clauses != NIL);
+ result = create_agg_path(root, rel, subpath, rel->gpi->target, AGG_SORTED,
+ AGGSPLIT_INITIAL_SERIAL, *group_clauses, NIL,
+ &agg_costs, dNumGroups);
+
+ return result;
+ }
+
+ /*
+ * Appy partial AGG_HASHED aggregation to subpath.
+ *
+ * Arguments have the same meaning as those of create_agg_sorted_path.
+ *
+ */
+ AggPath *
+ create_partial_agg_hashed_path(PlannerInfo *root, Path *subpath,
+ bool first_call,
+ List **group_clauses, List **group_exprs,
+ List **agg_exprs, double input_rows)
+ {
+ RelOptInfo *rel;
+ bool can_hash;
+ AggClauseCosts agg_costs;
+ double dNumGroups;
+ Size hashaggtablesize;
+ Query *parse = root->parse;
+ AggPath *result = NULL;
+
+ rel = subpath->parent;
+ Assert(rel->gpi != NULL);
+
+ if (first_call)
+ {
+ /*
+ * Find one grouping clause per grouping column.
+ *
+ * All that create_agg_plan eventually needs of the clause is
+ * tleSortGroupRef, so we don't have to care that the clause
+ * expression might differ from texpr, in case texpr was derived from
+ * EC.
+ */
+ get_grouping_expressions(root, rel->gpi->target, group_clauses,
+ group_exprs, agg_exprs);
+ }
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ Assert(*agg_exprs != NIL);
+ get_agg_clause_costs(root, (Node *) *agg_exprs, AGGSPLIT_INITIAL_SERIAL,
+ &agg_costs);
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(*group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, *group_exprs, input_rows,
+ NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * Create the partial aggregation path.
+ */
+ Assert(*group_clauses != NIL);
+
+ result = create_agg_path(root, rel, subpath,
+ rel->gpi->target,
+ AGG_HASHED,
+ AGGSPLIT_INITIAL_SERIAL,
+ *group_clauses, NIL,
+ &agg_costs,
+ dNumGroups);
+
+ /*
+ * The agg path should require no fewer parameters than the plain
+ * one.
+ */
+ result->path.param_info = subpath->param_info;
+ }
+ }
+
+ return result;
+ }
+
+ /*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
*
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
new file mode 100644
index 7912df0..cc7f6d3
*** a/src/backend/optimizer/util/relnode.c
--- b/src/backend/optimizer/util/relnode.c
***************
*** 25,30 ****
--- 25,31 ----
#include "optimizer/plancat.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+ #include "optimizer/var.h"
#include "utils/hsearch.h"
*************** typedef struct JoinHashEntry
*** 35,41 ****
} JoinHashEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *input_rel);
static List *build_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outer_rel,
--- 36,42 ----
} JoinHashEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *input_rel, bool grouped);
static List *build_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outer_rel,
*************** build_simple_rel(PlannerInfo *root, int
*** 120,125 ****
--- 121,127 ----
rel->cheapest_parameterized_paths = NIL;
rel->direct_lateral_relids = NULL;
rel->lateral_relids = NULL;
+ rel->gpi = NULL;
rel->relid = relid;
rel->rtekind = rte->rtekind;
/* min_attr, max_attr, attr_needed, attr_widths are set below */
*************** build_join_rel(PlannerInfo *root,
*** 478,483 ****
--- 480,486 ----
inner_rel->direct_lateral_relids);
joinrel->lateral_relids = min_join_parameterization(root, joinrel->relids,
outer_rel, inner_rel);
+ joinrel->gpi = NULL;
joinrel->relid = 0; /* indicates not a baserel */
joinrel->rtekind = RTE_JOIN;
joinrel->min_attr = 0;
*************** build_join_rel(PlannerInfo *root,
*** 516,525 ****
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
! build_joinrel_tlist(root, joinrel, outer_rel);
! build_joinrel_tlist(root, joinrel, inner_rel);
add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
* sets of any PlaceHolderVars computed here to direct_lateral_relids, so
--- 519,535 ----
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
! build_joinrel_tlist(root, joinrel, outer_rel, false);
! build_joinrel_tlist(root, joinrel, inner_rel, false);
add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ /* Try to build grouped target. */
+ /*
+ * TODO Consider if placeholders make sense here. If not, also make the
+ * related code below conditional.
+ */
+ prepare_rel_for_grouping(root, joinrel);
+
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
* sets of any PlaceHolderVars computed here to direct_lateral_relids, so
*************** min_join_parameterization(PlannerInfo *r
*** 647,663 ****
*/
static void
build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *input_rel)
{
Relids relids = joinrel->relids;
ListCell *vars;
! foreach(vars, input_rel->reltarget->exprs)
{
Var *var = (Var *) lfirst(vars);
RelOptInfo *baserel;
int ndx;
/*
* Ignore PlaceHolderVars in the input tlists; we'll make our own
* decisions about whether to copy them.
--- 657,699 ----
*/
static void
build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *input_rel, bool grouped)
{
Relids relids = joinrel->relids;
+ PathTarget *input_target, *result;
ListCell *vars;
+ int i = -1;
! if (!grouped)
! {
! input_target = input_rel->reltarget;
! result = joinrel->reltarget;
! }
! else
! {
! if (input_rel->gpi != NULL)
! {
! input_target = input_rel->gpi->target;
! Assert(input_target != NULL);
! }
! else
! input_target = input_rel->reltarget;
!
! /* Caller should have initialized this. */
! Assert(joinrel->gpi != NULL);
!
! /* Default to the plain target. */
! result = joinrel->gpi->target;
! }
!
! foreach(vars, input_target->exprs)
{
Var *var = (Var *) lfirst(vars);
RelOptInfo *baserel;
int ndx;
+ i++;
+
/*
* Ignore PlaceHolderVars in the input tlists; we'll make our own
* decisions about whether to copy them.
*************** build_joinrel_tlist(PlannerInfo *root, R
*** 681,690 ****
ndx = var->varattno - baserel->min_attr;
if (bms_nonempty_difference(baserel->attr_needed[ndx], relids))
{
/* Yup, add it to the output */
! joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs, var);
/* Vars have cost zero, so no need to adjust reltarget->cost */
! joinrel->reltarget->width += baserel->attr_widths[ndx];
}
}
}
--- 717,740 ----
ndx = var->varattno - baserel->min_attr;
if (bms_nonempty_difference(baserel->attr_needed[ndx], relids))
{
+ Index sortgroupref = 0;
+
/* Yup, add it to the output */
! if (input_target->sortgrouprefs)
! sortgroupref = input_target->sortgrouprefs[i];
!
! /*
! * Even if not used for grouping in the input path (the input path
! * is not necessarily grouped), it might be useful for grouping
! * higher in the join tree.
! */
! if (sortgroupref == 0)
! sortgroupref = get_expr_sortgroupref(root, (Expr *) var);
!
! add_column_to_pathtarget(result, (Expr *) var, sortgroupref);
!
/* Vars have cost zero, so no need to adjust reltarget->cost */
! result->width += baserel->attr_widths[ndx];
}
}
}
*************** get_appendrel_parampathinfo(RelOptInfo *
*** 1360,1362 ****
--- 1410,1561 ----
return ppi;
}
+
+ /*
+ * If the relation can produce grouped paths, create GroupedPathInfo for it
+ * and create target for the grouped paths.
+ */
+ void
+ prepare_rel_for_grouping(PlannerInfo *root, RelOptInfo *rel)
+ {
+ List *rel_aggregates;
+ Relids rel_agg_attrs = NULL;
+ List *rel_agg_vars = NIL;
+ bool found_higher;
+ ListCell *lc;
+ PathTarget *target_grouped;
+
+ if (rel->relid > 0)
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return;
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the aggregate
+ * would receive different input at the base rel level.
+ *
+ * TODO For RELOPT_JOINREL, do not return if all the joins that can set
+ * any entry of the grouped target (do we need to postpone this check
+ * until the grouped target is available, and should create_grouped_target
+ * take care?) of this rel to NULL are provably below rel. (It's ok if rel
+ * is one of these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return;
+
+ /*
+ * Check if some aggregates can be evaluated in this relation's target,
+ * and collect all vars referenced by these aggregates.
+ */
+ rel_aggregates = NIL;
+ found_higher = false;
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = castNode(GroupedVarInfo, lfirst(lc));
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which typically means
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ Aggref *aggref = castNode(Aggref, gvi->gvexpr);
+
+ /*
+ * Accept the aggregate.
+ *
+ * GroupedVarInfo is more convenient for the next processing than
+ * Aggref, see add_aggregates_to_grouped_target.
+ */
+ rel_aggregates = lappend(rel_aggregates, gvi);
+
+ if (rel->relid > 0)
+ {
+ /*
+ * Simple relation. Collect attributes referenced by the
+ * aggregate arguments.
+ */
+ pull_varattnos((Node *) aggref, rel->relid, &rel_agg_attrs);
+ }
+ else
+ {
+ List *agg_vars;
+
+ /*
+ * Join. Collect vars referenced by the aggregate
+ * arguments.
+ */
+ /*
+ * TODO Can any argument contain PHVs? And if so, does it matter?
+ * Consider PVC_INCLUDE_PLACEHOLDERS | PVC_RECURSE_PLACEHOLDERS.
+ */
+ agg_vars = pull_var_clause((Node *) aggref,
+ PVC_RECURSE_AGGREGATES);
+ rel_agg_vars = list_concat(rel_agg_vars, agg_vars);
+ }
+ }
+ else if (bms_overlap(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * Remember that there is at least one aggregate that needs more
+ * than this rel.
+ */
+ found_higher = true;
+ }
+ }
+
+ /*
+ * Grouping makes little sense w/o aggregate function.
+ */
+ if (rel_aggregates == NIL)
+ {
+ bms_free(rel_agg_attrs);
+ return;
+ }
+
+ if (found_higher)
+ {
+ /*
+ * If some aggregate(s) need only this rel but some other need
+ * multiple relations including the the current one, grouping of the
+ * current rel could steal some input variables from the "higher
+ * aggregate" (besides decreasing the number of input rows).
+ */
+ list_free(rel_aggregates);
+ bms_free(rel_agg_attrs);
+ return;
+ }
+
+ /*
+ * If rel->reltarget can be used for aggregation, mark the relation as
+ * capable of grouping.
+ */
+ Assert(rel->gpi == NULL);
+ target_grouped = create_grouped_target(root, rel, rel_agg_attrs,
+ rel_agg_vars);
+ if (target_grouped != NULL)
+ {
+ GroupedPathInfo *gpi;
+
+ gpi = makeNode(GroupedPathInfo);
+ gpi->target = copy_pathtarget(target_grouped);
+ gpi->pathlist = NIL;
+ gpi->partial_pathlist = NIL;
+ rel->gpi = gpi;
+
+ /*
+ * Add aggregates (in the form of GroupedVar) to the target.
+ */
+ add_aggregates_to_target(root, gpi->target, rel_aggregates, rel);
+ }
+ }
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
new file mode 100644
index 0952385..dd962b7
*** a/src/backend/optimizer/util/tlist.c
--- b/src/backend/optimizer/util/tlist.c
*************** get_sortgrouplist_exprs(List *sgClauses,
*** 408,413 ****
--- 408,487 ----
return result;
}
+ /*
+ * get_sortgrouplist_clauses
+ *
+ * Given a "grouped target" (i.e. target where each non-GroupedVar
+ * element must have sortgroupref set), build a list of the referencing
+ * SortGroupClauses, a list of the corresponding grouping expressions and
+ * a list of aggregate expressions.
+ */
+ /* Refine the function name. */
+ void
+ get_grouping_expressions(PlannerInfo *root, PathTarget *target,
+ List **grouping_clauses, List **grouping_exprs,
+ List **agg_exprs)
+ {
+ ListCell *l;
+ int i = 0;
+
+ foreach(l, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(l);
+
+ /* The target should contain at least one grouping column. */
+ Assert(target->sortgrouprefs != NULL);
+
+ if (IsA(texpr, GroupedVar))
+ {
+ /*
+ * texpr should represent the first aggregate in the targetlist.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides aggregates, the target should contain no expressions w/o
+ * sortgroupref. Plain relation being joined to grouped can have
+ * sortgroupref equal to zero for expressions contained neither in
+ * grouping expression nor in aggregate arguments, but if the target
+ * contains such an expression, it shouldn't be used for aggregation
+ * --- see can_aggregate field of GroupedPathInfo.
+ */
+ Assert(sortgroupref > 0);
+
+ cl = get_sortgroupref_clause(sortgroupref, root->parse->groupClause);
+ *grouping_clauses = list_append_unique(*grouping_clauses, cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ *grouping_exprs = list_append_unique(*grouping_exprs, texpr);
+ }
+
+ /* Now collect the aggregates. */
+ while (l != NULL)
+ {
+ GroupedVar *gvar = castNode(GroupedVar, lfirst(l));
+
+ /* Currently, GroupedVarInfo can only represent aggregate. */
+ Assert(gvar->agg_partial != NULL);
+ *agg_exprs = lappend(*agg_exprs, gvar->agg_partial);
+ l = lnext(l);
+ }
+ }
+
/*****************************************************************************
* Functions to extract data from a list of SortGroupClauses
*************** apply_pathtarget_labeling_to_tlist(List
*** 783,788 ****
--- 857,1081 ----
}
/*
+ * Replace each "grouped var" in the source targetlist with the original
+ * expression.
+ *
+ * TODO Think of more suitable name. Although "grouped var" may substitute for
+ * grouping expressions in the future, currently Aggref is the only outcome of
+ * the replacement. undo_grouped_var_substitutions?
+ */
+ List *
+ restore_grouping_expressions(PlannerInfo *root, List *src)
+ {
+ List *result = NIL;
+ ListCell *l;
+
+ foreach(l, src)
+ {
+ TargetEntry *te, *te_new;
+ Aggref *expr_new = NULL;
+
+ te = castNode(TargetEntry, lfirst(l));
+
+ if (IsA(te->expr, GroupedVar))
+ {
+ GroupedVar *gvar;
+
+ gvar = castNode(GroupedVar, te->expr);
+ expr_new = gvar->agg_partial;
+ }
+
+ if (expr_new != NULL)
+ {
+ te_new = flatCopyTargetEntry(te);
+ te_new->expr = (Expr *) expr_new;
+ }
+ else
+ te_new = te;
+ result = lappend(result, te_new);
+ }
+
+ return result;
+ }
+
+ /*
+ * For each aggregate add GroupedVar to target if "vars" is true, or the
+ * Aggref (marked as partial) if "vars" is false.
+ *
+ * If caller passes the aggregates, he must do so in the form of
+ * GroupedVarInfos so that we don't have to look for gvid. If NULL is passed,
+ * the function retrieves the suitable aggregates itself.
+ *
+ * List of the aggregates added is returned. This is only useful if the
+ * function had to retrieve the aggregates itself (i.e. NIL was passed for
+ * aggregates) -- caller is expected to do extra checks in that case (and to
+ * also free the list).
+ */
+ List *
+ add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *aggregates, RelOptInfo *rel)
+ {
+ ListCell *lc;
+ GroupedVarInfo *gvi;
+
+ if (aggregates == NIL)
+ {
+ /* Caller should pass the aggregates for base relation. */
+ Assert(rel->reloptkind != RELOPT_BASEREL);
+
+ /* Collect all aggregates that this rel can evaluate. */
+ foreach(lc, root->grouped_var_list)
+ {
+ gvi = castNode(GroupedVarInfo, lfirst(lc));
+
+ /*
+ * Overlap is not guarantee of correctness alone, but caller needs
+ * to do additional checks, so we're optimistic here.
+ *
+ * If gv_eval_at is NULL, the underlying Aggref should have
+ * aggstar set.
+ */
+ if (bms_overlap(gvi->gv_eval_at, rel->relids) ||
+ gvi->gv_eval_at == NULL)
+ aggregates = lappend(aggregates, gvi);
+ }
+
+ if (aggregates == NIL)
+ return NIL;
+ }
+
+ /* Create the vars and add them to the target. */
+ foreach(lc, aggregates)
+ {
+ GroupedVar *gvar;
+
+ gvi = castNode(GroupedVarInfo, lfirst(lc));
+ gvar = makeNode(GroupedVar);
+ gvar->gvid = gvi->gvid;
+ gvar->gvexpr = gvi->gvexpr;
+ gvar->agg_partial = gvi->agg_partial;
+ add_new_column_to_pathtarget(target, (Expr *) gvar);
+ }
+
+ return aggregates;
+ }
+
+ /*
+ * Return ressortgroupref of the target entry that is either equal to the
+ * expression or exists in the same equivalence class.
+ */
+ Index
+ get_expr_sortgroupref(PlannerInfo *root, Expr *expr)
+ {
+ ListCell *lc;
+ Index sortgroupref;
+
+ /*
+ * First, check if the query group clause contains exactly this
+ * expression.
+ */
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = castNode(TargetEntry, lfirst(lc));
+
+ if (equal(expr, te->expr) && te->ressortgroupref > 0)
+ return te->ressortgroupref;
+ }
+
+ /*
+ * If exactly this expression is not there, check if a grouping clause
+ * exists that belongs to the same equivalence class as the expression.
+ */
+ foreach(lc, root->group_pathkeys)
+ {
+ PathKey *pk = castNode(PathKey, lfirst(lc));
+ EquivalenceClass *ec = pk->pk_eclass;
+ ListCell *lm;
+ EquivalenceMember *em;
+ Expr *em_expr = NULL;
+ Query *query = root->parse;
+
+ /*
+ * Single-member EC cannot provide us with additional expression.
+ */
+ if (list_length(ec->ec_members) < 2)
+ continue;
+
+ /* We need equality anywhere in the join tree. */
+ if (ec->ec_below_outer_join)
+ continue;
+
+ /*
+ * TODO Reconsider this restriction. As the grouping expression is
+ * only evaluated at the relation level (and only the result will be
+ * propagated to the final targetlist), volatile function might be
+ * o.k. Need to think what volatile EC exactly means.
+ */
+ if (ec->ec_has_volatile)
+ continue;
+
+ foreach(lm, ec->ec_members)
+ {
+ em = (EquivalenceMember *) lfirst(lm);
+
+ /* The EC has !ec_below_outer_join. */
+ Assert(!em->em_nullable_relids);
+ if (equal(em->em_expr, expr))
+ {
+ em_expr = (Expr *) em->em_expr;
+ break;
+ }
+ }
+
+ if (em_expr == NULL)
+ /* Go for the next EC. */
+ continue;
+
+ /*
+ * Find the corresponding SortGroupClause, which provides us with
+ * sortgroupref. (It can belong to any EC member.)
+ */
+ sortgroupref = 0;
+ foreach(lm, ec->ec_members)
+ {
+ ListCell *lsg;
+
+ em = (EquivalenceMember *) lfirst(lm);
+ foreach(lsg, query->groupClause)
+ {
+ SortGroupClause *sgc;
+ Expr *expr;
+
+ sgc = (SortGroupClause *) lfirst(lsg);
+ expr = (Expr *) get_sortgroupclause_expr(sgc,
+ query->targetList);
+ if (equal(em->em_expr, expr))
+ {
+ Assert(sgc->tleSortGroupRef > 0);
+ sortgroupref = sgc->tleSortGroupRef;
+ break;
+ }
+ }
+
+ if (sortgroupref > 0)
+ break;
+ }
+
+ /*
+ * Since we searched in group_pathkeys, at least one EM of this EC
+ * should correspond to a SortGroupClause, otherwise the EC could
+ * not exist at all.
+ */
+ Assert(sortgroupref > 0);
+
+ return sortgroupref;
+ }
+
+ /* No EC found in group_pathkeys. */
+ return 0;
+ }
+
+ /*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
*
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
new file mode 100644
index 0c1a201..f4639c4
*** a/src/backend/utils/adt/ruleutils.c
--- b/src/backend/utils/adt/ruleutils.c
*************** get_rule_expr(Node *node, deparse_contex
*** 7499,7504 ****
--- 7499,7512 ----
get_agg_expr((Aggref *) node, context, (Aggref *) node);
break;
+ case T_GroupedVar:
+ {
+ GroupedVar *gvar = castNode(GroupedVar, node);
+
+ get_agg_expr(gvar->agg_partial, context, (Aggref *) gvar->gvexpr);
+ break;
+ }
+
case T_GroupingFunc:
{
GroupingFunc *gexpr = (GroupingFunc *) node;
*************** get_agg_combine_expr(Node *node, deparse
*** 8933,8942 ****
Aggref *aggref;
Aggref *original_aggref = private;
! if (!IsA(node, Aggref))
elog(ERROR, "combining Aggref does not point to an Aggref");
- aggref = (Aggref *) node;
get_agg_expr(aggref, context, original_aggref);
}
--- 8941,8958 ----
Aggref *aggref;
Aggref *original_aggref = private;
! if (IsA(node, Aggref))
! aggref = (Aggref *) node;
! else if (IsA(node, GroupedVar))
! {
! GroupedVar *gvar = castNode(GroupedVar, node);
!
! aggref = gvar->agg_partial;
! original_aggref = castNode(Aggref, gvar->gvexpr);
! }
! else
elog(ERROR, "combining Aggref does not point to an Aggref");
get_agg_expr(aggref, context, original_aggref);
}
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
new file mode 100644
index 5c382a2..1dd2d73
*** a/src/backend/utils/adt/selfuncs.c
--- b/src/backend/utils/adt/selfuncs.c
***************
*** 113,118 ****
--- 113,119 ----
#include "catalog/pg_statistic_ext.h"
#include "catalog/pg_type.h"
#include "executor/executor.h"
+ #include "executor/nodeAgg.h"
#include "mb/pg_wchar.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
*************** estimate_num_groups(PlannerInfo *root, L
*** 3473,3479 ****
/*
* Sanity check --- don't divide by zero if empty relation.
*/
! Assert(rel->reloptkind == RELOPT_BASEREL);
if (rel->tuples > 0)
{
/*
--- 3474,3481 ----
/*
* Sanity check --- don't divide by zero if empty relation.
*/
! Assert(rel->reloptkind == RELOPT_BASEREL ||
! rel->reloptkind == RELOPT_OTHER_MEMBER_REL);
if (rel->tuples > 0)
{
/*
*************** estimate_hash_bucketsize(PlannerInfo *ro
*** 3704,3709 ****
--- 3706,3744 ----
return (Selectivity) estfract;
}
+ /*
+ * estimate_hashagg_tablesize
+ * estimate the number of bytes that a hash aggregate hashtable will
+ * require based on the agg_costs, path width and dNumGroups.
+ *
+ * XXX this may be over-estimating the size now that hashagg knows to omit
+ * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
+ * grouping columns not in the hashed set are counted here even though hashagg
+ * won't store them. Is this a problem?
+ */
+ Size
+ estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
+ double dNumGroups)
+ {
+ Size hashentrysize;
+
+ /* Estimate per-hash-entry space at tuple width... */
+ hashentrysize = MAXALIGN(path->pathtarget->width) +
+ MAXALIGN(SizeofMinimalTupleHeader);
+
+ /* plus space for pass-by-ref transition values... */
+ hashentrysize += agg_costs->transitionSpace;
+ /* plus the per-hash-entry overhead */
+ hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
+
+ /*
+ * Note that this disregards the effect of fill-factor and growth policy
+ * of the hash-table. That's probably ok, given default the default
+ * fill-factor is relatively high. It'd be hard to meaningfully factor in
+ * "double-in-size" growth policies here.
+ */
+ return hashentrysize * dNumGroups;
+ }
/*-------------------------------------------------------------------------
*
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
new file mode 100644
index 177853b..df98ef7
*** a/src/include/nodes/nodes.h
--- b/src/include/nodes/nodes.h
*************** typedef enum NodeTag
*** 217,222 ****
--- 217,223 ----
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_GroupedPathInfo,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
*************** typedef enum NodeTag
*** 257,266 ****
--- 258,269 ----
T_PathTarget,
T_RestrictInfo,
T_PlaceHolderVar,
+ T_GroupedVar,
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PartitionedChildRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
new file mode 100644
index ebf9480..90588d9
*** a/src/include/nodes/relation.h
--- b/src/include/nodes/relation.h
*************** typedef struct PlannerInfo
*** 256,261 ****
--- 256,263 ----
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
*************** typedef struct PlannerInfo
*** 401,406 ****
--- 403,410 ----
* direct_lateral_relids - rels this rel has direct LATERAL references to
* lateral_relids - required outer rels for LATERAL, as a Relids set
* (includes both direct and indirect lateral references)
+ * gpi - GroupedPathInfo if the relation can produce grouped paths, NULL
+ * otherwise.
*
* If the relation is a base relation it will have these fields set:
*
*************** typedef struct RelOptInfo
*** 518,523 ****
--- 522,530 ----
Relids direct_lateral_relids; /* rels directly laterally referenced */
Relids lateral_relids; /* minimum parameterization of rel */
+ /* Information needed to produce grouped paths. */
+ struct GroupedPathInfo *gpi;
+
/* information about a base rel (not set for join rels!) */
Index relid;
Oid reltablespace; /* containing tablespace */
*************** typedef struct ParamPathInfo
*** 878,883 ****
--- 885,912 ----
List *ppi_clauses; /* join clauses available from outer rels */
} ParamPathInfo;
+ /*
+ * GroupedPathInfo
+ *
+ * If RelOptInfo points to this structure, grouped paths can be created for
+ * it.
+ *
+ * "target" will be used as pathtarget of grouped paths produced by this
+ * relation. Grouped path is either a result of aggregation of the relation
+ * that owns this structure or, if the owning relation is a join, a join path
+ * whose one side is a grouped path and the other is a plain (i.e. not
+ * grouped) one. (Two grouped paths cannot be joined in general because
+ * grouping of one side of the join essentially reduces occurrence of groups
+ * of the other side in the input of the final aggregation.)
+ */
+ typedef struct GroupedPathInfo
+ {
+ NodeTag type;
+
+ PathTarget *target; /* output of grouped paths. */
+ List *pathlist; /* List of grouped paths. */
+ List *partial_pathlist; /* List of partial grouped paths. */
+ } GroupedPathInfo;
/*
* Type "Path" is used as-is for sequential-scan paths, as well as some other
*************** typedef struct PlaceHolderVar
*** 1806,1811 ****
--- 1835,1873 ----
Index phlevelsup; /* > 0 if PHV belongs to outer query */
} PlaceHolderVar;
+
+ /*
+ * Similar to the concept of PlaceHolderVar, we treat aggregates and grouping
+ * columns as special variables if grouping is possible below the top-level
+ * join. The reason is that aggregates having start as the argument can be
+ * evaluated at various places in the join tree (i.e. cannot be assigned to
+ * target list of exactly one relation). Also this concept seems to be less
+ * invasive than adding the grouped vars to reltarget (in which case
+ * attr_needed and attr_widths arrays of RelOptInfo) would also need
+ * additional changes.
+ *
+ * gvexpr is a pointer to gvexpr field of the corresponding instance
+ * GroupedVarInfo. It's there for the sake of exprType(), exprCollation(),
+ * etc.
+ *
+ * agg_partial also points to the corresponding field of GroupedVarInfo if the
+ * GroupedVar is in the target of a parent relation (RELOPT_BASEREL). However
+ * within a child relation's (RELOPT_OTHER_MEMBER_REL) target it points to a
+ * copy which has argument expressions translated, so they no longer reference
+ * the parent.
+ *
+ * XXX Currently we only create GroupedVar for aggregates, but sometime we can
+ * do it for grouping keys as well. That would allow grouping below the
+ * top-level join by keys other than plain Var.
+ */
+ typedef struct GroupedVar
+ {
+ Expr xpr;
+ Expr *gvexpr; /* the represented expression */
+ Aggref *agg_partial; /* partial aggregate if gvexpr is aggregate */
+ Index gvid; /* GroupedVarInfo */
+ } GroupedVar;
+
/*
* "Special join" info.
*
*************** typedef struct PlaceHolderInfo
*** 2021,2026 ****
--- 2083,2104 ----
} PlaceHolderInfo;
/*
+ * Likewise, GroupedVarInfo exists for each distinct GroupedVar.
+ */
+ typedef struct GroupedVarInfo
+ {
+ NodeTag type;
+
+ Index gvid; /* GroupedVar.gvid */
+ Expr *gvexpr; /* the represented expression. */
+ Aggref *agg_partial; /* if gvexpr is aggregate, agg_partial is
+ * the corresponding partial aggregate */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+ int32 gv_width; /* estimated width of the expression */
+ } GroupedVarInfo;
+
+ /*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
* path, the list is stored into root->minmax_aggs for use during setrefs.c.
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
new file mode 100644
index 82d4e87..91f0a57
*** a/src/include/optimizer/pathnode.h
--- b/src/include/optimizer/pathnode.h
*************** extern int compare_path_costs(Path *path
*** 25,37 ****
extern int compare_fractional_path_costs(Path *path1, Path *path2,
double fraction);
extern void set_cheapest(RelOptInfo *parent_rel);
! extern void add_path(RelOptInfo *parent_rel, Path *new_path);
extern bool add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer);
! extern void add_partial_path(RelOptInfo *parent_rel, Path *new_path);
extern bool add_partial_path_precheck(RelOptInfo *parent_rel,
! Cost total_cost, List *pathkeys);
extern Path *create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
Relids required_outer, int parallel_workers);
--- 25,39 ----
extern int compare_fractional_path_costs(Path *path1, Path *path2,
double fraction);
extern void set_cheapest(RelOptInfo *parent_rel);
! extern void add_path(RelOptInfo *parent_rel, Path *new_path, bool grouped);
extern bool add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer, bool grouped);
! extern void add_partial_path(RelOptInfo *parent_rel, Path *new_path,
! bool grouped);
extern bool add_partial_path_precheck(RelOptInfo *parent_rel,
! Cost total_cost, List *pathkeys,
! bool grouped);
extern Path *create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
Relids required_outer, int parallel_workers);
*************** extern NestPath *create_nestloop_path(Pl
*** 125,131 ****
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer);
extern MergePath *create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
--- 127,134 ----
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer,
! PathTarget *target);
extern MergePath *create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
*************** extern MergePath *create_mergejoin_path(
*** 139,145 ****
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys);
extern HashPath *create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
--- 142,149 ----
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys,
! PathTarget *target);
extern HashPath *create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
*************** extern HashPath *create_hashjoin_path(Pl
*** 151,157 ****
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses);
extern ProjectionPath *create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
--- 155,162 ----
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses,
! PathTarget *target);
extern ProjectionPath *create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
*************** extern AggPath *create_agg_path(PlannerI
*** 192,197 ****
--- 197,216 ----
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+ extern AggPath *create_partial_agg_sorted_path(PlannerInfo *root,
+ Path *subpath,
+ bool first_call,
+ List **group_clauses,
+ List **group_exprs,
+ List **agg_exprs,
+ double input_rows);
+ extern AggPath *create_partial_agg_hashed_path(PlannerInfo *root,
+ Path *subpath,
+ bool first_call,
+ List **group_clauses,
+ List **group_exprs,
+ List **agg_exprs,
+ double input_rows);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
*************** extern ParamPathInfo *get_joinrel_paramp
*** 288,292 ****
--- 307,312 ----
List **restrict_clauses);
extern ParamPathInfo *get_appendrel_parampathinfo(RelOptInfo *appendrel,
Relids required_outer);
+ extern void prepare_rel_for_grouping(PlannerInfo *root, RelOptInfo *rel);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
new file mode 100644
index 25fe78c..38967da
*** a/src/include/optimizer/paths.h
--- b/src/include/optimizer/paths.h
*************** extern void set_dummy_rel_pathlist(RelOp
*** 53,59 ****
extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
List *initial_rels);
! extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
--- 53,64 ----
extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
List *initial_rels);
! extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
! bool grouped);
!
! extern void create_grouped_path(PlannerInfo *root, RelOptInfo *rel,
! Path *subpath, bool precheck, bool partial,
! AggStrategy aggstrategy);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
*************** extern void debug_print_rel(PlannerInfo
*** 67,73 ****
* indxpath.c
* routines to generate index paths
*/
! extern void create_index_paths(PlannerInfo *root, RelOptInfo *rel);
extern bool relation_has_unique_index_for(PlannerInfo *root, RelOptInfo *rel,
List *restrictlist,
List *exprlist, List *oprlist);
--- 72,79 ----
* indxpath.c
* routines to generate index paths
*/
! extern void create_index_paths(PlannerInfo *root, RelOptInfo *rel,
! bool grouped);
extern bool relation_has_unique_index_for(PlannerInfo *root, RelOptInfo *rel,
List *restrictlist,
List *exprlist, List *oprlist);
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
new file mode 100644
index 94ef84b..159ccff
*** a/src/include/optimizer/planmain.h
--- b/src/include/optimizer/planmain.h
*************** extern int join_collapse_limit;
*** 74,80 ****
extern void add_base_rels_to_query(PlannerInfo *root, Node *jtnode);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
! Relids where_needed, bool create_new_ph);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
--- 74,82 ----
extern void add_base_rels_to_query(PlannerInfo *root, Node *jtnode);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
! Relids where_needed, bool create_new_ph);
! extern void add_grouping_info_to_base_rels(PlannerInfo *root);
! extern void add_grouped_vars_to_rels(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
new file mode 100644
index ccb93d8..ddea03c
*** a/src/include/optimizer/tlist.h
--- b/src/include/optimizer/tlist.h
*************** extern Node *get_sortgroupclause_expr(So
*** 41,46 ****
--- 41,49 ----
List *targetList);
extern List *get_sortgrouplist_exprs(List *sgClauses,
List *targetList);
+ extern void get_grouping_expressions(PlannerInfo *root, PathTarget *target,
+ List **grouping_clauses,
+ List **grouping_exprs, List **agg_exprs);
extern SortGroupClause *get_sortgroupref_clause(Index sortref,
List *clauses);
*************** extern void split_pathtarget_at_srfs(Pla
*** 65,70 ****
--- 68,84 ----
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+ /* TODO Find the best location (position and in some cases even file) for the
+ * following ones. */
+ extern List *restore_grouping_expressions(PlannerInfo *root, List *src);
+ extern List *add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *aggregates, RelOptInfo *rel);
+ extern Index get_expr_sortgroupref(PlannerInfo *root, Expr *expr);
+ /* TODO Move definition from initsplan.c to tlist.c. */
+ extern PathTarget *create_grouped_target(PlannerInfo *root, RelOptInfo *rel,
+ Relids rel_agg_attrs,
+ List *rel_agg_vars);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
diff --git a/src/include/utils/selfuncs.h b/src/include/utils/selfuncs.h
new file mode 100644
index 9f9d2dc..e05e6f6
*** a/src/include/utils/selfuncs.h
--- b/src/include/utils/selfuncs.h
*************** extern double estimate_num_groups(Planne
*** 206,211 ****
--- 206,214 ----
extern Selectivity estimate_hash_bucketsize(PlannerInfo *root, Node *hashkey,
double nbuckets);
+ extern Size estimate_hashagg_tablesize(Path *path,
+ const AggClauseCosts *agg_costs,
+ double dNumGroups);
extern List *deconstruct_indexquals(IndexPath *path);
extern void genericcostestimate(PlannerInfo *root, IndexPath *path,
Re: WIP: Aggregation push-down
Antonin Houska <ah@cybertec.at> wrote:
This is a new version of the patch I presented in [1].
Rebased.
cat .git/refs/heads/master
b9a3ef55b253d885081c2d0e9dc45802cab71c7b
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de, http://www.cybertec.at
Attachments:
agg_pushdown_v2.difftext/x-diffDownload
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
new file mode 100644
index 5a34a46..717763d
*** a/src/backend/executor/execExpr.c
--- b/src/backend/executor/execExpr.c
*************** ExecInitExprRec(Expr *node, PlanState *p
*** 723,728 ****
--- 723,755 ----
break;
}
+ case T_GroupedVar:
+ /*
+ * GroupedVar is treated as an aggregate if it appears in the
+ * targetlist of Agg node, but as a normal variable elsewhere.
+ */
+ if (parent && (IsA(parent, AggState)))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ /*
+ * Currently GroupedVar can only represent partial aggregate.
+ */
+ Assert(gvar->agg_partial != NULL);
+
+ ExecInitExprRec((Expr *) gvar->agg_partial, parent, state,
+ resv, resnull);
+ break;
+ }
+ else
+ {
+ /*
+ * set_plan_refs should have replaced GroupedVar in the
+ * targetlist with an ordinary Var.
+ */
+ elog(ERROR, "parent of GroupedVar is not Agg node");
+ }
+
case T_GroupingFunc:
{
GroupingFunc *grp_node = (GroupingFunc *) node;
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
new file mode 100644
index c2b8618..c4cb4c0
*** a/src/backend/executor/nodeAgg.c
--- b/src/backend/executor/nodeAgg.c
*************** find_unaggregated_cols_walker(Node *node
*** 1829,1834 ****
--- 1829,1845 ----
/* do not descend into aggregate exprs */
return false;
}
+ if (IsA(node, GroupedVar))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ /*
+ * GroupedVar is currently used only for partial aggregation, so treat
+ * it like an Aggref above.
+ */
+ Assert(gvar->agg_partial != NULL);
+ return false;
+ }
return expression_tree_walker(node, find_unaggregated_cols_walker,
(void *) colnos);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
new file mode 100644
index 00a0fed..7d188ea
*** a/src/backend/nodes/copyfuncs.c
--- b/src/backend/nodes/copyfuncs.c
*************** _copyPlaceHolderVar(const PlaceHolderVar
*** 2206,2211 ****
--- 2206,2226 ----
}
/*
+ * _copyGroupedVar
+ */
+ static GroupedVar *
+ _copyGroupedVar(const GroupedVar *from)
+ {
+ GroupedVar *newnode = makeNode(GroupedVar);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(gvid);
+
+ return newnode;
+ }
+
+ /*
* _copySpecialJoinInfo
*/
static SpecialJoinInfo *
*************** copyObjectImpl(const void *from)
*** 4984,4989 ****
--- 4999,5007 ----
case T_PlaceHolderVar:
retval = _copyPlaceHolderVar(from);
break;
+ case T_GroupedVar:
+ retval = _copyGroupedVar(from);
+ break;
case T_SpecialJoinInfo:
retval = _copySpecialJoinInfo(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
new file mode 100644
index 46573ae..f1dacd5
*** a/src/backend/nodes/equalfuncs.c
--- b/src/backend/nodes/equalfuncs.c
*************** _equalPlaceHolderVar(const PlaceHolderVa
*** 874,879 ****
--- 874,887 ----
}
static bool
+ _equalGroupedVar(const GroupedVar *a, const GroupedVar *b)
+ {
+ COMPARE_SCALAR_FIELD(gvid);
+
+ return true;
+ }
+
+ static bool
_equalSpecialJoinInfo(const SpecialJoinInfo *a, const SpecialJoinInfo *b)
{
COMPARE_BITMAPSET_FIELD(min_lefthand);
*************** equal(const void *a, const void *b)
*** 3148,3153 ****
--- 3156,3164 ----
case T_PlaceHolderVar:
retval = _equalPlaceHolderVar(a, b);
break;
+ case T_GroupedVar:
+ retval = _equalGroupedVar(a, b);
+ break;
case T_SpecialJoinInfo:
retval = _equalSpecialJoinInfo(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
new file mode 100644
index 3e8189c..5c00e55
*** a/src/backend/nodes/nodeFuncs.c
--- b/src/backend/nodes/nodeFuncs.c
*************** exprType(const Node *expr)
*** 259,264 ****
--- 259,267 ----
case T_PlaceHolderVar:
type = exprType((Node *) ((const PlaceHolderVar *) expr)->phexpr);
break;
+ case T_GroupedVar:
+ type = exprType((Node *) ((const GroupedVar *) expr)->agg_partial);
+ break;
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(expr));
type = InvalidOid; /* keep compiler quiet */
*************** exprCollation(const Node *expr)
*** 931,936 ****
--- 934,942 ----
case T_PlaceHolderVar:
coll = exprCollation((Node *) ((const PlaceHolderVar *) expr)->phexpr);
break;
+ case T_GroupedVar:
+ coll = exprCollation((Node *) ((const GroupedVar *) expr)->gvexpr);
+ break;
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(expr));
coll = InvalidOid; /* keep compiler quiet */
*************** expression_tree_walker(Node *node,
*** 2198,2203 ****
--- 2204,2211 ----
break;
case T_PlaceHolderVar:
return walker(((PlaceHolderVar *) node)->phexpr, context);
+ case T_GroupedVar:
+ return walker(((GroupedVar *) node)->gvexpr, context);
case T_InferenceElem:
return walker(((InferenceElem *) node)->expr, context);
case T_AppendRelInfo:
*************** expression_tree_mutator(Node *node,
*** 2989,2994 ****
--- 2997,3012 ----
return (Node *) newnode;
}
break;
+ case T_GroupedVar:
+ {
+ GroupedVar *gv = (GroupedVar *) node;
+ GroupedVar *newnode;
+
+ FLATCOPY(newnode, gv, GroupedVar);
+ MUTATE(newnode->gvexpr, gv->gvexpr, Expr *);
+ MUTATE(newnode->agg_partial, gv->agg_partial, Aggref *);
+ return (Node *) newnode;
+ }
case T_InferenceElem:
{
InferenceElem *inferenceelemdexpr = (InferenceElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
new file mode 100644
index 28cef85..4b6ee30
*** a/src/backend/nodes/outfuncs.c
--- b/src/backend/nodes/outfuncs.c
*************** _outPlannerInfo(StringInfo str, const Pl
*** 2186,2191 ****
--- 2186,2192 ----
WRITE_NODE_FIELD(pcinfo_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
*************** _outParamPathInfo(StringInfo str, const
*** 2408,2413 ****
--- 2409,2424 ----
}
static void
+ _outGroupedPathInfo(StringInfo str, const GroupedPathInfo *node)
+ {
+ WRITE_NODE_TYPE("GROUPEDPATHINFO");
+
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(pathlist);
+ WRITE_NODE_FIELD(partial_pathlist);
+ }
+
+ static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
WRITE_NODE_TYPE("RESTRICTINFO");
*************** _outPlaceHolderVar(StringInfo str, const
*** 2451,2456 ****
--- 2462,2477 ----
}
static void
+ _outGroupedVar(StringInfo str, const GroupedVar *node)
+ {
+ WRITE_NODE_TYPE("GROUPEDVAR");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(gvid);
+ }
+
+ static void
_outSpecialJoinInfo(StringInfo str, const SpecialJoinInfo *node)
{
WRITE_NODE_TYPE("SPECIALJOININFO");
*************** outNode(StringInfo str, const void *obj)
*** 3996,4007 ****
--- 4017,4034 ----
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_GroupedPathInfo:
+ _outGroupedPathInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
case T_PlaceHolderVar:
_outPlaceHolderVar(str, obj);
break;
+ case T_GroupedVar:
+ _outGroupedVar(str, obj);
+ break;
case T_SpecialJoinInfo:
_outSpecialJoinInfo(str, obj);
break;
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
new file mode 100644
index a883220..138f71c
*** a/src/backend/nodes/readfuncs.c
--- b/src/backend/nodes/readfuncs.c
*************** _readVar(void)
*** 522,527 ****
--- 522,542 ----
}
/*
+ * _readGroupedVar
+ */
+ static GroupedVar *
+ _readGroupedVar(void)
+ {
+ READ_LOCALS(GroupedVar);
+
+ READ_NODE_FIELD(gvexpr);
+ READ_NODE_FIELD(agg_partial);
+ READ_UINT_FIELD(gvid);
+
+ READ_DONE();
+ }
+
+ /*
* _readConst
*/
static Const *
*************** parseNodeString(void)
*** 2440,2445 ****
--- 2455,2462 ----
return_value = _readTableFunc();
else if (MATCH("VAR", 3))
return_value = _readVar();
+ else if (MATCH("GROUPEDVAR", 10))
+ return_value = _readGroupedVar();
else if (MATCH("CONST", 5))
return_value = _readConst();
else if (MATCH("PARAM", 5))
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
new file mode 100644
index b5cab0c..f89406d
*** a/src/backend/optimizer/geqo/geqo_eval.c
--- b/src/backend/optimizer/geqo/geqo_eval.c
*************** merge_clump(PlannerInfo *root, List *clu
*** 265,271 ****
if (joinrel)
{
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, joinrel);
/* Find and save the cheapest paths for this joinrel */
set_cheapest(joinrel);
--- 265,271 ----
if (joinrel)
{
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, joinrel, false);
/* Find and save the cheapest paths for this joinrel */
set_cheapest(joinrel);
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
new file mode 100644
index b93b4fc..ad14578
*** a/src/backend/optimizer/path/allpaths.c
--- b/src/backend/optimizer/path/allpaths.c
*************** set_rel_pathlist(PlannerInfo *root, RelO
*** 486,492 ****
* we'll consider gathering partial paths for the parent appendrel.)
*/
if (rel->reloptkind == RELOPT_BASEREL)
! generate_gather_paths(root, rel);
/*
* Allow a plugin to editorialize on the set of Paths for this base
--- 486,495 ----
* we'll consider gathering partial paths for the parent appendrel.)
*/
if (rel->reloptkind == RELOPT_BASEREL)
! {
! generate_gather_paths(root, rel, false);
! generate_gather_paths(root, rel, true);
! }
/*
* Allow a plugin to editorialize on the set of Paths for this base
*************** static void
*** 686,691 ****
--- 689,695 ----
set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
{
Relids required_outer;
+ Path *seq_path;
/*
* We don't support pushing join clauses into the quals of a seqscan, but
*************** set_plain_rel_pathlist(PlannerInfo *root
*** 694,708 ****
*/
required_outer = rel->lateral_relids;
! /* Consider sequential scan */
! add_path(rel, create_seqscan_path(root, rel, required_outer, 0));
! /* If appropriate, consider parallel sequential scan */
if (rel->consider_parallel && required_outer == NULL)
create_plain_partial_paths(root, rel);
/* Consider index scans */
! create_index_paths(root, rel);
/* Consider TID scans */
create_tidscan_paths(root, rel);
--- 698,725 ----
*/
required_outer = rel->lateral_relids;
! /* Consider sequential scan, both plain and grouped. */
! seq_path = create_seqscan_path(root, rel, required_outer, 0);
! add_path(rel, seq_path, false);
! if (rel->gpi != NULL && required_outer == NULL)
! create_grouped_path(root, rel, seq_path, false, false, AGG_HASHED);
! /* If appropriate, consider parallel sequential scan (plain or grouped) */
if (rel->consider_parallel && required_outer == NULL)
create_plain_partial_paths(root, rel);
/* Consider index scans */
! create_index_paths(root, rel, false);
! if (rel->gpi != NULL)
! {
! /*
! * TODO Instead of calling the whole clause-matching machinery twice
! * (there should be no difference between plain and grouped paths from
! * this point of view), consider returning a separate list of paths
! * usable as grouped ones.
! */
! create_index_paths(root, rel, true);
! }
/* Consider TID scans */
create_tidscan_paths(root, rel);
*************** static void
*** 716,721 ****
--- 733,739 ----
create_plain_partial_paths(PlannerInfo *root, RelOptInfo *rel)
{
int parallel_workers;
+ Path *path;
parallel_workers = compute_parallel_worker(rel, rel->pages, -1);
*************** create_plain_partial_paths(PlannerInfo *
*** 724,730 ****
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! add_partial_path(rel, create_seqscan_path(root, rel, NULL, parallel_workers));
}
/*
--- 742,849 ----
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! path = create_seqscan_path(root, rel, NULL, parallel_workers);
! add_partial_path(rel, path, false);
!
! /*
! * Do partial aggregation at base relation level if the relation is
! * eligible for it.
! */
! if (rel->gpi != NULL)
! create_grouped_path(root, rel, path, false, true, AGG_HASHED);
! }
!
! /*
! * Apply partial aggregation to a subpath and add the AggPath to the
! * appropriate pathlist.
! *
! * "precheck" tells whether the aggregation path should first be checked using
! * add_path_precheck().
! *
! * If "partial" is true, the resulting path is considered partial in terms of
! * parallel execution.
! *
! * The path we create here shouldn't be parameterized because of supposedly
! * high startup cost of aggregation (whether due to build of hash table for
! * AGG_HASHED strategy or due to explicit sort for AGG_SORTED).
! *
! * XXX IndexPath as an input for AGG_SORTED might seem to be an exception, but
! * aggregation of its output is only beneficial if it's performed by multiple
! * workers, i.e. the resulting path is partial (Besides parallel aggregation,
! * the other use case of aggregation push-down is aggregation performed on
! * remote database, but that has nothing to do with IndexScan). And partial
! * path cannot be parameterized because it's semantically wrong to use it on
! * the inner side of NL join.
! */
! void
! create_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
! bool precheck, bool partial, AggStrategy aggstrategy)
! {
! List *group_clauses = NIL;
! List *group_exprs = NIL;
! List *agg_exprs = NIL;
! Path *agg_path;
!
! /*
! * If the AggPath should be partial, the subpath must be too, and
! * therefore the subpath is essentially parallel_safe.
! */
! Assert(subpath->parallel_safe || !partial);
!
! /*
! * Grouped path should never be parameterized, so we're not supposed to
! * receive parameterized subpath.
! */
! Assert(subpath->param_info == NULL);
!
! /*
! * Note that "partial" in the following function names refers to 2-stage
! * aggregation, not to parallel processing.
! */
! if (aggstrategy == AGG_HASHED)
! agg_path = (Path *) create_partial_agg_hashed_path(root, subpath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! subpath->rows);
! else if (aggstrategy == AGG_SORTED)
! agg_path = (Path *) create_partial_agg_sorted_path(root, subpath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! subpath->rows);
! else
! elog(ERROR, "unexpected strategy %d", aggstrategy);
!
! /* Add the grouped path to the list of grouped base paths. */
! if (agg_path != NULL)
! {
! if (precheck)
! {
! List *pathkeys;
!
! /* AGG_HASH is not supposed to generate sorted output. */
! pathkeys = aggstrategy == AGG_SORTED ? subpath->pathkeys : NIL;
!
! if (!partial &&
! !add_path_precheck(rel, agg_path->startup_cost,
! agg_path->total_cost, pathkeys, NULL,
! true))
! return;
!
! if (partial &&
! !add_partial_path_precheck(rel, agg_path->total_cost, pathkeys,
! true))
! return;
! }
!
! if (!partial)
! add_path(rel, (Path *) agg_path, true);
! else
! add_partial_path(rel, (Path *) agg_path, true);
! }
}
/*
*************** set_tablesample_rel_pathlist(PlannerInfo
*** 810,816 ****
path = (Path *) create_material_path(rel, path);
}
! add_path(rel, path);
/* For the moment, at least, there are no other paths to consider */
}
--- 929,935 ----
path = (Path *) create_material_path(rel, path);
}
! add_path(rel, path, false);
/* For the moment, at least, there are no other paths to consider */
}
*************** set_append_rel_size(PlannerInfo *root, R
*** 1067,1072 ****
--- 1186,1234 ----
appinfo);
/*
+ * If grouping is applicable to the parent relation, it should be
+ * applicable to the children too. Make sure the child rel has valid
+ * sortgrouprefs.
+ *
+ * TODO Consider if this is really needed --- child rel is not joined
+ * to grouped rel itself, so it might not participate on creation of
+ * the grouped path target that upper joins will see.
+ */
+ if (rel->reltarget->sortgrouprefs)
+ {
+ Assert(childrel->reltarget->sortgrouprefs == NULL);
+ childrel->reltarget->sortgrouprefs = rel->reltarget->sortgrouprefs;
+ }
+
+ /*
+ * Also the grouped target needs to be adjusted, if one exists.
+ */
+ if (rel->gpi != NULL)
+ {
+ PathTarget *target = rel->gpi->target;
+
+ Assert(target->sortgrouprefs != NULL);
+
+ Assert(childrel->gpi == NULL);
+ childrel->gpi = makeNode(GroupedPathInfo);
+ memcpy(childrel->gpi, rel->gpi, sizeof(GroupedPathInfo));
+
+ /*
+ * add_grouping_info_to_base_rels was not sure if grouping makes
+ * sense for the parent rel, so create a separate copy of the
+ * target now.
+ */
+ childrel->gpi->target = copy_pathtarget(childrel->gpi->target);
+
+ /* Translate vars of the grouping target. */
+ Assert(childrel->gpi->target->exprs != NIL);
+ childrel->gpi->target->exprs = (List *)
+ adjust_appendrel_attrs(root,
+ (Node *) childrel->gpi->target->exprs,
+ appinfo);
+ }
+
+ /*
* We have to make child entries in the EquivalenceClass data
* structures as well. This is needed either if the parent
* participates in some eclass joins (because we will want to consider
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1281,1286 ****
--- 1443,1450 ----
bool subpaths_valid = true;
List *partial_subpaths = NIL;
bool partial_subpaths_valid = true;
+ List *grouped_subpaths = NIL;
+ bool grouped_subpaths_valid = true;
List *all_child_pathkeys = NIL;
List *all_child_outers = NIL;
ListCell *l;
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1324,1329 ****
--- 1488,1516 ----
partial_subpaths_valid = false;
/*
+ * For grouped paths, use only the unparameterized subpaths.
+ *
+ * XXX Consider if the parameterized subpaths should be processed
+ * below. It's probably not useful for sequential scans (due to
+ * repeated aggregation), but might be worthwhile for other child
+ * nodes.
+ */
+ if (childrel->gpi != NULL && childrel->gpi->pathlist != NIL)
+ {
+ Path *path;
+
+ path = (Path *) linitial(childrel->gpi->pathlist);
+ if (path->param_info == NULL)
+ grouped_subpaths = accumulate_append_subpath(grouped_subpaths,
+ path);
+ else
+ grouped_subpaths_valid = false;
+ }
+ else
+ grouped_subpaths_valid = false;
+
+
+ /*
* Collect lists of all the available path orderings and
* parameterizations for all the children. We use these as a
* heuristic to indicate which sort orderings and parameterizations we
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1395,1401 ****
*/
if (subpaths_valid)
add_path(rel, (Path *) create_append_path(rel, subpaths, NULL, 0,
! partitioned_rels));
/*
* Consider an append of partial unordered, unparameterized partial paths.
--- 1582,1589 ----
*/
if (subpaths_valid)
add_path(rel, (Path *) create_append_path(rel, subpaths, NULL, 0,
! partitioned_rels),
! false);
/*
* Consider an append of partial unordered, unparameterized partial paths.
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1422,1429 ****
/* Generate a partial append path. */
appendpath = create_append_path(rel, partial_subpaths, NULL,
! parallel_workers, partitioned_rels);
! add_partial_path(rel, (Path *) appendpath);
}
/*
--- 1610,1630 ----
/* Generate a partial append path. */
appendpath = create_append_path(rel, partial_subpaths, NULL,
! parallel_workers,
! partitioned_rels);
! add_partial_path(rel, (Path *) appendpath, false);
! }
!
! /* TODO Also partial grouped paths? */
! if (grouped_subpaths_valid)
! {
! Path *path;
!
! path = (Path *) create_append_path(rel, grouped_subpaths, NULL, 0,
! partitioned_rels);
! /* pathtarget will produce the grouped relation.. */
! path->pathtarget = rel->gpi->target;
! add_path(rel, path, true);
}
/*
*************** add_paths_to_append_rel(PlannerInfo *roo
*** 1476,1482 ****
if (subpaths_valid)
add_path(rel, (Path *)
create_append_path(rel, subpaths, required_outer, 0,
! partitioned_rels));
}
}
--- 1677,1684 ----
if (subpaths_valid)
add_path(rel, (Path *)
create_append_path(rel, subpaths, required_outer, 0,
! partitioned_rels),
! false);
}
}
*************** generate_mergeappend_paths(PlannerInfo *
*** 1572,1585 ****
startup_subpaths,
pathkeys,
NULL,
! partitioned_rels));
if (startup_neq_total)
add_path(rel, (Path *) create_merge_append_path(root,
rel,
total_subpaths,
pathkeys,
NULL,
! partitioned_rels));
}
}
--- 1774,1789 ----
startup_subpaths,
pathkeys,
NULL,
! partitioned_rels),
! false);
if (startup_neq_total)
add_path(rel, (Path *) create_merge_append_path(root,
rel,
total_subpaths,
pathkeys,
NULL,
! partitioned_rels),
! false);
}
}
*************** set_dummy_rel_pathlist(RelOptInfo *rel)
*** 1712,1718 ****
rel->pathlist = NIL;
rel->partial_pathlist = NIL;
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL));
/*
* We set the cheapest path immediately, to ensure that IS_DUMMY_REL()
--- 1916,1922 ----
rel->pathlist = NIL;
rel->partial_pathlist = NIL;
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL), false);
/*
* We set the cheapest path immediately, to ensure that IS_DUMMY_REL()
*************** set_subquery_pathlist(PlannerInfo *root,
*** 1926,1932 ****
/* Generate outer path using this subpath */
add_path(rel, (Path *)
create_subqueryscan_path(root, rel, subpath,
! pathkeys, required_outer));
}
}
--- 2130,2136 ----
/* Generate outer path using this subpath */
add_path(rel, (Path *)
create_subqueryscan_path(root, rel, subpath,
! pathkeys, required_outer), false);
}
}
*************** set_function_pathlist(PlannerInfo *root,
*** 1995,2001 ****
/* Generate appropriate path */
add_path(rel, create_functionscan_path(root, rel,
! pathkeys, required_outer));
}
/*
--- 2199,2205 ----
/* Generate appropriate path */
add_path(rel, create_functionscan_path(root, rel,
! pathkeys, required_outer), false);
}
/*
*************** set_values_pathlist(PlannerInfo *root, R
*** 2015,2021 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_valuesscan_path(root, rel, required_outer));
}
/*
--- 2219,2225 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_valuesscan_path(root, rel, required_outer), false);
}
/*
*************** set_tablefunc_pathlist(PlannerInfo *root
*** 2036,2042 ****
/* Generate appropriate path */
add_path(rel, create_tablefuncscan_path(root, rel,
! required_outer));
}
/*
--- 2240,2246 ----
/* Generate appropriate path */
add_path(rel, create_tablefuncscan_path(root, rel,
! required_outer), false);
}
/*
*************** set_cte_pathlist(PlannerInfo *root, RelO
*** 2102,2108 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_ctescan_path(root, rel, required_outer));
}
/*
--- 2306,2312 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_ctescan_path(root, rel, required_outer), false);
}
/*
*************** set_namedtuplestore_pathlist(PlannerInfo
*** 2129,2135 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_namedtuplestorescan_path(root, rel, required_outer));
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(rel);
--- 2333,2340 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_namedtuplestorescan_path(root, rel, required_outer),
! false);
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(rel);
*************** set_worktable_pathlist(PlannerInfo *root
*** 2182,2188 ****
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_worktablescan_path(root, rel, required_outer));
}
/*
--- 2387,2394 ----
required_outer = rel->lateral_relids;
/* Generate appropriate path */
! add_path(rel, create_worktablescan_path(root, rel, required_outer),
! false);
}
/*
*************** set_worktable_pathlist(PlannerInfo *root
*** 2195,2208 ****
* path that some GatherPath or GatherMergePath has a reference to.)
*/
void
! generate_gather_paths(PlannerInfo *root, RelOptInfo *rel)
{
Path *cheapest_partial_path;
Path *simple_gather_path;
ListCell *lc;
/* If there are no partial paths, there's nothing to do here. */
! if (rel->partial_pathlist == NIL)
return;
/*
--- 2401,2421 ----
* path that some GatherPath or GatherMergePath has a reference to.)
*/
void
! generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool grouped)
{
Path *cheapest_partial_path;
Path *simple_gather_path;
+ List *pathlist = NIL;
+ PathTarget *partial_target;
ListCell *lc;
+ if (!grouped)
+ pathlist = rel->partial_pathlist;
+ else if (rel->gpi != NULL)
+ pathlist = rel->gpi->partial_pathlist;
+
/* If there are no partial paths, there's nothing to do here. */
! if (pathlist == NIL)
return;
/*
*************** generate_gather_paths(PlannerInfo *root,
*** 2210,2226 ****
* path of interest: the cheapest one. That will be the one at the front
* of partial_pathlist because of the way add_partial_path works.
*/
! cheapest_partial_path = linitial(rel->partial_pathlist);
simple_gather_path = (Path *)
! create_gather_path(root, rel, cheapest_partial_path, rel->reltarget,
NULL, NULL);
! add_path(rel, simple_gather_path);
/*
* For each useful ordering, we can consider an order-preserving Gather
* Merge.
*/
! foreach (lc, rel->partial_pathlist)
{
Path *subpath = (Path *) lfirst(lc);
GatherMergePath *path;
--- 2423,2445 ----
* path of interest: the cheapest one. That will be the one at the front
* of partial_pathlist because of the way add_partial_path works.
*/
! cheapest_partial_path = linitial(pathlist);
!
! if (!grouped)
! partial_target = rel->reltarget;
! else if (rel->gpi != NULL)
! partial_target = rel->gpi->target;
!
simple_gather_path = (Path *)
! create_gather_path(root, rel, cheapest_partial_path, partial_target,
NULL, NULL);
! add_path(rel, simple_gather_path, grouped);
/*
* For each useful ordering, we can consider an order-preserving Gather
* Merge.
*/
! foreach (lc, pathlist)
{
Path *subpath = (Path *) lfirst(lc);
GatherMergePath *path;
*************** generate_gather_paths(PlannerInfo *root,
*** 2228,2236 ****
if (subpath->pathkeys == NIL)
continue;
! path = create_gather_merge_path(root, rel, subpath, rel->reltarget,
subpath->pathkeys, NULL, NULL);
! add_path(rel, &path->path);
}
}
--- 2447,2455 ----
if (subpath->pathkeys == NIL)
continue;
! path = create_gather_merge_path(root, rel, subpath, partial_target,
subpath->pathkeys, NULL, NULL);
! add_path(rel, &path->path, grouped);
}
}
*************** standard_join_search(PlannerInfo *root,
*** 2396,2402 ****
rel = (RelOptInfo *) lfirst(lc);
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, rel);
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
--- 2615,2622 ----
rel = (RelOptInfo *) lfirst(lc);
/* Create GatherPaths for any useful partial paths for rel */
! generate_gather_paths(root, rel, false);
! generate_gather_paths(root, rel, true);
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
*************** create_partial_bitmap_paths(PlannerInfo
*** 3047,3053 ****
return;
add_partial_path(rel, (Path *) create_bitmap_heap_path(root, rel,
! bitmapqual, rel->lateral_relids, 1.0, parallel_workers));
}
/*
--- 3267,3273 ----
return;
add_partial_path(rel, (Path *) create_bitmap_heap_path(root, rel,
! bitmapqual, rel->lateral_relids, 1.0, parallel_workers), false);
}
/*
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
new file mode 100644
index 6e4bae8..a6fa713
*** a/src/backend/optimizer/path/indxpath.c
--- b/src/backend/optimizer/path/indxpath.c
***************
*** 32,37 ****
--- 32,38 ----
#include "optimizer/predtest.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "utils/builtins.h"
#include "utils/bytea.h"
*************** static bool eclass_already_used(Equivale
*** 107,119 ****
static bool bms_equal_any(Relids relids, List *relids_list);
static void get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths);
static List *build_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop);
static List *build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
List *clauses, List *other_clauses);
static List *generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
--- 108,121 ----
static bool bms_equal_any(Relids relids, List *relids_list);
static void get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths, bool grouped);
static List *build_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop,
! bool grouped);
static List *build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
List *clauses, List *other_clauses);
static List *generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
*************** static Const *string_to_const(const char
*** 229,235 ****
* as meaning "unparameterized so far as the indexquals are concerned".
*/
void
! create_index_paths(PlannerInfo *root, RelOptInfo *rel)
{
List *indexpaths;
List *bitindexpaths;
--- 231,237 ----
* as meaning "unparameterized so far as the indexquals are concerned".
*/
void
! create_index_paths(PlannerInfo *root, RelOptInfo *rel, bool grouped)
{
List *indexpaths;
List *bitindexpaths;
*************** create_index_paths(PlannerInfo *root, Re
*** 274,281 ****
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
! get_index_paths(root, rel, index, &rclauseset,
! &bitindexpaths);
/*
* Identify the join clauses that can match the index. For the moment
--- 276,283 ----
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
! get_index_paths(root, rel, index, &rclauseset, &bitindexpaths,
! grouped);
/*
* Identify the join clauses that can match the index. For the moment
*************** create_index_paths(PlannerInfo *root, Re
*** 338,344 ****
bitmapqual = choose_bitmap_and(root, rel, bitindexpaths);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
rel->lateral_relids, 1.0, 0);
! add_path(rel, (Path *) bpath);
/* create a partial bitmap heap path */
if (rel->consider_parallel && rel->lateral_relids == NULL)
--- 340,346 ----
bitmapqual = choose_bitmap_and(root, rel, bitindexpaths);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
rel->lateral_relids, 1.0, 0);
! add_path(rel, (Path *) bpath, false);
/* create a partial bitmap heap path */
if (rel->consider_parallel && rel->lateral_relids == NULL)
*************** create_index_paths(PlannerInfo *root, Re
*** 415,421 ****
loop_count = get_loop_count(root, rel->relid, required_outer);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
required_outer, loop_count, 0);
! add_path(rel, (Path *) bpath);
}
}
}
--- 417,423 ----
loop_count = get_loop_count(root, rel->relid, required_outer);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
required_outer, loop_count, 0);
! add_path(rel, (Path *) bpath, false);
}
}
}
*************** get_join_index_paths(PlannerInfo *root,
*** 667,673 ****
Assert(clauseset.nonempty);
/* Build index path(s) using the collected set of clauses */
! get_index_paths(root, rel, index, &clauseset, bitindexpaths);
/*
* Remember we considered paths for this set of relids. We use lcons not
--- 669,675 ----
Assert(clauseset.nonempty);
/* Build index path(s) using the collected set of clauses */
! get_index_paths(root, rel, index, &clauseset, bitindexpaths, false);
/*
* Remember we considered paths for this set of relids. We use lcons not
*************** bms_equal_any(Relids relids, List *relid
*** 736,742 ****
static void
get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths)
{
List *indexpaths;
bool skip_nonnative_saop = false;
--- 738,744 ----
static void
get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
! List **bitindexpaths, bool grouped)
{
List *indexpaths;
bool skip_nonnative_saop = false;
*************** get_index_paths(PlannerInfo *root, RelOp
*** 754,760 ****
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! &skip_lower_saop);
/*
* If we skipped any lower-order ScalarArrayOpExprs on an index with an AM
--- 756,762 ----
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! &skip_lower_saop, grouped);
/*
* If we skipped any lower-order ScalarArrayOpExprs on an index with an AM
*************** get_index_paths(PlannerInfo *root, RelOp
*** 769,775 ****
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! NULL));
}
/*
--- 771,777 ----
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
! NULL, grouped));
}
/*
*************** get_index_paths(PlannerInfo *root, RelOp
*** 789,797 ****
IndexPath *ipath = (IndexPath *) lfirst(lc);
if (index->amhasgettuple)
! add_path(rel, (Path *) ipath);
! if (index->amhasgetbitmap &&
(ipath->path.pathkeys == NIL ||
ipath->indexselectivity < 1.0))
*bitindexpaths = lappend(*bitindexpaths, ipath);
--- 791,799 ----
IndexPath *ipath = (IndexPath *) lfirst(lc);
if (index->amhasgettuple)
! add_path(rel, (Path *) ipath, grouped);
! if (!grouped && index->amhasgetbitmap &&
(ipath->path.pathkeys == NIL ||
ipath->indexselectivity < 1.0))
*bitindexpaths = lappend(*bitindexpaths, ipath);
*************** get_index_paths(PlannerInfo *root, RelOp
*** 802,815 ****
* natively, generate bitmap scan paths relying on executor-managed
* ScalarArrayOpExpr.
*/
! if (skip_nonnative_saop)
{
indexpaths = build_index_paths(root, rel,
index, clauses,
false,
ST_BITMAPSCAN,
NULL,
! NULL);
*bitindexpaths = list_concat(*bitindexpaths, indexpaths);
}
}
--- 804,818 ----
* natively, generate bitmap scan paths relying on executor-managed
* ScalarArrayOpExpr.
*/
! if (!grouped && skip_nonnative_saop)
{
indexpaths = build_index_paths(root, rel,
index, clauses,
false,
ST_BITMAPSCAN,
NULL,
! NULL,
! false);
*bitindexpaths = list_concat(*bitindexpaths, indexpaths);
}
}
*************** build_index_paths(PlannerInfo *root, Rel
*** 861,867 ****
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop)
{
List *result = NIL;
IndexPath *ipath;
--- 864,870 ----
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
! bool *skip_lower_saop, bool grouped)
{
List *result = NIL;
IndexPath *ipath;
*************** build_index_paths(PlannerInfo *root, Rel
*** 878,883 ****
--- 881,890 ----
bool index_is_ordered;
bool index_only_scan;
int indexcol;
+ bool can_agg_sorted;
+ List *group_clauses, *group_exprs, *agg_exprs;
+ AggPath *agg_path;
+ double agg_input_rows;
/*
* Check that index supports the desired scan type(s)
*************** build_index_paths(PlannerInfo *root, Rel
*** 891,896 ****
--- 898,906 ----
case ST_BITMAPSCAN:
if (!index->amhasgetbitmap)
return NIL;
+
+ if (grouped)
+ return NIL;
break;
case ST_ANYSCAN:
/* either or both are OK */
*************** build_index_paths(PlannerInfo *root, Rel
*** 1032,1037 ****
--- 1042,1051 ----
* later merging or final output ordering, OR the index has a useful
* predicate, OR an index-only scan is possible.
*/
+ can_agg_sorted = true;
+ group_clauses = NIL;
+ group_exprs = NIL;
+ agg_exprs = NIL;
if (index_clauses != NIL || useful_pathkeys != NIL || useful_predicate ||
index_only_scan)
{
*************** build_index_paths(PlannerInfo *root, Rel
*** 1048,1054 ****
outer_relids,
loop_count,
false);
! result = lappend(result, ipath);
/*
* If appropriate, consider parallel index scan. We don't allow
--- 1062,1086 ----
outer_relids,
loop_count,
false);
! if (!grouped)
! result = lappend(result, ipath);
! else
! {
! /* TODO Double-check if this is the correct input value. */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root, (Path *) ipath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
!
! if (agg_path != NULL)
! result = lappend(result, agg_path);
! else
! can_agg_sorted = false;
! }
/*
* If appropriate, consider parallel index scan. We don't allow
*************** build_index_paths(PlannerInfo *root, Rel
*** 1077,1083 ****
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath);
else
pfree(ipath);
}
--- 1109,1139 ----
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! {
! if (!grouped)
! add_partial_path(rel, (Path *) ipath, grouped);
! else if (can_agg_sorted && outer_relids == NULL)
! {
! /* TODO Double-check if this is the correct input value. */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! false,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
!
! /*
! * If create_agg_sorted_path succeeded once, it should
! * always do.
! */
! Assert(agg_path != NULL);
!
! add_partial_path(rel, (Path *) agg_path, grouped);
! }
! }
else
pfree(ipath);
}
*************** build_index_paths(PlannerInfo *root, Rel
*** 1105,1111 ****
outer_relids,
loop_count,
false);
! result = lappend(result, ipath);
/* If appropriate, consider parallel index scan */
if (index->amcanparallel &&
--- 1161,1185 ----
outer_relids,
loop_count,
false);
!
! if (!grouped)
! result = lappend(result, ipath);
! else if (can_agg_sorted)
! {
! /* TODO Double-check if this is the correct input value. */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! true,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
!
! Assert(agg_path != NULL);
! result = lappend(result, agg_path);
! }
/* If appropriate, consider parallel index scan */
if (index->amcanparallel &&
*************** build_index_paths(PlannerInfo *root, Rel
*** 1129,1135 ****
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath);
else
pfree(ipath);
}
--- 1203,1227 ----
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! {
! if (!grouped)
! add_partial_path(rel, (Path *) ipath, grouped);
! else if (can_agg_sorted && outer_relids == NULL)
! {
! /* TODO Double-check if this is the correct input value. */
! agg_input_rows = rel->rows * ipath->indexselectivity;
!
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! false,
! &group_clauses,
! &group_exprs,
! &agg_exprs,
! agg_input_rows);
! Assert(agg_path != NULL);
! add_partial_path(rel, (Path *) agg_path, grouped);
! }
! }
else
pfree(ipath);
}
*************** build_paths_for_OR(PlannerInfo *root, Re
*** 1244,1250 ****
useful_predicate,
ST_BITMAPSCAN,
NULL,
! NULL);
result = list_concat(result, indexpaths);
}
--- 1336,1343 ----
useful_predicate,
ST_BITMAPSCAN,
NULL,
! NULL,
! false);
result = list_concat(result, indexpaths);
}
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
new file mode 100644
index 5aedcd1..12f0356
*** a/src/backend/optimizer/path/joinpath.c
--- b/src/backend/optimizer/path/joinpath.c
***************
*** 22,27 ****
--- 22,28 ----
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/planmain.h"
+ #include "optimizer/tlist.h"
/* Hook for plugins to get control in add_paths_to_joinrel() */
set_join_pathlist_hook_type set_join_pathlist_hook = NULL;
*************** static void try_partial_mergejoin_path(P
*** 38,66 ****
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra);
static void sort_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra);
static void match_unsorted_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra);
static void consider_parallel_nestloop(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra);
static void consider_parallel_mergejoin(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total);
static void hash_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra);
static List *select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
--- 39,87 ----
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate);
static void sort_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra,
! bool grouped);
! static void sort_inner_and_outer_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! RelOptInfo *outerrel,
! RelOptInfo *innerrel,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate);
static void match_unsorted_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra,
! bool grouped);
static void consider_parallel_nestloop(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped, bool do_aggregate);
static void consider_parallel_mergejoin(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total,
! bool grouped);
static void hash_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra,
! bool grouped);
! static bool is_grouped_join_target_complete(PlannerInfo *root,
! PathTarget *jointarget,
! Path *outer_path,
! Path *inner_path);
static List *select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
*************** static void generate_mergejoin_paths(Pla
*** 77,83 ****
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial);
/*
--- 98,107 ----
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate);
/*
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 227,234 ****
* sorted. Skip this if we can't mergejoin.
*/
if (mergejoin_allowed)
sort_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra);
/*
* 2. Consider paths where the outer relation need not be explicitly
--- 251,262 ----
* sorted. Skip this if we can't mergejoin.
*/
if (mergejoin_allowed)
+ {
sort_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, false);
! sort_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, true);
! }
/*
* 2. Consider paths where the outer relation need not be explicitly
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 238,245 ****
* joins at all, so it wouldn't work in the prohibited cases either.)
*/
if (mergejoin_allowed)
match_unsorted_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra);
#ifdef NOT_USED
--- 266,277 ----
* joins at all, so it wouldn't work in the prohibited cases either.)
*/
if (mergejoin_allowed)
+ {
match_unsorted_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, false);
! match_unsorted_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, true);
! }
#ifdef NOT_USED
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 265,272 ****
* joins, because there may be no other alternative.
*/
if (enable_hashjoin || jointype == JOIN_FULL)
hash_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra);
/*
* 5. If inner and outer relations are foreign tables (or joins) belonging
--- 297,308 ----
* joins, because there may be no other alternative.
*/
if (enable_hashjoin || jointype == JOIN_FULL)
+ {
hash_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, false);
! hash_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, true);
! }
/*
* 5. If inner and outer relations are foreign tables (or joins) belonging
*************** try_nestloop_path(PlannerInfo *root,
*** 330,339 ****
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
Relids required_outer;
JoinCostWorkspace workspace;
/*
* Check to see if proposed path is still parameterized, and reject if the
--- 366,385 ----
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* Caller should not request aggregation w/o grouped output. */
+ Assert(!do_aggregate || grouped);
+
+ /* GroupedPathInfo is necessary for us to produce a grouped set. */
+ Assert(joinrel->gpi != NULL || !grouped);
/*
* Check to see if proposed path is still parameterized, and reject if the
*************** try_nestloop_path(PlannerInfo *root,
*** 341,359 ****
* says to allow it anyway. Also, we must reject if have_dangerous_phv
* doesn't like the look of it, which could only happen if the nestloop is
* still parameterized.
*/
! required_outer = calc_nestloop_required_outer(outer_path,
! inner_path);
! if (required_outer &&
! ((!bms_overlap(required_outer, extra->param_source_rels) &&
! !allow_star_schema_join(root, outer_path, inner_path)) ||
! have_dangerous_phv(root,
! outer_path->parent->relids,
! PATH_REQ_OUTER(inner_path))))
{
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
}
/*
--- 387,409 ----
* says to allow it anyway. Also, we must reject if have_dangerous_phv
* doesn't like the look of it, which could only happen if the nestloop is
* still parameterized.
+ *
+ * Grouped path should never be parameterized.
*/
! required_outer = calc_nestloop_required_outer(outer_path, inner_path);
! if (required_outer)
{
! if (grouped ||
! (!bms_overlap(required_outer, extra->param_source_rels) &&
! !allow_star_schema_join(root, outer_path, inner_path)) ||
! have_dangerous_phv(root,
! outer_path->parent->relids,
! PATH_REQ_OUTER(inner_path)))
! {
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
! }
}
/*
*************** try_nestloop_path(PlannerInfo *root,
*** 368,388 ****
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer))
{
! add_path(joinrel, (Path *)
! create_nestloop_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer));
}
else
{
--- 418,453 ----
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! join_target = joinrel->gpi->target;
!
! join_path = (Path *) create_nestloop_path(root, joinrel, jointype,
! &workspace, extra,
! outer_path, inner_path,
! extra->restrictlist, pathkeys,
! required_outer, join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate && required_outer == NULL)
! {
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_SORTED);
! }
! else if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, grouped))
{
! add_path(joinrel, join_path, grouped);
}
else
{
*************** try_partial_nestloop_path(PlannerInfo *r
*** 403,411 ****
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* If the inner path is parameterized, the parameterization must be fully
--- 468,484 ----
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* The same checks we do in try_nestloop_path. */
+ Assert(!do_aggregate || grouped);
+ Assert(joinrel->gpi != NULL || !grouped);
/*
* If the inner path is parameterized, the parameterization must be fully
*************** try_partial_nestloop_path(PlannerInfo *r
*** 428,448 ****
*/
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys))
return;
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_nestloop_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL));
}
/*
--- 501,581 ----
*/
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
!
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! {
! Assert(joinrel->gpi != NULL);
! join_target = joinrel->gpi->target;
! }
!
! join_path = (Path *) create_nestloop_path(root, joinrel, jointype,
! &workspace, extra,
! outer_path, inner_path,
! extra->restrictlist, pathkeys,
! NULL, join_target);
!
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, true, AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, true, AGG_SORTED);
! }
! else if (add_partial_path_precheck(joinrel, workspace.total_cost,
! pathkeys, grouped))
! {
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *) join_path, grouped);
! }
! }
!
! static void
! try_grouped_nestloop_path(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *pathkeys,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate,
! bool partial)
! {
! /*
! * Missing GroupedPathInfo indicates that we should not try to create a
! * grouped join.
! */
! if (joinrel->gpi == NULL)
return;
! /*
! * Reject the path if we're supposed to combine grouped and plain relation
! * but the grouped one does not evaluate all the relevant aggregates.
! */
! if (!do_aggregate &&
! !is_grouped_join_target_complete(root, joinrel->gpi->target,
! outer_path, inner_path))
! return;
!
! /*
! * As repeated aggregation doesn't seem to be attractive, make sure that
! * the resulting grouped relation is not parameterized.
! */
! if (outer_path->param_info != NULL || inner_path->param_info != NULL)
! return;
!
! if (!partial)
! try_nestloop_path(root, joinrel, outer_path, inner_path, pathkeys,
! jointype, extra, true, do_aggregate);
! else
! try_partial_nestloop_path(root, joinrel, outer_path, inner_path,
! pathkeys, jointype, extra, true,
! do_aggregate);
}
/*
*************** try_mergejoin_path(PlannerInfo *root,
*** 461,470 ****
List *innersortkeys,
JoinType jointype,
JoinPathExtraData *extra,
! bool is_partial)
{
Relids required_outer;
JoinCostWorkspace workspace;
if (is_partial)
{
--- 594,613 ----
List *innersortkeys,
JoinType jointype,
JoinPathExtraData *extra,
! bool is_partial,
! bool grouped,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* Caller should not request aggregation w/o grouped output. */
+ Assert(!do_aggregate || grouped);
+
+ /* GroupedPathInfo is necessary for us to produce a grouped set. */
+ Assert(joinrel->gpi != NULL || !grouped);
if (is_partial)
{
*************** try_mergejoin_path(PlannerInfo *root,
*** 477,498 ****
outersortkeys,
innersortkeys,
jointype,
! extra);
return;
}
/*
! * Check to see if proposed path is still parameterized, and reject if the
! * parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path,
! inner_path);
! if (required_outer &&
! !bms_overlap(required_outer, extra->param_source_rels))
{
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
}
/*
--- 620,644 ----
outersortkeys,
innersortkeys,
jointype,
! extra,
! grouped,
! do_aggregate);
return;
}
/*
! * Check to see if proposed path is still parameterized, and reject if
! * it's grouped or if the parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path, inner_path);
! if (required_outer)
{
! if (grouped || !bms_overlap(required_outer, extra->param_source_rels))
! {
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
! }
}
/*
*************** try_mergejoin_path(PlannerInfo *root,
*** 511,537 ****
*/
initial_cost_mergejoin(root, &workspace, jointype, mergeclauses,
outer_path, inner_path,
! outersortkeys, innersortkeys,
! extra);
! if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer))
{
! add_path(joinrel, (Path *)
! create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer,
! mergeclauses,
! outersortkeys,
! innersortkeys));
}
else
{
--- 657,704 ----
*/
initial_cost_mergejoin(root, &workspace, jointype, mergeclauses,
outer_path, inner_path,
! outersortkeys, innersortkeys, extra);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! join_target = joinrel->gpi->target;
!
!
! join_path = (Path *) create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_SORTED);
! }
! else if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer, grouped))
{
! add_path(joinrel, (Path *) join_path, grouped);
}
else
{
*************** try_partial_mergejoin_path(PlannerInfo *
*** 555,563 ****
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* See comments in try_partial_hashjoin_path().
--- 722,738 ----
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* The same checks we do in try_mergejoin_path. */
+ Assert(!do_aggregate || grouped);
+ Assert(joinrel->gpi != NULL || !grouped);
/*
* See comments in try_partial_hashjoin_path().
*************** try_partial_mergejoin_path(PlannerInfo *
*** 587,613 ****
*/
initial_cost_mergejoin(root, &workspace, jointype, mergeclauses,
outer_path, inner_path,
! outersortkeys, innersortkeys,
! extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys))
return;
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL,
! mergeclauses,
! outersortkeys,
! innersortkeys));
}
/*
--- 762,934 ----
*/
initial_cost_mergejoin(root, &workspace, jointype, mergeclauses,
outer_path, inner_path,
! outersortkeys, innersortkeys, extra);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! {
! Assert(joinrel->gpi != NULL);
! join_target = joinrel->gpi->target;
! }
!
! join_path = (Path *) create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! join_target);
!
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, true, AGG_HASHED);
! create_grouped_path(root, joinrel, join_path, true, true, AGG_SORTED);
! }
! else if (add_partial_path_precheck(joinrel, workspace.total_cost,
! pathkeys, grouped))
! {
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *) join_path, grouped);
! }
! }
!
! static void
! try_grouped_mergejoin_path(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *pathkeys,
! List *mergeclauses,
! List *outersortkeys,
! List *innersortkeys,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool do_aggregate)
! {
! /*
! * Missing GroupedPathInfo indicates that we should not try to create a
! * grouped join.
! */
! if (joinrel->gpi == NULL)
return;
! /*
! * Reject the path if we're supposed to combine grouped and plain relation
! * but the grouped one does not evaluate all the relevant aggregates.
! */
! if (!do_aggregate &&
! !is_grouped_join_target_complete(root, joinrel->gpi->target,
! outer_path, inner_path))
! return;
!
! /*
! * As repeated aggregation doesn't seem to be attractive, make sure that
! * the resulting grouped relation is not parameterized.
! */
! if (outer_path->param_info != NULL || inner_path->param_info != NULL)
! return;
!
! if (!partial)
! try_mergejoin_path(root, joinrel, outer_path, inner_path, pathkeys,
! mergeclauses, outersortkeys, innersortkeys,
! jointype, extra, false, true, do_aggregate);
! else
! try_partial_mergejoin_path(root, joinrel, outer_path, inner_path,
! pathkeys,
! mergeclauses, outersortkeys, innersortkeys,
! jointype, extra, true, do_aggregate);
! }
!
! static void
! try_mergejoin_path_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *pathkeys,
! List *mergeclauses,
! List *outersortkeys,
! List *innersortkeys,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
! {
! bool grouped_join;
!
! grouped_join = grouped_outer || grouped_inner || do_aggregate;
!
! /* Join of two grouped paths is not supported. */
! Assert(!(grouped_outer && grouped_inner));
!
! if (!grouped_join)
! {
! /* Only join plain paths. */
! try_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! jointype,
! extra,
! partial,
! false, false);
! }
! else if (grouped_outer || grouped_inner)
! {
! Assert(!do_aggregate);
!
! /*
! * Exactly one of the input paths is grouped, so create a grouped join
! * path.
! */
! try_grouped_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! jointype,
! extra,
! partial,
! false);
! }
! /* Preform explicit aggregation only if suitable target exists. */
! else if (joinrel->gpi != NULL)
! {
! try_grouped_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! pathkeys,
! mergeclauses,
! outersortkeys,
! innersortkeys,
! jointype,
! extra,
! partial, true);
! }
}
/*
*************** try_hashjoin_path(PlannerInfo *root,
*** 622,668 ****
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra)
{
Relids required_outer;
JoinCostWorkspace workspace;
/*
! * Check to see if proposed path is still parameterized, and reject if the
! * parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path,
! inner_path);
! if (required_outer &&
! !bms_overlap(required_outer, extra->param_source_rels))
{
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
}
/*
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra);
! if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! NIL, required_outer))
{
! add_path(joinrel, (Path *)
! create_hashjoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! required_outer,
! hashclauses));
}
else
{
--- 943,1017 ----
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* Caller should not request aggregation w/o grouped output. */
+ Assert(!do_aggregate || grouped);
+
+ /* GroupedPathInfo is necessary for us to produce a grouped set. */
+ Assert(joinrel->gpi != NULL || !grouped);
/*
! * Check to see if proposed path is still parameterized, and reject if
! * it's grouped or if the parameterization wouldn't be sensible.
*/
! required_outer = calc_non_nestloop_required_outer(outer_path, inner_path);
! if (required_outer)
{
! if (grouped || !bms_overlap(required_outer, extra->param_source_rels))
! {
! /* Waste no memory when we reject a path here */
! bms_free(required_outer);
! return;
! }
}
/*
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
+ *
+ * TODO Need to consider aggregation here?
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra);
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! join_target = joinrel->gpi->target;
!
! join_path = (Path *) create_hashjoin_path(root, joinrel, jointype,
! &workspace,
! extra,
! outer_path, inner_path,
! extra->restrictlist,
! required_outer, hashclauses,
! join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, false,
! AGG_HASHED);
! }
! else if (add_path_precheck(joinrel,
workspace.startup_cost, workspace.total_cost,
! NIL, required_outer, grouped))
{
! add_path(joinrel, (Path *) join_path, grouped);
}
else
{
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 683,691 ****
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* If the inner path is parameterized, the parameterization must be fully
--- 1032,1048 ----
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *join_path;
+ PathTarget *join_target;
+
+ /* The same checks we do in try_hashjoin_path. */
+ Assert(!do_aggregate || grouped);
+ Assert(joinrel->gpi != NULL || !grouped);
/*
* If the inner path is parameterized, the parameterization must be fully
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 708,728 ****
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, NIL))
return;
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_hashjoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! NULL,
! hashclauses));
}
/*
--- 1065,1160 ----
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra);
!
! /*
! * Determine which target the join should produce.
! *
! * In the case of explicit aggregation, output of the join itself is
! * plain.
! */
! if (!grouped || do_aggregate)
! join_target = joinrel->reltarget;
! else
! {
! Assert(joinrel->gpi != NULL);
! join_target = joinrel->gpi->target;
! }
!
! join_path = (Path *) create_hashjoin_path(root, joinrel, jointype,
! &workspace,
! extra,
! outer_path, inner_path,
! extra->restrictlist, NULL,
! hashclauses, join_target);
!
! /* Do partial aggregation if needed. */
! if (do_aggregate)
! {
! create_grouped_path(root, joinrel, join_path, true, true, AGG_HASHED);
! }
! else if (add_partial_path_precheck(joinrel, workspace.total_cost,
! NIL, grouped))
! {
! add_partial_path(joinrel, (Path *) join_path , grouped);
! }
! }
!
! /*
! * Create a new grouped hash join path by joining a grouped path to plain
! * (non-grouped) one, or by joining 2 plain relations and applying grouping on
! * the result.
! *
! * Joining of 2 grouped paths is not supported. If a grouped relation A was
! * joined to grouped relation B, then the grouping of B reduces the number of
! * times each group of A is appears in the join output. This makes difference
! * for some aggregates, e.g. sum().
! *
! * If do_aggregate is true, neither input rel is grouped so we need to
! * aggregate the join result explicitly.
! *
! * partial argument tells whether the join path should be considered partial.
! */
! static void
! try_grouped_hashjoin_path(PlannerInfo *root,
! RelOptInfo *joinrel,
! Path *outer_path,
! Path *inner_path,
! List *hashclauses,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate,
! bool partial)
! {
! /*
! * Missing GroupedPathInfo indicates that we should not try to create a
! * grouped join.
! */
! if (joinrel->gpi == NULL)
return;
! /*
! * Reject the path if we're supposed to combine grouped and plain relation
! * but the grouped one does not evaluate all the relevant aggregates.
! */
! if (!do_aggregate &&
! !is_grouped_join_target_complete(root, joinrel->gpi->target,
! outer_path, inner_path))
! return;
!
! /*
! * As repeated aggregation doesn't seem to be attractive, make sure that
! * the resulting grouped relation is not parameterized.
! */
! if (outer_path->param_info != NULL || inner_path->param_info != NULL)
! return;
!
! if (!partial)
! try_hashjoin_path(root, joinrel, outer_path, inner_path, hashclauses,
! jointype, extra, true, do_aggregate);
! else
! try_partial_hashjoin_path(root, joinrel, outer_path, inner_path,
! hashclauses, jointype, extra, true,
! do_aggregate);
}
/*
*************** sort_inner_and_outer(PlannerInfo *root,
*** 773,779 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
Path *outer_path;
--- 1205,1244 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped)
! {
! if (!grouped)
! {
! sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, false, false, false);
! }
! else
! {
! /* Use all the supported strategies to generate grouped join. */
! sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, true, false, false);
! sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, false, true, false);
! sort_inner_and_outer_common(root, joinrel, outerrel, innerrel,
! jointype, extra, false, false, true);
! }
! }
!
! /*
! * TODO As merge_pathkeys shouldn't differ across execution, use a separate
! * function to derive them and pass them here in a list.
! */
! static void
! sort_inner_and_outer_common(PlannerInfo *root,
! RelOptInfo *joinrel,
! RelOptInfo *outerrel,
! RelOptInfo *innerrel,
! JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
{
JoinType save_jointype = jointype;
Path *outer_path;
*************** sort_inner_and_outer(PlannerInfo *root,
*** 782,787 ****
--- 1247,1253 ----
Path *cheapest_safe_inner = NULL;
List *all_pathkeys;
ListCell *l;
+ bool grouped_result;
/*
* We only consider the cheapest-total-cost input paths, since we are
*************** sort_inner_and_outer(PlannerInfo *root,
*** 796,803 ****
* against mergejoins with parameterized inputs; see comments in
* src/backend/optimizer/README.
*/
! outer_path = outerrel->cheapest_total_path;
! inner_path = innerrel->cheapest_total_path;
/*
* If either cheapest-total path is parameterized by the other rel, we
--- 1262,1288 ----
* against mergejoins with parameterized inputs; see comments in
* src/backend/optimizer/README.
*/
! if (grouped_outer)
! {
! if (outerrel->gpi != NULL && outerrel->gpi->pathlist != NIL)
! outer_path = linitial(outerrel->gpi->pathlist);
! else
! return;
! }
! else
! outer_path = outerrel->cheapest_total_path;
!
! if (grouped_inner)
! {
! if (innerrel->gpi != NULL && innerrel->gpi->pathlist != NIL)
! inner_path = linitial(innerrel->gpi->pathlist);
! else
! return;
! }
! else
! inner_path = innerrel->cheapest_total_path;
!
! grouped_result = grouped_outer || grouped_inner || do_aggregate;
/*
* If either cheapest-total path is parameterized by the other rel, we
*************** sort_inner_and_outer(PlannerInfo *root,
*** 843,855 ****
outerrel->partial_pathlist != NIL &&
bms_is_empty(joinrel->lateral_relids))
{
! cheapest_partial_outer = (Path *) linitial(outerrel->partial_pathlist);
if (inner_path->parallel_safe)
cheapest_safe_inner = inner_path;
else if (save_jointype != JOIN_UNIQUE_INNER)
cheapest_safe_inner =
! get_cheapest_parallel_safe_total_inner(innerrel->pathlist);
}
/*
--- 1328,1377 ----
outerrel->partial_pathlist != NIL &&
bms_is_empty(joinrel->lateral_relids))
{
! if (grouped_outer)
! {
! if (outerrel->gpi != NULL && outerrel->gpi->partial_pathlist != NIL)
! cheapest_partial_outer = (Path *)
! linitial(outerrel->gpi->partial_pathlist);
! else
! return;
! }
! else
! cheapest_partial_outer = (Path *)
! linitial(outerrel->partial_pathlist);
!
! if (grouped_inner)
! {
! if (innerrel->gpi != NULL && innerrel->gpi->pathlist != NIL)
! inner_path = linitial(innerrel->gpi->pathlist);
! else
! return;
! }
! else
! inner_path = innerrel->cheapest_total_path;
if (inner_path->parallel_safe)
cheapest_safe_inner = inner_path;
else if (save_jointype != JOIN_UNIQUE_INNER)
+ {
+ List *inner_pathlist;
+
+ if (!grouped_inner)
+ inner_pathlist = innerrel->pathlist;
+ else
+ {
+ Assert(innerrel->gpi != NULL);
+ inner_pathlist = innerrel->gpi->pathlist;
+ }
+
+ /*
+ * All the grouped paths should be unparameterized, so the
+ * function is overly stringent in the grouped_inner case, but
+ * still useful.
+ */
cheapest_safe_inner =
! get_cheapest_parallel_safe_total_inner(inner_pathlist);
! }
}
/*
*************** sort_inner_and_outer(PlannerInfo *root,
*** 925,957 ****
* properly. try_mergejoin_path will detect that case and suppress an
* explicit sort step, so we needn't do so here.
*/
! try_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! merge_pathkeys,
! cur_mergeclauses,
! outerkeys,
! innerkeys,
! jointype,
! extra,
! false);
/*
* If we have partial outer and parallel safe inner path then try
* partial mergejoin path.
*/
if (cheapest_partial_outer && cheapest_safe_inner)
! try_partial_mergejoin_path(root,
! joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! merge_pathkeys,
! cur_mergeclauses,
! outerkeys,
! innerkeys,
! jointype,
! extra);
}
}
--- 1447,1505 ----
* properly. try_mergejoin_path will detect that case and suppress an
* explicit sort step, so we needn't do so here.
*/
! if (!grouped_result)
! try_mergejoin_path(root,
! joinrel,
! outer_path,
! inner_path,
! merge_pathkeys,
! cur_mergeclauses,
! outerkeys,
! innerkeys,
! jointype,
! extra,
! false, false, false);
! else
! {
! try_mergejoin_path_common(root, joinrel, outer_path, inner_path,
! merge_pathkeys, cur_mergeclauses,
! outerkeys, innerkeys, jointype, extra,
! false,
! grouped_outer, grouped_inner,
! do_aggregate);
! }
/*
* If we have partial outer and parallel safe inner path then try
* partial mergejoin path.
*/
if (cheapest_partial_outer && cheapest_safe_inner)
! {
! if (!grouped_result)
! {
! try_partial_mergejoin_path(root,
! joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! merge_pathkeys,
! cur_mergeclauses,
! outerkeys,
! innerkeys,
! jointype,
! extra, false, false);
! }
! else
! {
! try_mergejoin_path_common(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! merge_pathkeys, cur_mergeclauses,
! outerkeys, innerkeys, jointype, extra,
! true,
! grouped_outer, grouped_inner,
! do_aggregate);
! }
! }
}
}
*************** sort_inner_and_outer(PlannerInfo *root,
*** 968,973 ****
--- 1516,1529 ----
* some sort key requirements). So, we consider truncations of the
* mergeclause list as well as the full list. (Ideally we'd consider all
* subsets of the mergeclause list, but that seems way too expensive.)
+ *
+ * grouped_outer - is outerpath grouped?
+ * grouped_inner - use grouped paths of innerrel?
+ * do_aggregate - apply (partial) aggregation to the output?
+ *
+ * TODO If subsequent calls often differ only by the 3 arguments above,
+ * consider a workspace structure to share useful info (eg merge clauses)
+ * across calls.
*/
static void
generate_mergejoin_paths(PlannerInfo *root,
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 979,985 ****
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial)
{
List *mergeclauses;
List *innersortkeys;
--- 1535,1544 ----
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial,
! bool grouped_outer,
! bool grouped_inner,
! bool do_aggregate)
{
List *mergeclauses;
List *innersortkeys;
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1030,1046 ****
* try_mergejoin_path will do the right thing if inner_cheapest_total is
* already correctly sorted.)
*/
! try_mergejoin_path(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! mergeclauses,
! NIL,
! innersortkeys,
! jointype,
! extra,
! is_partial);
/* Can't do anything else if inner path needs to be unique'd */
if (save_jointype == JOIN_UNIQUE_INNER)
--- 1589,1606 ----
* try_mergejoin_path will do the right thing if inner_cheapest_total is
* already correctly sorted.)
*/
! try_mergejoin_path_common(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! mergeclauses,
! NIL,
! innersortkeys,
! jointype,
! extra,
! is_partial,
! grouped_outer, grouped_inner, do_aggregate);
/* Can't do anything else if inner path needs to be unique'd */
if (save_jointype == JOIN_UNIQUE_INNER)
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1096,1111 ****
for (sortkeycnt = num_sortkeys; sortkeycnt > 0; sortkeycnt--)
{
Path *innerpath;
List *newclauses = NIL;
/*
* Look for an inner path ordered well enough for the first
* 'sortkeycnt' innersortkeys. NB: trialsortkeys list is modified
* destructively, which is why we made a copy...
*/
trialsortkeys = list_truncate(trialsortkeys, sortkeycnt);
! innerpath = get_cheapest_path_for_pathkeys(innerrel->pathlist,
trialsortkeys,
NULL,
TOTAL_COST,
--- 1656,1677 ----
for (sortkeycnt = num_sortkeys; sortkeycnt > 0; sortkeycnt--)
{
+ List *inner_pathlist = NIL;
Path *innerpath;
List *newclauses = NIL;
+ if (!grouped_inner)
+ inner_pathlist = innerrel->pathlist;
+ else if (innerrel->gpi != NULL)
+ inner_pathlist = innerrel->gpi->pathlist;
+
/*
* Look for an inner path ordered well enough for the first
* 'sortkeycnt' innersortkeys. NB: trialsortkeys list is modified
* destructively, which is why we made a copy...
*/
trialsortkeys = list_truncate(trialsortkeys, sortkeycnt);
! innerpath = get_cheapest_path_for_pathkeys(inner_pathlist,
trialsortkeys,
NULL,
TOTAL_COST,
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1128,1148 ****
}
else
newclauses = mergeclauses;
! try_mergejoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial);
cheapest_total_inner = innerpath;
}
/* Same on the basis of cheapest startup cost ... */
! innerpath = get_cheapest_path_for_pathkeys(innerrel->pathlist,
trialsortkeys,
NULL,
STARTUP_COST,
--- 1694,1718 ----
}
else
newclauses = mergeclauses;
!
! try_mergejoin_path_common(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial,
! grouped_outer, grouped_inner,
! do_aggregate);
!
cheapest_total_inner = innerpath;
}
/* Same on the basis of cheapest startup cost ... */
! innerpath = get_cheapest_path_for_pathkeys(inner_pathlist,
trialsortkeys,
NULL,
STARTUP_COST,
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1173,1189 ****
else
newclauses = mergeclauses;
}
! try_mergejoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial);
}
cheapest_startup_inner = innerpath;
}
--- 1743,1761 ----
else
newclauses = mergeclauses;
}
! try_mergejoin_path_common(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! newclauses,
! NIL,
! NIL,
! jointype,
! extra,
! is_partial,
! grouped_outer, grouped_inner,
! do_aggregate);
}
cheapest_startup_inner = innerpath;
}
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1218,1223 ****
--- 1790,1797 ----
* 'innerrel' is the inner join relation
* 'jointype' is the type of join to do
* 'extra' contains additional input values
+ * 'grouped' indicates that the at least one relation in the join has been
+ * aggregated.
*/
static void
match_unsorted_outer(PlannerInfo *root,
*************** match_unsorted_outer(PlannerInfo *root,
*** 1225,1231 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
bool nestjoinOK;
--- 1799,1806 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped)
{
JoinType save_jointype = jointype;
bool nestjoinOK;
*************** match_unsorted_outer(PlannerInfo *root,
*** 1235,1240 ****
--- 1810,1837 ----
ListCell *lc1;
/*
+ * If grouped join path is requested, we ignore cases where either input
+ * path needs to be unique. For each side we should expect either grouped
+ * or plain relation, which differ quite a bit.
+ *
+ * XXX Although unique-ification of grouped path might result in too
+ * expensive input path (note that grouped input relation is not
+ * necessarily unique, regardless the grouping keys --- one or more plain
+ * relation could already have been joined to it), we might want to
+ * unique-ify the input relation in the future at least in the case it's a
+ * plain relation.
+ *
+ * (Materialization is not involved in grouped paths for similar reasons.)
+ */
+ if (grouped &&
+ (jointype == JOIN_UNIQUE_OUTER || jointype == JOIN_UNIQUE_INNER))
+ return;
+
+ /* No grouped join w/o grouped target. */
+ if (grouped && joinrel->gpi == NULL)
+ return;
+
+ /*
* Nestloop only supports inner, left, semi, and anti joins. Also, if we
* are doing a right or full mergejoin, we must use *all* the mergeclauses
* as join clauses, else we will not have a valid plan. (Although these
*************** match_unsorted_outer(PlannerInfo *root,
*** 1290,1296 ****
create_unique_path(root, innerrel, inner_cheapest_total, extra->sjinfo);
Assert(inner_cheapest_total);
}
! else if (nestjoinOK)
{
/*
* Consider materializing the cheapest inner path, unless
--- 1887,1893 ----
create_unique_path(root, innerrel, inner_cheapest_total, extra->sjinfo);
Assert(inner_cheapest_total);
}
! else if (nestjoinOK && !grouped)
{
/*
* Consider materializing the cheapest inner path, unless
*************** match_unsorted_outer(PlannerInfo *root,
*** 1321,1326 ****
--- 1918,1925 ----
*/
if (save_jointype == JOIN_UNIQUE_OUTER)
{
+ Assert(!grouped);
+
if (outerpath != outerrel->cheapest_total_path)
continue;
outerpath = (Path *) create_unique_path(root, outerrel,
*************** match_unsorted_outer(PlannerInfo *root,
*** 1348,1354 ****
inner_cheapest_total,
merge_pathkeys,
jointype,
! extra);
}
else if (nestjoinOK)
{
--- 1947,1954 ----
inner_cheapest_total,
merge_pathkeys,
jointype,
! extra,
! false, false);
}
else if (nestjoinOK)
{
*************** match_unsorted_outer(PlannerInfo *root,
*** 1364,1387 ****
{
Path *innerpath = (Path *) lfirst(lc2);
! try_nestloop_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! jointype,
! extra);
}
! /* Also consider materialized form of the cheapest inner path */
! if (matpath != NULL)
try_nestloop_path(root,
joinrel,
outerpath,
matpath,
merge_pathkeys,
jointype,
! extra);
}
/* Can't do anything else if outer path needs to be unique'd */
--- 1964,2009 ----
{
Path *innerpath = (Path *) lfirst(lc2);
! if (!grouped)
! try_nestloop_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! jointype,
! extra, false, false);
! else
! {
! /*
! * Since both input paths are plain, request explicit
! * aggregation.
! */
! try_grouped_nestloop_path(root,
! joinrel,
! outerpath,
! innerpath,
! merge_pathkeys,
! jointype,
! extra,
! true,
! false);
! }
}
! /*
! * Also consider materialized form of the cheapest inner path.
! *
! * (There's no matpath for grouped join.)
! */
! if (matpath != NULL && !grouped)
try_nestloop_path(root,
joinrel,
outerpath,
matpath,
merge_pathkeys,
jointype,
! extra,
! false, false);
}
/* Can't do anything else if outer path needs to be unique'd */
*************** match_unsorted_outer(PlannerInfo *root,
*** 1396,1402 ****
generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
save_jointype, extra, useallclauses,
inner_cheapest_total, merge_pathkeys,
! false);
}
/*
--- 2018,2094 ----
generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
save_jointype, extra, useallclauses,
inner_cheapest_total, merge_pathkeys,
! false, false, false, grouped);
!
! /* Try to join the plain outer relation to grouped inner. */
! if (grouped && nestjoinOK &&
! save_jointype != JOIN_UNIQUE_OUTER &&
! save_jointype != JOIN_UNIQUE_INNER &&
! innerrel->gpi != NULL && outerrel->gpi == NULL)
! {
! Path *inner_cheapest_grouped = (Path *) linitial(innerrel->gpi->pathlist);
!
! if (PATH_PARAM_BY_REL(inner_cheapest_grouped, outerrel))
! continue;
!
! /* XXX Shouldn't Assert() be used here instead? */
! if (PATH_PARAM_BY_REL(outerpath, innerrel))
! continue;
!
! /*
! * Only outer grouped path is interesting in this case: grouped
! * path on the inner side of NL join would imply repeated
! * aggregation somewhere in the inner path.
! */
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! save_jointype, extra, useallclauses,
! inner_cheapest_grouped, merge_pathkeys,
! false, false, true, false);
! }
! }
!
! /*
! * Combine grouped outer and plain inner paths.
! */
! if (grouped && nestjoinOK &&
! save_jointype != JOIN_UNIQUE_OUTER &&
! save_jointype != JOIN_UNIQUE_INNER)
! {
! /*
! * If the inner rel had a grouped target, its plain paths should be
! * ignored. Otherwise we could create grouped paths with different
! * targets.
! */
! if (outerrel->gpi != NULL && innerrel->gpi == NULL &&
! inner_cheapest_total != NULL)
! {
! /* Nested loop paths. */
! foreach(lc1, outerrel->gpi->pathlist)
! {
! Path *outerpath = (Path *) lfirst(lc1);
! List *merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
! outerpath->pathkeys);
!
! if (PATH_PARAM_BY_REL(outerpath, innerrel))
! continue;
!
! try_grouped_nestloop_path(root,
! joinrel,
! outerpath,
! inner_cheapest_total,
! merge_pathkeys,
! jointype,
! extra,
! false,
! false);
!
! /* Merge join paths. */
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! save_jointype, extra, useallclauses,
! inner_cheapest_total, merge_pathkeys,
! false, true, false, false);
! }
! }
}
/*
*************** match_unsorted_outer(PlannerInfo *root,
*** 1416,1423 ****
bms_is_empty(joinrel->lateral_relids))
{
if (nestjoinOK)
! consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra);
/*
* If inner_cheapest_total is NULL or non parallel-safe then find the
--- 2108,2128 ----
bms_is_empty(joinrel->lateral_relids))
{
if (nestjoinOK)
! {
! if (!grouped)
! /* Plain partial paths. */
! consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra, false, false);
! else
! {
! /* Grouped partial paths with explicit aggregation. */
! consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra, true, true);
! /* Grouped partial paths w/o explicit aggregation. */
! consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra, true, false);
! }
! }
/*
* If inner_cheapest_total is NULL or non parallel-safe then find the
*************** match_unsorted_outer(PlannerInfo *root,
*** 1437,1443 ****
if (inner_cheapest_total)
consider_parallel_mergejoin(root, joinrel, outerrel, innerrel,
save_jointype, extra,
! inner_cheapest_total);
}
}
--- 2142,2148 ----
if (inner_cheapest_total)
consider_parallel_mergejoin(root, joinrel, outerrel, innerrel,
save_jointype, extra,
! inner_cheapest_total, grouped);
}
}
*************** consider_parallel_mergejoin(PlannerInfo
*** 1460,1469 ****
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total)
{
ListCell *lc1;
/* generate merge join path for each partial outer path */
foreach(lc1, outerrel->partial_pathlist)
{
--- 2165,2183 ----
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total,
! bool grouped)
{
ListCell *lc1;
+ if (grouped)
+ {
+ /* TODO Consider if these types should be supported. */
+ if (jointype == JOIN_UNIQUE_OUTER ||
+ jointype == JOIN_UNIQUE_INNER)
+ return;
+ }
+
/* generate merge join path for each partial outer path */
foreach(lc1, outerrel->partial_pathlist)
{
*************** consider_parallel_mergejoin(PlannerInfo
*** 1476,1484 ****
merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
outerpath->pathkeys);
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath, jointype,
! extra, false, inner_cheapest_total,
! merge_pathkeys, true);
}
}
--- 2190,2245 ----
merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
outerpath->pathkeys);
! if (!grouped)
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! jointype, extra, false,
! inner_cheapest_total, merge_pathkeys,
! true,
! false, false, false);
! else
! {
! /*
! * Create grouped join by joining plain rels and aggregating the
! * result.
! */
! Assert(joinrel->gpi != NULL);
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! jointype, extra, false,
! inner_cheapest_total, merge_pathkeys,
! true, false, false, true);
!
! /* Combine the plain outer with grouped inner one(s). */
! if (outerrel->gpi == NULL && innerrel->gpi != NULL)
! {
! Path *inner_cheapest_grouped = (Path *)
! linitial(innerrel->gpi->pathlist);
!
! if (inner_cheapest_grouped != NULL &&
! inner_cheapest_grouped->parallel_safe)
! generate_mergejoin_paths(root, joinrel, innerrel,
! outerpath, jointype, extra,
! false, inner_cheapest_grouped,
! merge_pathkeys,
! true, false, true, false);
! }
! }
! }
!
! /* In addition, try to join grouped outer to plain inner one(s). */
! if (grouped && outerrel->gpi != NULL && innerrel->gpi == NULL)
! {
! foreach(lc1, outerrel->gpi->partial_pathlist)
! {
! Path *outerpath = (Path *) lfirst(lc1);
! List *merge_pathkeys;
!
! merge_pathkeys = build_join_pathkeys(root, joinrel, jointype,
! outerpath->pathkeys);
! generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
! jointype, extra, false,
! inner_cheapest_total, merge_pathkeys,
! true, true, false, false);
! }
}
}
*************** consider_parallel_nestloop(PlannerInfo *
*** 1499,1513 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
ListCell *lc1;
if (jointype == JOIN_UNIQUE_INNER)
jointype = JOIN_INNER;
! foreach(lc1, outerrel->partial_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *pathkeys;
--- 2260,2304 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped, bool do_aggregate)
{
JoinType save_jointype = jointype;
+ List *outer_pathlist;
ListCell *lc1;
+ if (grouped)
+ {
+ /* TODO Consider if these types should be supported. */
+ if (save_jointype == JOIN_UNIQUE_OUTER ||
+ save_jointype == JOIN_UNIQUE_INNER)
+ return;
+ }
+
if (jointype == JOIN_UNIQUE_INNER)
jointype = JOIN_INNER;
! if (!grouped || do_aggregate)
! {
! /*
! * If creating grouped paths by explicit aggregation, the input paths
! * must be plain.
! */
! outer_pathlist = outerrel->partial_pathlist;
! }
! else if (outerrel->gpi != NULL)
! {
! /*
! * Only the outer paths are accepted as grouped when we try to combine
! * grouped and plain ones. Grouped inner path implies repeated
! * aggregation, which doesn't sound as a good idea.
! */
! outer_pathlist = outerrel->gpi->partial_pathlist;
! }
! else
! return;
!
! foreach(lc1, outer_pathlist)
{
Path *outerpath = (Path *) lfirst(lc1);
List *pathkeys;
*************** consider_parallel_nestloop(PlannerInfo *
*** 1538,1544 ****
* inner paths, but right now create_unique_path is not on board
* with that.)
*/
! if (save_jointype == JOIN_UNIQUE_INNER)
{
if (innerpath != innerrel->cheapest_total_path)
continue;
--- 2329,2335 ----
* inner paths, but right now create_unique_path is not on board
* with that.)
*/
! if (save_jointype == JOIN_UNIQUE_INNER && !grouped)
{
if (innerpath != innerrel->cheapest_total_path)
continue;
*************** consider_parallel_nestloop(PlannerInfo *
*** 1548,1555 ****
Assert(innerpath);
}
! try_partial_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra);
}
}
}
--- 2339,2364 ----
Assert(innerpath);
}
! if (!grouped)
! try_partial_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra,
! false, false);
! else if (do_aggregate)
! {
! /* Request aggregation as both input rels are plain. */
! try_grouped_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra,
! true, true);
! }
! /*
! * Only combine the grouped outer path with the plain inner if the
! * inner relation cannot produce grouped paths. Otherwise we could
! * generate grouped paths with different targets.
! */
! else if (innerrel->gpi == NULL)
! try_grouped_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra,
! false, true);
}
}
}
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1571,1583 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
bool isouterjoin = IS_OUTER_JOIN(jointype);
List *hashclauses;
ListCell *l;
/*
* We need to build only one hashclauses list for any given pair of outer
* and inner relations; all of the hashable clauses will be used as keys.
--- 2380,2397 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool grouped)
{
JoinType save_jointype = jointype;
bool isouterjoin = IS_OUTER_JOIN(jointype);
List *hashclauses;
ListCell *l;
+ /* No grouped join w/o grouped target. */
+ if (grouped && joinrel->gpi == NULL)
+ return;
+
/*
* We need to build only one hashclauses list for any given pair of outer
* and inner relations; all of the hashable clauses will be used as keys.
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1627,1632 ****
--- 2441,2449 ----
* can't use a hashjoin. (There's no use looking for alternative
* input paths, since these should already be the least-parameterized
* available paths.)
+ *
+ * (The same check should work for grouped paths, as these don't
+ * differ in parameterization.)
*/
if (PATH_PARAM_BY_REL(cheapest_total_outer, innerrel) ||
PATH_PARAM_BY_REL(cheapest_total_inner, outerrel))
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1646,1652 ****
cheapest_total_inner,
hashclauses,
jointype,
! extra);
/* no possibility of cheap startup here */
}
else if (jointype == JOIN_UNIQUE_INNER)
--- 2463,2470 ----
cheapest_total_inner,
hashclauses,
jointype,
! extra,
! false, false);
/* no possibility of cheap startup here */
}
else if (jointype == JOIN_UNIQUE_INNER)
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1662,1668 ****
cheapest_total_inner,
hashclauses,
jointype,
! extra);
if (cheapest_startup_outer != NULL &&
cheapest_startup_outer != cheapest_total_outer)
try_hashjoin_path(root,
--- 2480,2487 ----
cheapest_total_inner,
hashclauses,
jointype,
! extra,
! false, false);
if (cheapest_startup_outer != NULL &&
cheapest_startup_outer != cheapest_total_outer)
try_hashjoin_path(root,
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1671,1733 ****
cheapest_total_inner,
hashclauses,
jointype,
! extra);
}
else
{
! /*
! * For other jointypes, we consider the cheapest startup outer
! * together with the cheapest total inner, and then consider
! * pairings of cheapest-total paths including parameterized ones.
! * There is no use in generating parameterized paths on the basis
! * of possibly cheap startup cost, so this is sufficient.
! */
! ListCell *lc1;
! ListCell *lc2;
!
! if (cheapest_startup_outer != NULL)
! try_hashjoin_path(root,
! joinrel,
! cheapest_startup_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra);
!
! foreach(lc1, outerrel->cheapest_parameterized_paths)
{
- Path *outerpath = (Path *) lfirst(lc1);
-
/*
! * We cannot use an outer path that is parameterized by the
! * inner rel.
*/
! if (PATH_PARAM_BY_REL(outerpath, innerrel))
! continue;
! foreach(lc2, innerrel->cheapest_parameterized_paths)
{
! Path *innerpath = (Path *) lfirst(lc2);
/*
! * We cannot use an inner path that is parameterized by
! * the outer rel, either.
*/
! if (PATH_PARAM_BY_REL(innerpath, outerrel))
continue;
! if (outerpath == cheapest_startup_outer &&
! innerpath == cheapest_total_inner)
! continue; /* already tried it */
! try_hashjoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! hashclauses,
! jointype,
! extra);
}
}
}
--- 2490,2643 ----
cheapest_total_inner,
hashclauses,
jointype,
! extra,
! false, false);
}
else
{
! if (!grouped)
{
/*
! * For other jointypes, we consider the cheapest startup outer
! * together with the cheapest total inner, and then consider
! * pairings of cheapest-total paths including parameterized
! * ones. There is no use in generating parameterized paths on
! * the basis of possibly cheap startup cost, so this is
! * sufficient.
*/
! ListCell *lc1;
! if (cheapest_startup_outer != NULL)
! try_hashjoin_path(root,
! joinrel,
! cheapest_startup_outer,
! cheapest_total_inner,
! hashclauses,
! jointype,
! extra,
! false, false);
!
! foreach(lc1, outerrel->cheapest_parameterized_paths)
{
! Path *outerpath = (Path *) lfirst(lc1);
! ListCell *lc2;
/*
! * We cannot use an outer path that is parameterized by the
! * inner rel.
*/
! if (PATH_PARAM_BY_REL(outerpath, innerrel))
continue;
! foreach(lc2, innerrel->cheapest_parameterized_paths)
! {
! Path *innerpath = (Path *) lfirst(lc2);
! /*
! * We cannot use an inner path that is parameterized by
! * the outer rel, either.
! */
! if (PATH_PARAM_BY_REL(innerpath, outerrel))
! continue;
!
! if (outerpath == cheapest_startup_outer &&
! innerpath == cheapest_total_inner)
! continue; /* already tried it */
!
! try_hashjoin_path(root,
! joinrel,
! outerpath,
! innerpath,
! hashclauses,
! jointype,
! extra,
! false, false);
! }
! }
! }
! else
! {
! /* Create grouped paths if possible. */
! /*
! * TODO
! *
! * Consider processing JOIN_UNIQUE_INNER and JOIN_UNIQUE_OUTER
! * join types, ie perform grouping of the inner / outer rel if
! * it's not unique yet and if the grouping is legal.
! */
! if (jointype == JOIN_UNIQUE_OUTER ||
! jointype == JOIN_UNIQUE_INNER)
! return;
!
! /*
! * Join grouped relation to non-grouped one.
! *
! * Do not use plain path of the input rel whose target does
! * have GroupedPahtInfo. For example (assuming that join of
! * two grouped rels is not supported), the only way to
! * evaluate SELECT sum(a.x), sum(b.y) ... is to join "a" and
! * "b" and aggregate the result. Otherwise the path target
! * wouldn't match joinrel->gpi->target. TODO Move this comment
! * elsewhere as it seems common to all join kinds.
! */
! /*
! * TODO Allow outer join if the grouped rel is on the
! * non-nullable side.
! */
! if (jointype == JOIN_INNER)
! {
! Path *grouped_path, *plain_path;
!
! if (outerrel->gpi != NULL &&
! outerrel->gpi->pathlist != NIL &&
! innerrel->gpi == NULL)
! {
! grouped_path = (Path *)
! linitial(outerrel->gpi->pathlist);
! plain_path = cheapest_total_inner;
! try_grouped_hashjoin_path(root, joinrel,
! grouped_path, plain_path,
! hashclauses, jointype,
! extra, false, false);
! }
! else if (innerrel->gpi != NULL &&
! innerrel->gpi->pathlist != NIL &&
! outerrel->gpi == NULL)
! {
! grouped_path = (Path *)
! linitial(innerrel->gpi->pathlist);
! plain_path = cheapest_total_outer;
! try_grouped_hashjoin_path(root, joinrel, plain_path,
! grouped_path, hashclauses,
! jointype, extra,
! false, false);
!
! if (cheapest_startup_outer != NULL &&
! cheapest_startup_outer != cheapest_total_outer)
! {
! plain_path = cheapest_startup_outer;
! try_grouped_hashjoin_path(root, joinrel,
! plain_path,
! grouped_path,
! hashclauses,
! jointype, extra,
! false, false);
! }
! }
}
+
+ /*
+ * Try to join plain relations and make a grouped rel out of
+ * the join.
+ *
+ * Since aggregation needs the whole relation, we are only
+ * interested in total costs.
+ */
+ try_grouped_hashjoin_path(root, joinrel,
+ cheapest_total_outer,
+ cheapest_total_inner,
+ hashclauses,
+ jointype, extra, true, false);
}
}
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1765,1777 ****
cheapest_safe_inner =
get_cheapest_parallel_safe_total_inner(innerrel->pathlist);
! if (cheapest_safe_inner != NULL)
! try_partial_hashjoin_path(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! hashclauses, jointype, extra);
}
}
}
/*
--- 2675,2898 ----
cheapest_safe_inner =
get_cheapest_parallel_safe_total_inner(innerrel->pathlist);
! if (!grouped)
! {
! if (cheapest_safe_inner != NULL)
! try_partial_hashjoin_path(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! hashclauses, jointype, extra,
! false, false);
! }
! else if (joinrel->gpi != NULL)
! {
! /*
! * Grouped partial path.
! *
! * 1. Apply aggregation to the plain partial join path.
! */
! if (cheapest_safe_inner != NULL)
! try_grouped_hashjoin_path(root, joinrel,
! cheapest_partial_outer,
! cheapest_safe_inner,
! hashclauses,
! jointype, extra, true, true);
!
! /*
! * 2. Join the cheapest partial grouped outer path (if one
! * exists) to cheapest_safe_inner (there's no reason to look
! * for another inner path than what we used for non-grouped
! * partial join path).
! */
! if (outerrel->gpi != NULL &&
! outerrel->gpi->partial_pathlist != NIL &&
! innerrel->gpi == NULL &&
! cheapest_safe_inner != NULL)
! {
! Path *outer_path;
!
! outer_path = (Path *)
! linitial(outerrel->gpi->partial_pathlist);
!
! try_grouped_hashjoin_path(root, joinrel, outer_path,
! cheapest_safe_inner,
! hashclauses,
! jointype, extra, false, true);
! }
!
! /*
! * 3. Join the cheapest_partial_outer path (again, no reason
! * to use different outer path than the one we used for plain
! * partial join) to the cheapest grouped inner path if the
! * latter exists and is parallel-safe.
! */
! if (innerrel->gpi != NULL &&
! innerrel->gpi->pathlist != NIL &&
! outerrel->gpi == NULL)
! {
! Path *inner_path;
!
! inner_path = (Path *) linitial(innerrel->gpi->pathlist);
!
! if (inner_path->parallel_safe)
! try_grouped_hashjoin_path(root, joinrel,
! cheapest_partial_outer,
! inner_path,
! hashclauses,
! jointype, extra,
! false, true);
! }
!
! /*
! * Other combinations seem impossible because: 1. At most 1
! * input relation of the join can be grouped, 2. the inner
! * path must not be partial.
! */
! }
! }
! }
! }
!
! /*
! * Do the input paths emit all the aggregates contained in the grouped target
! * of the join?
! *
! * The point is that one input relation might be unable to evaluate some
! * aggregate(s), so it'll only generate plain paths. It's wrong to combine
! * such plain paths with grouped ones that the other input rel might be able
! * to generate because the result would miss the aggregate(s) the first
! * relation failed to evaluate.
! *
! * TODO For better efficiency, consider storing Bitmapset of
! * GroupedVarInfo.gvid in GroupedPathInfo.
! */
! static bool
! is_grouped_join_target_complete(PlannerInfo *root, PathTarget *jointarget,
! Path *outer_path, Path *inner_path)
! {
! RelOptInfo *outer_rel = outer_path->parent;
! RelOptInfo *inner_rel = inner_path->parent;
! ListCell *l1;
!
! /*
! * Join of two grouped relations is not supported.
! *
! * This actually isn't check of target completeness --- can it be located
! * elsewhere?
! */
! if (outer_rel->gpi != NULL && inner_rel->gpi != NULL)
! return false;
!
! foreach(l1, jointarget->exprs)
! {
! Expr *expr = (Expr *) lfirst(l1);
! GroupedVar *gvar;
! GroupedVarInfo *gvi = NULL;
! ListCell *l2;
! bool found = false;
!
! /* Only interested in aggregates. */
! if (!IsA(expr, GroupedVar))
! continue;
!
! gvar = castNode(GroupedVar, expr);
!
! /* Find the corresponding GroupedVarInfo. */
! foreach(l2, root->grouped_var_list)
! {
! GroupedVarInfo *gvi_tmp = castNode(GroupedVarInfo, lfirst(l2));
!
! if (gvi_tmp->gvid == gvar->gvid)
! {
! gvi = gvi_tmp;
! break;
! }
! }
! Assert(gvi != NULL);
!
! /*
! * If any aggregate references both input relations, something went
! * wrong during construction of one of the input targets: one input
! * rel is grouped, but no grouping target should have been created for
! * it if some aggregate required more than that input rel.
! */
! Assert(gvi->gv_eval_at == NULL ||
! !(bms_overlap(gvi->gv_eval_at, outer_rel->relids) &&
! bms_overlap(gvi->gv_eval_at, inner_rel->relids)));
!
! /*
! * If the aggregate belongs to the plain relation, it probably
! * means that non-grouping expression made aggregation of that
! * input relation impossible. Since that expression is not
! * necessarily emitted by the current join, aggregation might be
! * possible here. On the other hand, aggregation of a join which
! * already contains a grouped relation does not seem too
! * beneficial.
! *
! * XXX The condition below is also met if the query contains both
! * "star aggregate" and a normal one. Since the earlier can be
! * added to any base relation, and since we don't support join of
! * 2 grouped relations, join of arbitrary 2 relations will always
! * result in a plain relation.
! *
! * XXX If we conclude that aggregation is worth, only consider
! * this test failed if target usable for aggregation cannot be
! * created (i.e. the non-grouping expression is in the output of
! * the current join).
! */
! if ((outer_rel->gpi == NULL &&
! bms_overlap(gvi->gv_eval_at, outer_rel->relids))
! || (inner_rel->gpi == NULL &&
! bms_overlap(gvi->gv_eval_at, inner_rel->relids)))
! return false;
!
! /* Look for the aggregate in the input targets. */
! if (outer_rel->gpi != NULL)
! {
! /* No more than one input path should be grouped. */
! Assert(inner_rel->gpi == NULL);
!
! foreach(l2, outer_path->pathtarget->exprs)
! {
! expr = (Expr *) lfirst(l2);
!
! if (!IsA(expr, GroupedVar))
! continue;
!
! gvar = castNode(GroupedVar, expr);
! if (gvar->gvid == gvi->gvid)
! {
! found = true;
! break;
! }
! }
! }
! else if (!found && inner_rel->gpi != NULL)
! {
! Assert(outer_rel->gpi == NULL);
!
! foreach(l2, inner_path->pathtarget->exprs)
! {
! expr = (Expr *) lfirst(l2);
!
! if (!IsA(expr, GroupedVar))
! continue;
!
! gvar = castNode(GroupedVar, expr);
! if (gvar->gvid == gvi->gvid)
! {
! found = true;
! break;
! }
! }
}
+
+ /* Even a single missing aggregate causes the whole test to fail. */
+ if (!found)
+ return false;
}
+
+ return true;
}
/*
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
new file mode 100644
index 5a68de3..eadf3ee
*** a/src/backend/optimizer/path/joinrels.c
--- b/src/backend/optimizer/path/joinrels.c
*************** mark_dummy_rel(RelOptInfo *rel)
*** 1217,1223 ****
rel->partial_pathlist = NIL;
/* Set up the dummy path */
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL));
/* Set or update cheapest_total_path and related fields */
set_cheapest(rel);
--- 1217,1223 ----
rel->partial_pathlist = NIL;
/* Set up the dummy path */
! add_path(rel, (Path *) create_append_path(rel, NIL, NULL, 0, NIL), false);
/* Set or update cheapest_total_path and related fields */
set_cheapest(rel);
diff --git a/src/backend/optimizer/path/tidpath.c b/src/backend/optimizer/path/tidpath.c
new file mode 100644
index a2fe661..91d855c
*** a/src/backend/optimizer/path/tidpath.c
--- b/src/backend/optimizer/path/tidpath.c
*************** create_tidscan_paths(PlannerInfo *root,
*** 266,270 ****
if (tidquals)
add_path(rel, (Path *) create_tidscan_path(root, rel, tidquals,
! required_outer));
}
--- 266,270 ----
if (tidquals)
add_path(rel, (Path *) create_tidscan_path(root, rel, tidquals,
! required_outer), false);
}
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
new file mode 100644
index ebd442a..a7be1b2
*** a/src/backend/optimizer/plan/initsplan.c
--- b/src/backend/optimizer/plan/initsplan.c
***************
*** 14,19 ****
--- 14,20 ----
*/
#include "postgres.h"
+ #include "access/sysattr.h"
#include "catalog/pg_type.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/clauses.h"
***************
*** 26,31 ****
--- 27,33 ----
#include "optimizer/planner.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "parser/analyze.h"
#include "rewrite/rewriteManip.h"
*************** typedef struct PostponedQual
*** 45,50 ****
--- 47,53 ----
} PostponedQual;
+ static void create_grouped_var_infos(PlannerInfo *root);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
*************** add_vars_to_targetlist(PlannerInfo *root
*** 240,245 ****
--- 243,532 ----
}
}
+ /*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate and setup
+ * GroupedPathInfo for each base relation that can product grouped paths.
+ *
+ * XXX In the future we might want to create GroupedVarInfo for grouping
+ * expressions too, so that grouping key is not limited to plain Var if the
+ * grouping takes place below the top-level join.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+ extern void
+ add_grouping_info_to_base_rels(PlannerInfo *root)
+ {
+ int i;
+
+ /* No grouping in the query? */
+ if (!root->parse->groupClause || root->group_pathkeys == NIL)
+ return;
+
+ /* TODO This is just for PoC. Relax the limitation later. */
+ if (root->parse->havingQual)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_grouped_var_infos(root);
+
+ /* Is no grouping is possible below the top-level join? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Process the individual base relations. */
+ for (i = 1; i < root->simple_rel_array_size; i++)
+ {
+ RelOptInfo *rel = root->simple_rel_array[i];
+
+ /*
+ * "other rels" will have their targets built later, by translation of
+ * the target of the parent rel - see set_append_rel_size. If we
+ * wanted to prepare the child rels here, we'd need another iteration
+ * of simple_rel_array_size.
+ */
+ if (rel != NULL && rel->reloptkind == RELOPT_BASEREL)
+ prepare_rel_for_grouping(root, rel);
+ }
+ }
+
+ /*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ *
+ * TODO Include aggregates from HAVING clause.
+ */
+ static void
+ create_grouped_var_infos(PlannerInfo *root)
+ {
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ /*
+ * TODO Check if processed_tlist contains the HAVING aggregates. If not,
+ * get them elsewhere.
+ */
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES);
+ if (tlist_exprs == NIL)
+ return;
+
+ /* tlist_exprs may also contain Vars, but we only need Aggrefs. */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ if (IsA(expr, Var))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct || aggref->aggfilter)
+ {
+ /*
+ * Partial aggregation is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ Expr *expr = (Expr *) lfirst(lc2);
+
+ gvi = castNode(GroupedVarInfo, expr);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ /* TODO Initialize gv_width. */
+ gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvid = list_length(root->grouped_var_list);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+ gvi->agg_partial = copyObject(aggref);
+ mark_partial_aggref(gvi->agg_partial, AGGSPLIT_INITIAL_SERIAL);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ {
+ Assert(aggref->aggstar);
+ gvi->gv_eval_at = NULL;
+ }
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+ }
+
+ /*
+ * Check if all the expressions of rel->reltarget can be used as grouping
+ * expressions and create target for grouped paths.
+ *
+ * If we succeed to create the grouping target, also replace rel->reltarget
+ * with a new one that has sortgrouprefs initialized -- this is necessary for
+ * create_agg_plan to match the grouping clauses against the input target
+ * expressions.
+ *
+ * rel_agg_attrs is a set attributes of the relation referenced by aggregate
+ * arguments. These can exist in the (plain) target without being grouping
+ * expressions.
+ *
+ * rel_agg_vars should be passed instead if rel is a join.
+ *
+ * TODO How about PHVs?
+ *
+ * TODO Make sure cost / width of both "result" and "plain" are correct.
+ */
+ PathTarget *
+ create_grouped_target(PlannerInfo *root, RelOptInfo *rel,
+ Relids rel_agg_attrs, List *rel_agg_vars)
+ {
+ PathTarget *result, *plain;
+ ListCell *lc;
+
+ /* The plan to be returned. */
+ result = create_empty_pathtarget();
+ /* The one to replace rel->reltarget. */
+ plain = create_empty_pathtarget();
+
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *texpr;
+ Index sortgroupref;
+ bool agg_arg_only = false;
+
+ texpr = (Expr *) lfirst(lc);
+
+ sortgroupref = get_expr_sortgroupref(root, texpr);
+ if (sortgroupref > 0)
+ {
+ /* It's o.k. to use the target expression for grouping. */
+ add_column_to_pathtarget(result, texpr, sortgroupref);
+
+ /*
+ * As for the plain target, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(plain, texpr, sortgroupref);
+
+ /* Process the next expression. */
+ continue;
+ }
+
+ /*
+ * It may still be o.k. if the expression is only contained in Aggref
+ * - then it's not expected in the grouped output.
+ *
+ * TODO Try to handle generic expression, not only Var. That might
+ * require us to create rel->reltarget of the grouping rel in
+ * parallel to that of the plain rel, and adding whole expressions
+ * instead of individual vars.
+ */
+ if (IsA(texpr, Var))
+ {
+ Var *arg_var = castNode(Var, texpr);
+
+ if (rel->relid > 0)
+ {
+ AttrNumber varattno;
+
+ /*
+ * For a single relation we only need to check attribute
+ * number.
+ *
+ * Apply the same offset that pull_varattnos() did.
+ */
+ varattno = arg_var->varattno - FirstLowInvalidHeapAttributeNumber;
+
+ if (bms_is_member(varattno, rel_agg_attrs))
+ agg_arg_only = true;
+ }
+ else
+ {
+ ListCell *lc2;
+
+ /* Join case. */
+ foreach(lc2, rel_agg_vars)
+ {
+ Var *var = castNode(Var, lfirst(lc2));
+
+ if (var->varno == arg_var->varno &&
+ var->varattno == arg_var->varattno)
+ {
+ agg_arg_only = true;
+ break;
+ }
+ }
+ }
+
+ if (agg_arg_only)
+ {
+ /*
+ * This expression is not suitable for grouping, but the
+ * aggregation input target ought to stay complete.
+ */
+ add_column_to_pathtarget(plain, texpr, 0);
+ }
+ }
+
+ /*
+ * A single mismatched expression makes the whole relation useless
+ * for grouping.
+ */
+ if (!agg_arg_only)
+ {
+ /*
+ * TODO This seems possible to happen multiple times per relation,
+ * so result might be worth freeing. Implement free_pathtarget()?
+ * Or mark the relation as inappropriate for grouping?
+ */
+ /* TODO Free both result and plain. */
+ return NULL;
+ }
+ }
+
+ if (list_length(result->exprs) == 0)
+ {
+ /* TODO free_pathtarget(result); free_pathtarget(plain) */
+ result = NULL;
+ }
+
+ /* Apply the adjusted input target as the replacement is complete now.q */
+ rel->reltarget = plain;
+
+ return result;
+ }
+
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c
new file mode 100644
index 5565736..058af2c
*** a/src/backend/optimizer/plan/planagg.c
--- b/src/backend/optimizer/plan/planagg.c
*************** preprocess_minmax_aggregates(PlannerInfo
*** 223,229 ****
create_minmaxagg_path(root, grouped_rel,
create_pathtarget(root, tlist),
aggs_list,
! (List *) parse->havingQual));
}
/*
--- 223,229 ----
create_minmaxagg_path(root, grouped_rel,
create_pathtarget(root, tlist),
aggs_list,
! (List *) parse->havingQual), false);
}
/*
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
new file mode 100644
index ef0de3f..f70b445
*** a/src/backend/optimizer/plan/planmain.c
--- b/src/backend/optimizer/plan/planmain.c
*************** query_planner(PlannerInfo *root, List *t
*** 83,89 ****
add_path(final_rel, (Path *)
create_result_path(root, final_rel,
final_rel->reltarget,
! (List *) parse->jointree->quals));
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(final_rel);
--- 83,89 ----
add_path(final_rel, (Path *)
create_result_path(root, final_rel,
final_rel->reltarget,
! (List *) parse->jointree->quals), false);
/* Select cheapest path (pretty easy in this case...) */
set_cheapest(final_rel);
*************** query_planner(PlannerInfo *root, List *t
*** 114,119 ****
--- 114,120 ----
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
*************** query_planner(PlannerInfo *root, List *t
*** 177,182 ****
--- 178,191 ----
(*qp_callback) (root, qp_extra);
/*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, Initialize GroupedPathInfo
+ * of base relations capable to do the grouping and setup
+ * root->grouped_var_list.
+ */
+ add_grouping_info_to_base_rels(root);
+
+ /*
* Examine any "placeholder" expressions generated during subquery pullup.
* Make sure that the Vars they need are marked as needed at the relevant
* join level. This must be done before join removal because it might
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
new file mode 100644
index 649a233..2c44f42
*** a/src/backend/optimizer/plan/planner.c
--- b/src/backend/optimizer/plan/planner.c
*************** static void standard_qp_callback(Planner
*** 130,138 ****
static double get_number_of_groups(PlannerInfo *root,
double path_rows,
grouping_sets_data *gd);
- static Size estimate_hashagg_tablesize(Path *path,
- const AggClauseCosts *agg_costs,
- double dNumGroups);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *target,
--- 130,135 ----
*************** inheritance_planner(PlannerInfo *root)
*** 1419,1425 ****
returningLists,
rowMarks,
NULL,
! SS_assign_special_param(root)));
}
/*--------------------
--- 1416,1422 ----
returningLists,
rowMarks,
NULL,
! SS_assign_special_param(root)), false);
}
/*--------------------
*************** grouping_planner(PlannerInfo *root, bool
*** 2040,2046 ****
}
/* And shove it into final_rel */
! add_path(final_rel, path);
}
/*
--- 2037,2043 ----
}
/* And shove it into final_rel */
! add_path(final_rel, path, false);
}
/*
*************** get_number_of_groups(PlannerInfo *root,
*** 3446,3485 ****
}
/*
- * estimate_hashagg_tablesize
- * estimate the number of bytes that a hash aggregate hashtable will
- * require based on the agg_costs, path width and dNumGroups.
- *
- * XXX this may be over-estimating the size now that hashagg knows to omit
- * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
- * grouping columns not in the hashed set are counted here even though hashagg
- * won't store them. Is this a problem?
- */
- static Size
- estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
- double dNumGroups)
- {
- Size hashentrysize;
-
- /* Estimate per-hash-entry space at tuple width... */
- hashentrysize = MAXALIGN(path->pathtarget->width) +
- MAXALIGN(SizeofMinimalTupleHeader);
-
- /* plus space for pass-by-ref transition values... */
- hashentrysize += agg_costs->transitionSpace;
- /* plus the per-hash-entry overhead */
- hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
-
- /*
- * Note that this disregards the effect of fill-factor and growth policy
- * of the hash-table. That's probably ok, given default the default
- * fill-factor is relatively high. It'd be hard to meaningfully factor in
- * "double-in-size" growth policies here.
- */
- return hashentrysize * dNumGroups;
- }
-
- /*
* create_grouping_paths
*
* Build a new upperrel containing Paths for grouping and/or aggregation.
--- 3443,3448 ----
*************** create_grouping_paths(PlannerInfo *root,
*** 3600,3606 ****
(List *) parse->havingQual);
}
! add_path(grouped_rel, path);
/* No need to consider any other alternatives. */
set_cheapest(grouped_rel);
--- 3563,3569 ----
(List *) parse->havingQual);
}
! add_path(grouped_rel, path, false);
/* No need to consider any other alternatives. */
set_cheapest(grouped_rel);
*************** create_grouping_paths(PlannerInfo *root,
*** 3777,3783 ****
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups));
else
add_partial_path(grouped_rel, (Path *)
create_group_path(root,
--- 3740,3747 ----
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups),
! false);
else
add_partial_path(grouped_rel, (Path *)
create_group_path(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 3786,3792 ****
partial_grouping_target,
parse->groupClause,
NIL,
! dNumPartialGroups));
}
}
}
--- 3750,3757 ----
partial_grouping_target,
parse->groupClause,
NIL,
! dNumPartialGroups),
! false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 3817,3823 ****
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups));
}
}
}
--- 3782,3789 ----
parse->groupClause,
NIL,
&agg_partial_costs,
! dNumPartialGroups),
! false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 3869,3875 ****
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups));
}
else if (parse->groupClause)
{
--- 3835,3841 ----
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups), false);
}
else if (parse->groupClause)
{
*************** create_grouping_paths(PlannerInfo *root,
*** 3884,3890 ****
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups));
}
else
{
--- 3850,3856 ----
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups), false);
}
else
{
*************** create_grouping_paths(PlannerInfo *root,
*** 3933,3939 ****
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups));
else
add_path(grouped_rel, (Path *)
create_group_path(root,
--- 3899,3905 ----
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups), false);
else
add_path(grouped_rel, (Path *)
create_group_path(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 3942,3948 ****
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups));
/*
* The point of using Gather Merge rather than Gather is that it
--- 3908,3914 ----
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups), false);
/*
* The point of using Gather Merge rather than Gather is that it
*************** create_grouping_paths(PlannerInfo *root,
*** 3995,4001 ****
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups));
else
add_path(grouped_rel, (Path *)
create_group_path(root,
--- 3961,3967 ----
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups), false);
else
add_path(grouped_rel, (Path *)
create_group_path(root,
*************** create_grouping_paths(PlannerInfo *root,
*** 4004,4010 ****
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups));
}
}
}
--- 3970,3976 ----
target,
parse->groupClause,
(List *) parse->havingQual,
! dNumGroups), false);
}
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 4049,4055 ****
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups));
}
}
--- 4015,4021 ----
parse->groupClause,
(List *) parse->havingQual,
agg_costs,
! dNumGroups), false);
}
}
*************** create_grouping_paths(PlannerInfo *root,
*** 4087,4095 ****
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups));
}
}
}
/* Give a helpful error if we failed to find any implementation */
--- 4053,4129 ----
parse->groupClause,
(List *) parse->havingQual,
&agg_final_costs,
! dNumGroups), false);
}
}
+
+ /*
+ * If input_rel has partially aggregated partial paths, gather them
+ * and perform the final aggregation.
+ *
+ * TODO Allow havingQual - currently not supported at base relation
+ * level.
+ */
+ if (input_rel->gpi != NULL &&
+ input_rel->gpi->partial_pathlist != NIL &&
+ !parse->havingQual)
+ {
+ Path *path = (Path *) linitial(input_rel->gpi->partial_pathlist);
+ double total_groups = path->rows * path->parallel_workers;
+
+ path = (Path *) create_gather_path(root,
+ input_rel,
+ path,
+ path->pathtarget,
+ NULL,
+ &total_groups);
+
+ /*
+ * The input path is partially aggregated and the final
+ * aggregation - if the path wins - will be done below. So we're
+ * done with it for now.
+ *
+ * The top-level grouped_rel needs to receive the path into
+ * regular pathlist, as opposed grouped_rel->gpi->pathlist.
+ */
+
+ add_path(input_rel, path, false);
+ }
+
+ /*
+ * If input_rel has partially aggregated paths, perform the final
+ * aggregation.
+ *
+ * TODO Allow havingQual - currently not supported at base relation
+ * level.
+ */
+ if (input_rel->gpi != NULL && input_rel->gpi->pathlist != NIL &&
+ !parse->havingQual)
+ {
+ Path *pre_agg = (Path *) linitial(input_rel->gpi->pathlist);
+
+ dNumGroups = get_number_of_groups(root, pre_agg->rows, gd);
+
+ MemSet(&agg_final_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, (Node *) target->exprs,
+ AGGSPLIT_FINAL_DESERIAL,
+ &agg_final_costs);
+ get_agg_clause_costs(root, parse->havingQual,
+ AGGSPLIT_FINAL_DESERIAL,
+ &agg_final_costs);
+
+ add_path(grouped_rel,
+ (Path *) create_agg_path(root, grouped_rel,
+ pre_agg,
+ target,
+ AGG_HASHED,
+ AGGSPLIT_FINAL_DESERIAL,
+ parse->groupClause,
+ (List *) parse->havingQual,
+ &agg_final_costs,
+ dNumGroups),
+ false);
+ }
}
/* Give a helpful error if we failed to find any implementation */
*************** consider_groupingsets_paths(PlannerInfo
*** 4289,4295 ****
strat,
new_rollups,
agg_costs,
! dNumGroups));
return;
}
--- 4323,4329 ----
strat,
new_rollups,
agg_costs,
! dNumGroups), false);
return;
}
*************** consider_groupingsets_paths(PlannerInfo
*** 4447,4453 ****
AGG_MIXED,
rollups,
agg_costs,
! dNumGroups));
}
}
--- 4481,4487 ----
AGG_MIXED,
rollups,
agg_costs,
! dNumGroups), false);
}
}
*************** consider_groupingsets_paths(PlannerInfo
*** 4464,4470 ****
AGG_SORTED,
gd->rollups,
agg_costs,
! dNumGroups));
}
/*
--- 4498,4504 ----
AGG_SORTED,
gd->rollups,
agg_costs,
! dNumGroups), false);
}
/*
*************** create_one_window_path(PlannerInfo *root
*** 4649,4655 ****
window_pathkeys);
}
! add_path(window_rel, path);
}
/*
--- 4683,4689 ----
window_pathkeys);
}
! add_path(window_rel, path, false);
}
/*
*************** create_distinct_paths(PlannerInfo *root,
*** 4755,4761 ****
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows));
}
}
--- 4789,4795 ----
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows), false);
}
}
*************** create_distinct_paths(PlannerInfo *root,
*** 4782,4788 ****
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows));
}
/*
--- 4816,4822 ----
create_upper_unique_path(root, distinct_rel,
path,
list_length(root->distinct_pathkeys),
! numDistinctRows), false);
}
/*
*************** create_distinct_paths(PlannerInfo *root,
*** 4829,4835 ****
parse->distinctClause,
NIL,
NULL,
! numDistinctRows));
}
/* Give a helpful error if we failed to find any implementation */
--- 4863,4869 ----
parse->distinctClause,
NIL,
NULL,
! numDistinctRows), false);
}
/* Give a helpful error if we failed to find any implementation */
*************** create_ordered_paths(PlannerInfo *root,
*** 4927,4933 ****
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path);
}
}
--- 4961,4967 ----
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path, false);
}
}
*************** create_ordered_paths(PlannerInfo *root,
*** 4977,4983 ****
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path);
}
}
--- 5011,5017 ----
path = apply_projection_to_path(root, ordered_rel,
path, target);
! add_path(ordered_rel, path, false);
}
}
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
new file mode 100644
index 1278371..548b372
*** a/src/backend/optimizer/plan/setrefs.c
--- b/src/backend/optimizer/plan/setrefs.c
*************** typedef struct
*** 40,45 ****
--- 40,46 ----
List *tlist; /* underlying target list */
int num_vars; /* number of plain Var tlist entries */
bool has_ph_vars; /* are there PlaceHolderVar entries? */
+ bool has_grp_vars; /* are there GroupedVar entries? */
bool has_non_vars; /* are there other entries? */
tlist_vinfo vars[FLEXIBLE_ARRAY_MEMBER]; /* has num_vars entries */
} indexed_tlist;
*************** set_upper_references(PlannerInfo *root,
*** 1725,1733 ****
--- 1726,1777 ----
indexed_tlist *subplan_itlist;
List *output_targetlist;
ListCell *l;
+ List *sub_tlist_save = NIL;
+
+ if (root->grouped_var_list != NIL)
+ {
+ if (IsA(plan, Agg))
+ {
+ Agg *agg = (Agg *) plan;
+
+ if (agg->aggsplit == AGGSPLIT_FINAL_DESERIAL)
+ {
+ /*
+ * convert_combining_aggrefs could have replaced some vars
+ * with Aggref expressions representing the partial
+ * aggregation. We need to restore the same Aggrefs in the
+ * subplan targetlist, but this would break the subplan if
+ * it's something else than the partial aggregation (i.e. the
+ * partial aggregation takes place lower in the plan tree). So
+ * we'll eventually need to restore the original list.
+ */
+ if (!IsA(subplan, Agg))
+ sub_tlist_save = subplan->targetlist;
+ #ifdef USE_ASSERT_CHECKING
+ else
+ Assert(((Agg *) subplan)->aggsplit == AGGSPLIT_INITIAL_SERIAL);
+ #endif /* USE_ASSERT_CHECKING */
+
+ /*
+ * Restore the aggregate expressions that we might have
+ * removed when planning for aggregation at base relation
+ * level.
+ */
+ subplan->targetlist =
+ restore_grouping_expressions(root, subplan->targetlist);
+ }
+ }
+ }
subplan_itlist = build_tlist_index(subplan->targetlist);
+ /*
+ * The replacement of GroupVars by Aggrefs was only needed for the index
+ * build.
+ */
+ if (sub_tlist_save != NIL)
+ subplan->targetlist = sub_tlist_save;
+
output_targetlist = NIL;
foreach(l, plan->targetlist)
{
*************** build_tlist_index(List *tlist)
*** 1937,1942 ****
--- 1981,1987 ----
itlist->tlist = tlist;
itlist->has_ph_vars = false;
+ itlist->has_grp_vars = false;
itlist->has_non_vars = false;
/* Find the Vars and fill in the index array */
*************** build_tlist_index(List *tlist)
*** 1956,1961 ****
--- 2001,2008 ----
}
else if (tle->expr && IsA(tle->expr, PlaceHolderVar))
itlist->has_ph_vars = true;
+ else if (tle->expr && IsA(tle->expr, GroupedVar))
+ itlist->has_grp_vars = true;
else
itlist->has_non_vars = true;
}
*************** fix_join_expr_mutator(Node *node, fix_jo
*** 2233,2238 ****
--- 2280,2310 ----
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, GroupedVar))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ /* See if the GroupedVar has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_grp_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) gvar,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_grp_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) gvar,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+
+ /* No referent found for GroupedVar */
+ elog(ERROR, "grouped variable not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
*************** fix_upper_expr_mutator(Node *node, fix_u
*** 2389,2395 ****
/* If no match, just fall through to process it normally */
}
/* Try matching more complex expressions too, if tlist has any */
! if (context->subplan_itlist->has_non_vars)
{
newvar = search_indexed_tlist_for_non_var((Expr *) node,
context->subplan_itlist,
--- 2461,2468 ----
/* If no match, just fall through to process it normally */
}
/* Try matching more complex expressions too, if tlist has any */
! if (context->subplan_itlist->has_grp_vars ||
! context->subplan_itlist->has_non_vars)
{
newvar = search_indexed_tlist_for_non_var((Expr *) node,
context->subplan_itlist,
diff --git a/src/backend/optimizer/prep/prepunion.c b/src/backend/optimizer/prep/prepunion.c
new file mode 100644
index a1be858..87a74be
*** a/src/backend/optimizer/prep/prepunion.c
--- b/src/backend/optimizer/prep/prepunion.c
*************** plan_set_operations(PlannerInfo *root)
*** 207,213 ****
root->processed_tlist = top_tlist;
/* Add only the final path to the SETOP upperrel. */
! add_path(setop_rel, path);
/* Let extensions possibly add some more paths */
if (create_upper_paths_hook)
--- 207,213 ----
root->processed_tlist = top_tlist;
/* Add only the final path to the SETOP upperrel. */
! add_path(setop_rel, path, false);
/* Let extensions possibly add some more paths */
if (create_upper_paths_hook)
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
new file mode 100644
index 2d5caae..98e5d6a
*** a/src/backend/optimizer/util/pathnode.c
--- b/src/backend/optimizer/util/pathnode.c
***************
*** 24,29 ****
--- 24,31 ----
#include "optimizer/paths.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+ /* TODO Remove this if get_grouping_expressions ends up in another module. */
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "parser/parsetree.h"
#include "utils/lsyscache.h"
*************** set_cheapest(RelOptInfo *parent_rel)
*** 409,416 ****
* Returns nothing, but modifies parent_rel->pathlist.
*/
void
! add_path(RelOptInfo *parent_rel, Path *new_path)
{
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
List *new_path_pathkeys;
--- 411,419 ----
* Returns nothing, but modifies parent_rel->pathlist.
*/
void
! add_path(RelOptInfo *parent_rel, Path *new_path, bool grouped)
{
+ List *pathlist;
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
List *new_path_pathkeys;
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 427,432 ****
--- 430,443 ----
/* Pretend parameterized paths have no pathkeys, per comment above */
new_path_pathkeys = new_path->param_info ? NIL : new_path->pathkeys;
+ if (!grouped)
+ pathlist = parent_rel->pathlist;
+ else
+ {
+ Assert(parent_rel->gpi != NULL);
+ pathlist = parent_rel->gpi->pathlist;
+ }
+
/*
* Loop to check proposed new path against old paths. Note it is possible
* for more than one old path to be tossed out because new_path dominates
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 436,442 ****
* list cell.
*/
p1_prev = NULL;
! for (p1 = list_head(parent_rel->pathlist); p1 != NULL; p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
bool remove_old = false; /* unless new proves superior */
--- 447,453 ----
* list cell.
*/
p1_prev = NULL;
! for (p1 = list_head(pathlist); p1 != NULL; p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
bool remove_old = false; /* unless new proves superior */
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 582,589 ****
*/
if (remove_old)
{
! parent_rel->pathlist = list_delete_cell(parent_rel->pathlist,
! p1, p1_prev);
/*
* Delete the data pointed-to by the deleted cell, if possible
--- 593,599 ----
*/
if (remove_old)
{
! pathlist = list_delete_cell(pathlist, p1, p1_prev);
/*
* Delete the data pointed-to by the deleted cell, if possible
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 614,622 ****
{
/* Accept the new path: insert it at proper place in pathlist */
if (insert_after)
! lappend_cell(parent_rel->pathlist, insert_after, new_path);
else
! parent_rel->pathlist = lcons(new_path, parent_rel->pathlist);
}
else
{
--- 624,637 ----
{
/* Accept the new path: insert it at proper place in pathlist */
if (insert_after)
! lappend_cell(pathlist, insert_after, new_path);
else
! pathlist = lcons(new_path, pathlist);
!
! if (!grouped)
! parent_rel->pathlist = pathlist;
! else
! parent_rel->gpi->pathlist = pathlist;
}
else
{
*************** add_path(RelOptInfo *parent_rel, Path *n
*** 646,653 ****
bool
add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer)
{
List *new_path_pathkeys;
bool consider_startup;
ListCell *p1;
--- 661,669 ----
bool
add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer, bool grouped)
{
+ List *pathlist;
List *new_path_pathkeys;
bool consider_startup;
ListCell *p1;
*************** add_path_precheck(RelOptInfo *parent_rel
*** 656,664 ****
new_path_pathkeys = required_outer ? NIL : pathkeys;
/* Decide whether new path's startup cost is interesting */
! consider_startup = required_outer ? parent_rel->consider_param_startup : parent_rel->consider_startup;
! foreach(p1, parent_rel->pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
--- 672,689 ----
new_path_pathkeys = required_outer ? NIL : pathkeys;
/* Decide whether new path's startup cost is interesting */
! consider_startup = required_outer ? parent_rel->consider_param_startup :
! parent_rel->consider_startup;
! if (!grouped)
! pathlist = parent_rel->pathlist;
! else
! {
! Assert(parent_rel->gpi != NULL);
! pathlist = parent_rel->gpi->pathlist;
! }
!
! foreach(p1, pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
*************** add_path_precheck(RelOptInfo *parent_rel
*** 749,771 ****
* referenced by partial BitmapHeapPaths.
*/
void
! add_partial_path(RelOptInfo *parent_rel, Path *new_path)
{
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
ListCell *p1;
ListCell *p1_prev;
ListCell *p1_next;
/* Check for query cancel. */
CHECK_FOR_INTERRUPTS();
/*
* As in add_path, throw out any paths which are dominated by the new
* path, but throw out the new path if some existing path dominates it.
*/
p1_prev = NULL;
! for (p1 = list_head(parent_rel->partial_pathlist); p1 != NULL;
p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
--- 774,805 ----
* referenced by partial BitmapHeapPaths.
*/
void
! add_partial_path(RelOptInfo *parent_rel, Path *new_path, bool grouped)
{
bool accept_new = true; /* unless we find a superior old path */
ListCell *insert_after = NULL; /* where to insert new item */
ListCell *p1;
ListCell *p1_prev;
ListCell *p1_next;
+ List *pathlist;
/* Check for query cancel. */
CHECK_FOR_INTERRUPTS();
+ if (!grouped)
+ pathlist = parent_rel->partial_pathlist;
+ else
+ {
+ Assert(parent_rel->gpi != NULL);
+ pathlist = parent_rel->gpi->partial_pathlist;
+ }
+
/*
* As in add_path, throw out any paths which are dominated by the new
* path, but throw out the new path if some existing path dominates it.
*/
p1_prev = NULL;
! for (p1 = list_head(pathlist); p1 != NULL;
p1 = p1_next)
{
Path *old_path = (Path *) lfirst(p1);
*************** add_partial_path(RelOptInfo *parent_rel,
*** 819,830 ****
}
/*
! * Remove current element from partial_pathlist if dominated by new.
*/
if (remove_old)
{
! parent_rel->partial_pathlist =
! list_delete_cell(parent_rel->partial_pathlist, p1, p1_prev);
pfree(old_path);
/* p1_prev does not advance */
}
--- 853,863 ----
}
/*
! * Remove current element from pathlist if dominated by new.
*/
if (remove_old)
{
! pathlist = list_delete_cell(pathlist, p1, p1_prev);
pfree(old_path);
/* p1_prev does not advance */
}
*************** add_partial_path(RelOptInfo *parent_rel,
*** 839,845 ****
/*
* If we found an old path that dominates new_path, we can quit
! * scanning the partial_pathlist; we will not add new_path, and we
* assume new_path cannot dominate any later path.
*/
if (!accept_new)
--- 872,878 ----
/*
* If we found an old path that dominates new_path, we can quit
! * scanning the pathlist; we will not add new_path, and we
* assume new_path cannot dominate any later path.
*/
if (!accept_new)
*************** add_partial_path(RelOptInfo *parent_rel,
*** 850,859 ****
{
/* Accept the new path: insert it at proper place */
if (insert_after)
! lappend_cell(parent_rel->partial_pathlist, insert_after, new_path);
else
! parent_rel->partial_pathlist =
! lcons(new_path, parent_rel->partial_pathlist);
}
else
{
--- 883,896 ----
{
/* Accept the new path: insert it at proper place */
if (insert_after)
! lappend_cell(pathlist, insert_after, new_path);
else
! pathlist = lcons(new_path, pathlist);
!
! if (!grouped)
! parent_rel->partial_pathlist = pathlist;
! else
! parent_rel->gpi->partial_pathlist = pathlist;
}
else
{
*************** add_partial_path(RelOptInfo *parent_rel,
*** 874,882 ****
*/
bool
add_partial_path_precheck(RelOptInfo *parent_rel, Cost total_cost,
! List *pathkeys)
{
ListCell *p1;
/*
* Our goal here is twofold. First, we want to find out whether this path
--- 911,928 ----
*/
bool
add_partial_path_precheck(RelOptInfo *parent_rel, Cost total_cost,
! List *pathkeys, bool grouped)
{
ListCell *p1;
+ List *pathlist;
+
+ if (!grouped)
+ pathlist = parent_rel->partial_pathlist;
+ else
+ {
+ Assert(parent_rel->gpi != NULL);
+ pathlist = parent_rel->gpi->partial_pathlist;
+ }
/*
* Our goal here is twofold. First, we want to find out whether this path
*************** add_partial_path_precheck(RelOptInfo *pa
*** 886,895 ****
* final cost computations. If so, we definitely want to consider it.
*
* Unlike add_path(), we always compare pathkeys here. This is because we
! * expect partial_pathlist to be very short, and getting a definitive
! * answer at this stage avoids the need to call add_path_precheck.
*/
! foreach(p1, parent_rel->partial_pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
--- 932,942 ----
* final cost computations. If so, we definitely want to consider it.
*
* Unlike add_path(), we always compare pathkeys here. This is because we
! * expect partial_pathlist / grouped_pathlist to be very short, and
! * getting a definitive answer at this stage avoids the need to call
! * add_path_precheck.
*/
! foreach(p1, pathlist)
{
Path *old_path = (Path *) lfirst(p1);
PathKeysComparison keyscmp;
*************** add_partial_path_precheck(RelOptInfo *pa
*** 918,924 ****
* completion.
*/
if (!add_path_precheck(parent_rel, total_cost, total_cost, pathkeys,
! NULL))
return false;
return true;
--- 965,971 ----
* completion.
*/
if (!add_path_precheck(parent_rel, total_cost, total_cost, pathkeys,
! NULL, grouped))
return false;
return true;
*************** calc_non_nestloop_required_outer(Path *o
*** 2055,2060 ****
--- 2102,2108 ----
* 'restrict_clauses' are the RestrictInfo nodes to apply at the join
* 'pathkeys' are the path keys of the new join path
* 'required_outer' is the set of required outer rels
+ * 'target' can be passed to override that of joinrel.
*
* Returns the resulting path node.
*/
*************** create_nestloop_path(PlannerInfo *root,
*** 2068,2074 ****
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer)
{
NestPath *pathnode = makeNode(NestPath);
Relids inner_req_outer = PATH_REQ_OUTER(inner_path);
--- 2116,2123 ----
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer,
! PathTarget *target)
{
NestPath *pathnode = makeNode(NestPath);
Relids inner_req_outer = PATH_REQ_OUTER(inner_path);
*************** create_nestloop_path(PlannerInfo *root,
*** 2101,2107 ****
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
! pathnode->path.pathtarget = joinrel->reltarget;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2150,2156 ----
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
! pathnode->path.pathtarget = target == NULL ? joinrel->reltarget : target;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_mergejoin_path(PlannerInfo *root,
*** 2159,2171 ****
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys)
{
MergePath *pathnode = makeNode(MergePath);
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = joinrel->reltarget;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2208,2222 ----
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys,
! PathTarget *target)
{
MergePath *pathnode = makeNode(MergePath);
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = target == NULL ? joinrel->reltarget :
! target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_mergejoin_path(PlannerInfo *root,
*** 2210,2215 ****
--- 2261,2267 ----
* 'required_outer' is the set of required outer rels
* 'hashclauses' are the RestrictInfo nodes to use as hash clauses
* (this should be a subset of the restrict_clauses list)
+ * 'target' can be passed to override that of joinrel.
*/
HashPath *
create_hashjoin_path(PlannerInfo *root,
*************** create_hashjoin_path(PlannerInfo *root,
*** 2221,2233 ****
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses)
{
HashPath *pathnode = makeNode(HashPath);
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = joinrel->reltarget;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2273,2287 ----
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses,
! PathTarget *target)
{
HashPath *pathnode = makeNode(HashPath);
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = target == NULL ? joinrel->reltarget :
! target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_agg_path(PlannerInfo *root,
*** 2713,2718 ****
--- 2767,2942 ----
}
/*
+ * Apply partial AGG_SORTED aggregation path to subpath if it's suitably
+ * sorted.
+ *
+ * first_call indicates whether the function is being called first time for
+ * given index --- since the target should not change, we can skip the check
+ * of sorting during subsequent calls.
+ *
+ * group_clauses, group_exprs and agg_exprs are pointers to lists we populate
+ * when called first time for particular index, and that user passes for
+ * subsequent calls.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+ AggPath *
+ create_partial_agg_sorted_path(PlannerInfo *root, Path *subpath,
+ bool first_call,
+ List **group_clauses, List **group_exprs,
+ List **agg_exprs, double input_rows)
+ {
+ RelOptInfo *rel;
+ AggClauseCosts agg_costs;
+ double dNumGroups;
+ AggPath *result = NULL;
+
+ rel = subpath->parent;
+ Assert(rel->gpi != NULL);
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ if (first_call)
+ {
+ ListCell *lc1;
+ List *key_subset = NIL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+ }
+
+ if (first_call)
+ get_grouping_expressions(root, rel->gpi->target, group_clauses,
+ group_exprs, agg_exprs);
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ Assert(*agg_exprs != NIL);
+ get_agg_clause_costs(root, (Node *) *agg_exprs, AGGSPLIT_INITIAL_SERIAL,
+ &agg_costs);
+
+ Assert(*group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, *group_exprs, input_rows, NULL);
+
+ /* TODO HAVING qual. */
+ Assert(*group_clauses != NIL);
+ result = create_agg_path(root, rel, subpath, rel->gpi->target, AGG_SORTED,
+ AGGSPLIT_INITIAL_SERIAL, *group_clauses, NIL,
+ &agg_costs, dNumGroups);
+
+ return result;
+ }
+
+ /*
+ * Appy partial AGG_HASHED aggregation to subpath.
+ *
+ * Arguments have the same meaning as those of create_agg_sorted_path.
+ *
+ */
+ AggPath *
+ create_partial_agg_hashed_path(PlannerInfo *root, Path *subpath,
+ bool first_call,
+ List **group_clauses, List **group_exprs,
+ List **agg_exprs, double input_rows)
+ {
+ RelOptInfo *rel;
+ bool can_hash;
+ AggClauseCosts agg_costs;
+ double dNumGroups;
+ Size hashaggtablesize;
+ Query *parse = root->parse;
+ AggPath *result = NULL;
+
+ rel = subpath->parent;
+ Assert(rel->gpi != NULL);
+
+ if (first_call)
+ {
+ /*
+ * Find one grouping clause per grouping column.
+ *
+ * All that create_agg_plan eventually needs of the clause is
+ * tleSortGroupRef, so we don't have to care that the clause
+ * expression might differ from texpr, in case texpr was derived from
+ * EC.
+ */
+ get_grouping_expressions(root, rel->gpi->target, group_clauses,
+ group_exprs, agg_exprs);
+ }
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ Assert(*agg_exprs != NIL);
+ get_agg_clause_costs(root, (Node *) *agg_exprs, AGGSPLIT_INITIAL_SERIAL,
+ &agg_costs);
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(*group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, *group_exprs, input_rows,
+ NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * Create the partial aggregation path.
+ */
+ Assert(*group_clauses != NIL);
+
+ result = create_agg_path(root, rel, subpath,
+ rel->gpi->target,
+ AGG_HASHED,
+ AGGSPLIT_INITIAL_SERIAL,
+ *group_clauses, NIL,
+ &agg_costs,
+ dNumGroups);
+
+ /*
+ * The agg path should require no fewer parameters than the plain
+ * one.
+ */
+ result->path.param_info = subpath->param_info;
+ }
+ }
+
+ return result;
+ }
+
+ /*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
*
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
new file mode 100644
index 342d884..3cd7093
*** a/src/backend/optimizer/util/relnode.c
--- b/src/backend/optimizer/util/relnode.c
***************
*** 25,30 ****
--- 25,31 ----
#include "optimizer/plancat.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+ #include "optimizer/var.h"
#include "utils/hsearch.h"
*************** typedef struct JoinHashEntry
*** 35,41 ****
} JoinHashEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *input_rel);
static List *build_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outer_rel,
--- 36,42 ----
} JoinHashEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *input_rel, bool grouped);
static List *build_joinrel_restrictlist(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outer_rel,
*************** build_simple_rel(PlannerInfo *root, int
*** 120,125 ****
--- 121,127 ----
rel->cheapest_parameterized_paths = NIL;
rel->direct_lateral_relids = NULL;
rel->lateral_relids = NULL;
+ rel->gpi = NULL;
rel->relid = relid;
rel->rtekind = rte->rtekind;
/* min_attr, max_attr, attr_needed, attr_widths are set below */
*************** build_join_rel(PlannerInfo *root,
*** 497,502 ****
--- 499,505 ----
inner_rel->direct_lateral_relids);
joinrel->lateral_relids = min_join_parameterization(root, joinrel->relids,
outer_rel, inner_rel);
+ joinrel->gpi = NULL;
joinrel->relid = 0; /* indicates not a baserel */
joinrel->rtekind = RTE_JOIN;
joinrel->min_attr = 0;
*************** build_join_rel(PlannerInfo *root,
*** 539,548 ****
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
! build_joinrel_tlist(root, joinrel, outer_rel);
! build_joinrel_tlist(root, joinrel, inner_rel);
add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
* sets of any PlaceHolderVars computed here to direct_lateral_relids, so
--- 542,558 ----
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
! build_joinrel_tlist(root, joinrel, outer_rel, false);
! build_joinrel_tlist(root, joinrel, inner_rel, false);
add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ /* Try to build grouped target. */
+ /*
+ * TODO Consider if placeholders make sense here. If not, also make the
+ * related code below conditional.
+ */
+ prepare_rel_for_grouping(root, joinrel);
+
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
* sets of any PlaceHolderVars computed here to direct_lateral_relids, so
*************** min_join_parameterization(PlannerInfo *r
*** 670,686 ****
*/
static void
build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *input_rel)
{
Relids relids = joinrel->relids;
ListCell *vars;
! foreach(vars, input_rel->reltarget->exprs)
{
Var *var = (Var *) lfirst(vars);
RelOptInfo *baserel;
int ndx;
/*
* Ignore PlaceHolderVars in the input tlists; we'll make our own
* decisions about whether to copy them.
--- 680,722 ----
*/
static void
build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
! RelOptInfo *input_rel, bool grouped)
{
Relids relids = joinrel->relids;
+ PathTarget *input_target, *result;
ListCell *vars;
+ int i = -1;
! if (!grouped)
! {
! input_target = input_rel->reltarget;
! result = joinrel->reltarget;
! }
! else
! {
! if (input_rel->gpi != NULL)
! {
! input_target = input_rel->gpi->target;
! Assert(input_target != NULL);
! }
! else
! input_target = input_rel->reltarget;
!
! /* Caller should have initialized this. */
! Assert(joinrel->gpi != NULL);
!
! /* Default to the plain target. */
! result = joinrel->gpi->target;
! }
!
! foreach(vars, input_target->exprs)
{
Var *var = (Var *) lfirst(vars);
RelOptInfo *baserel;
int ndx;
+ i++;
+
/*
* Ignore PlaceHolderVars in the input tlists; we'll make our own
* decisions about whether to copy them.
*************** build_joinrel_tlist(PlannerInfo *root, R
*** 704,713 ****
ndx = var->varattno - baserel->min_attr;
if (bms_nonempty_difference(baserel->attr_needed[ndx], relids))
{
/* Yup, add it to the output */
! joinrel->reltarget->exprs = lappend(joinrel->reltarget->exprs, var);
/* Vars have cost zero, so no need to adjust reltarget->cost */
! joinrel->reltarget->width += baserel->attr_widths[ndx];
}
}
}
--- 740,763 ----
ndx = var->varattno - baserel->min_attr;
if (bms_nonempty_difference(baserel->attr_needed[ndx], relids))
{
+ Index sortgroupref = 0;
+
/* Yup, add it to the output */
! if (input_target->sortgrouprefs)
! sortgroupref = input_target->sortgrouprefs[i];
!
! /*
! * Even if not used for grouping in the input path (the input path
! * is not necessarily grouped), it might be useful for grouping
! * higher in the join tree.
! */
! if (sortgroupref == 0)
! sortgroupref = get_expr_sortgroupref(root, (Expr *) var);
!
! add_column_to_pathtarget(result, (Expr *) var, sortgroupref);
!
/* Vars have cost zero, so no need to adjust reltarget->cost */
! result->width += baserel->attr_widths[ndx];
}
}
}
*************** get_appendrel_parampathinfo(RelOptInfo *
*** 1359,1361 ****
--- 1409,1560 ----
return ppi;
}
+
+ /*
+ * If the relation can produce grouped paths, create GroupedPathInfo for it
+ * and create target for the grouped paths.
+ */
+ void
+ prepare_rel_for_grouping(PlannerInfo *root, RelOptInfo *rel)
+ {
+ List *rel_aggregates;
+ Relids rel_agg_attrs = NULL;
+ List *rel_agg_vars = NIL;
+ bool found_higher;
+ ListCell *lc;
+ PathTarget *target_grouped;
+
+ if (rel->relid > 0)
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return;
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the aggregate
+ * would receive different input at the base rel level.
+ *
+ * TODO For RELOPT_JOINREL, do not return if all the joins that can set
+ * any entry of the grouped target (do we need to postpone this check
+ * until the grouped target is available, and should create_grouped_target
+ * take care?) of this rel to NULL are provably below rel. (It's ok if rel
+ * is one of these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return;
+
+ /*
+ * Check if some aggregates can be evaluated in this relation's target,
+ * and collect all vars referenced by these aggregates.
+ */
+ rel_aggregates = NIL;
+ found_higher = false;
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = castNode(GroupedVarInfo, lfirst(lc));
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which typically means
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ Aggref *aggref = castNode(Aggref, gvi->gvexpr);
+
+ /*
+ * Accept the aggregate.
+ *
+ * GroupedVarInfo is more convenient for the next processing than
+ * Aggref, see add_aggregates_to_grouped_target.
+ */
+ rel_aggregates = lappend(rel_aggregates, gvi);
+
+ if (rel->relid > 0)
+ {
+ /*
+ * Simple relation. Collect attributes referenced by the
+ * aggregate arguments.
+ */
+ pull_varattnos((Node *) aggref, rel->relid, &rel_agg_attrs);
+ }
+ else
+ {
+ List *agg_vars;
+
+ /*
+ * Join. Collect vars referenced by the aggregate
+ * arguments.
+ */
+ /*
+ * TODO Can any argument contain PHVs? And if so, does it matter?
+ * Consider PVC_INCLUDE_PLACEHOLDERS | PVC_RECURSE_PLACEHOLDERS.
+ */
+ agg_vars = pull_var_clause((Node *) aggref,
+ PVC_RECURSE_AGGREGATES);
+ rel_agg_vars = list_concat(rel_agg_vars, agg_vars);
+ }
+ }
+ else if (bms_overlap(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * Remember that there is at least one aggregate that needs more
+ * than this rel.
+ */
+ found_higher = true;
+ }
+ }
+
+ /*
+ * Grouping makes little sense w/o aggregate function.
+ */
+ if (rel_aggregates == NIL)
+ {
+ bms_free(rel_agg_attrs);
+ return;
+ }
+
+ if (found_higher)
+ {
+ /*
+ * If some aggregate(s) need only this rel but some other need
+ * multiple relations including the the current one, grouping of the
+ * current rel could steal some input variables from the "higher
+ * aggregate" (besides decreasing the number of input rows).
+ */
+ list_free(rel_aggregates);
+ bms_free(rel_agg_attrs);
+ return;
+ }
+
+ /*
+ * If rel->reltarget can be used for aggregation, mark the relation as
+ * capable of grouping.
+ */
+ Assert(rel->gpi == NULL);
+ target_grouped = create_grouped_target(root, rel, rel_agg_attrs,
+ rel_agg_vars);
+ if (target_grouped != NULL)
+ {
+ GroupedPathInfo *gpi;
+
+ gpi = makeNode(GroupedPathInfo);
+ gpi->target = copy_pathtarget(target_grouped);
+ gpi->pathlist = NIL;
+ gpi->partial_pathlist = NIL;
+ rel->gpi = gpi;
+
+ /*
+ * Add aggregates (in the form of GroupedVar) to the target.
+ */
+ add_aggregates_to_target(root, gpi->target, rel_aggregates, rel);
+ }
+ }
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
new file mode 100644
index 0952385..dd962b7
*** a/src/backend/optimizer/util/tlist.c
--- b/src/backend/optimizer/util/tlist.c
*************** get_sortgrouplist_exprs(List *sgClauses,
*** 408,413 ****
--- 408,487 ----
return result;
}
+ /*
+ * get_sortgrouplist_clauses
+ *
+ * Given a "grouped target" (i.e. target where each non-GroupedVar
+ * element must have sortgroupref set), build a list of the referencing
+ * SortGroupClauses, a list of the corresponding grouping expressions and
+ * a list of aggregate expressions.
+ */
+ /* Refine the function name. */
+ void
+ get_grouping_expressions(PlannerInfo *root, PathTarget *target,
+ List **grouping_clauses, List **grouping_exprs,
+ List **agg_exprs)
+ {
+ ListCell *l;
+ int i = 0;
+
+ foreach(l, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(l);
+
+ /* The target should contain at least one grouping column. */
+ Assert(target->sortgrouprefs != NULL);
+
+ if (IsA(texpr, GroupedVar))
+ {
+ /*
+ * texpr should represent the first aggregate in the targetlist.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides aggregates, the target should contain no expressions w/o
+ * sortgroupref. Plain relation being joined to grouped can have
+ * sortgroupref equal to zero for expressions contained neither in
+ * grouping expression nor in aggregate arguments, but if the target
+ * contains such an expression, it shouldn't be used for aggregation
+ * --- see can_aggregate field of GroupedPathInfo.
+ */
+ Assert(sortgroupref > 0);
+
+ cl = get_sortgroupref_clause(sortgroupref, root->parse->groupClause);
+ *grouping_clauses = list_append_unique(*grouping_clauses, cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ *grouping_exprs = list_append_unique(*grouping_exprs, texpr);
+ }
+
+ /* Now collect the aggregates. */
+ while (l != NULL)
+ {
+ GroupedVar *gvar = castNode(GroupedVar, lfirst(l));
+
+ /* Currently, GroupedVarInfo can only represent aggregate. */
+ Assert(gvar->agg_partial != NULL);
+ *agg_exprs = lappend(*agg_exprs, gvar->agg_partial);
+ l = lnext(l);
+ }
+ }
+
/*****************************************************************************
* Functions to extract data from a list of SortGroupClauses
*************** apply_pathtarget_labeling_to_tlist(List
*** 783,788 ****
--- 857,1081 ----
}
/*
+ * Replace each "grouped var" in the source targetlist with the original
+ * expression.
+ *
+ * TODO Think of more suitable name. Although "grouped var" may substitute for
+ * grouping expressions in the future, currently Aggref is the only outcome of
+ * the replacement. undo_grouped_var_substitutions?
+ */
+ List *
+ restore_grouping_expressions(PlannerInfo *root, List *src)
+ {
+ List *result = NIL;
+ ListCell *l;
+
+ foreach(l, src)
+ {
+ TargetEntry *te, *te_new;
+ Aggref *expr_new = NULL;
+
+ te = castNode(TargetEntry, lfirst(l));
+
+ if (IsA(te->expr, GroupedVar))
+ {
+ GroupedVar *gvar;
+
+ gvar = castNode(GroupedVar, te->expr);
+ expr_new = gvar->agg_partial;
+ }
+
+ if (expr_new != NULL)
+ {
+ te_new = flatCopyTargetEntry(te);
+ te_new->expr = (Expr *) expr_new;
+ }
+ else
+ te_new = te;
+ result = lappend(result, te_new);
+ }
+
+ return result;
+ }
+
+ /*
+ * For each aggregate add GroupedVar to target if "vars" is true, or the
+ * Aggref (marked as partial) if "vars" is false.
+ *
+ * If caller passes the aggregates, he must do so in the form of
+ * GroupedVarInfos so that we don't have to look for gvid. If NULL is passed,
+ * the function retrieves the suitable aggregates itself.
+ *
+ * List of the aggregates added is returned. This is only useful if the
+ * function had to retrieve the aggregates itself (i.e. NIL was passed for
+ * aggregates) -- caller is expected to do extra checks in that case (and to
+ * also free the list).
+ */
+ List *
+ add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *aggregates, RelOptInfo *rel)
+ {
+ ListCell *lc;
+ GroupedVarInfo *gvi;
+
+ if (aggregates == NIL)
+ {
+ /* Caller should pass the aggregates for base relation. */
+ Assert(rel->reloptkind != RELOPT_BASEREL);
+
+ /* Collect all aggregates that this rel can evaluate. */
+ foreach(lc, root->grouped_var_list)
+ {
+ gvi = castNode(GroupedVarInfo, lfirst(lc));
+
+ /*
+ * Overlap is not guarantee of correctness alone, but caller needs
+ * to do additional checks, so we're optimistic here.
+ *
+ * If gv_eval_at is NULL, the underlying Aggref should have
+ * aggstar set.
+ */
+ if (bms_overlap(gvi->gv_eval_at, rel->relids) ||
+ gvi->gv_eval_at == NULL)
+ aggregates = lappend(aggregates, gvi);
+ }
+
+ if (aggregates == NIL)
+ return NIL;
+ }
+
+ /* Create the vars and add them to the target. */
+ foreach(lc, aggregates)
+ {
+ GroupedVar *gvar;
+
+ gvi = castNode(GroupedVarInfo, lfirst(lc));
+ gvar = makeNode(GroupedVar);
+ gvar->gvid = gvi->gvid;
+ gvar->gvexpr = gvi->gvexpr;
+ gvar->agg_partial = gvi->agg_partial;
+ add_new_column_to_pathtarget(target, (Expr *) gvar);
+ }
+
+ return aggregates;
+ }
+
+ /*
+ * Return ressortgroupref of the target entry that is either equal to the
+ * expression or exists in the same equivalence class.
+ */
+ Index
+ get_expr_sortgroupref(PlannerInfo *root, Expr *expr)
+ {
+ ListCell *lc;
+ Index sortgroupref;
+
+ /*
+ * First, check if the query group clause contains exactly this
+ * expression.
+ */
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = castNode(TargetEntry, lfirst(lc));
+
+ if (equal(expr, te->expr) && te->ressortgroupref > 0)
+ return te->ressortgroupref;
+ }
+
+ /*
+ * If exactly this expression is not there, check if a grouping clause
+ * exists that belongs to the same equivalence class as the expression.
+ */
+ foreach(lc, root->group_pathkeys)
+ {
+ PathKey *pk = castNode(PathKey, lfirst(lc));
+ EquivalenceClass *ec = pk->pk_eclass;
+ ListCell *lm;
+ EquivalenceMember *em;
+ Expr *em_expr = NULL;
+ Query *query = root->parse;
+
+ /*
+ * Single-member EC cannot provide us with additional expression.
+ */
+ if (list_length(ec->ec_members) < 2)
+ continue;
+
+ /* We need equality anywhere in the join tree. */
+ if (ec->ec_below_outer_join)
+ continue;
+
+ /*
+ * TODO Reconsider this restriction. As the grouping expression is
+ * only evaluated at the relation level (and only the result will be
+ * propagated to the final targetlist), volatile function might be
+ * o.k. Need to think what volatile EC exactly means.
+ */
+ if (ec->ec_has_volatile)
+ continue;
+
+ foreach(lm, ec->ec_members)
+ {
+ em = (EquivalenceMember *) lfirst(lm);
+
+ /* The EC has !ec_below_outer_join. */
+ Assert(!em->em_nullable_relids);
+ if (equal(em->em_expr, expr))
+ {
+ em_expr = (Expr *) em->em_expr;
+ break;
+ }
+ }
+
+ if (em_expr == NULL)
+ /* Go for the next EC. */
+ continue;
+
+ /*
+ * Find the corresponding SortGroupClause, which provides us with
+ * sortgroupref. (It can belong to any EC member.)
+ */
+ sortgroupref = 0;
+ foreach(lm, ec->ec_members)
+ {
+ ListCell *lsg;
+
+ em = (EquivalenceMember *) lfirst(lm);
+ foreach(lsg, query->groupClause)
+ {
+ SortGroupClause *sgc;
+ Expr *expr;
+
+ sgc = (SortGroupClause *) lfirst(lsg);
+ expr = (Expr *) get_sortgroupclause_expr(sgc,
+ query->targetList);
+ if (equal(em->em_expr, expr))
+ {
+ Assert(sgc->tleSortGroupRef > 0);
+ sortgroupref = sgc->tleSortGroupRef;
+ break;
+ }
+ }
+
+ if (sortgroupref > 0)
+ break;
+ }
+
+ /*
+ * Since we searched in group_pathkeys, at least one EM of this EC
+ * should correspond to a SortGroupClause, otherwise the EC could
+ * not exist at all.
+ */
+ Assert(sortgroupref > 0);
+
+ return sortgroupref;
+ }
+
+ /* No EC found in group_pathkeys. */
+ return 0;
+ }
+
+ /*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
*
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
new file mode 100644
index cbde1ff..49f87ac
*** a/src/backend/utils/adt/ruleutils.c
--- b/src/backend/utils/adt/ruleutils.c
*************** get_rule_expr(Node *node, deparse_contex
*** 7567,7572 ****
--- 7567,7580 ----
get_agg_expr((Aggref *) node, context, (Aggref *) node);
break;
+ case T_GroupedVar:
+ {
+ GroupedVar *gvar = castNode(GroupedVar, node);
+
+ get_agg_expr(gvar->agg_partial, context, (Aggref *) gvar->gvexpr);
+ break;
+ }
+
case T_GroupingFunc:
{
GroupingFunc *gexpr = (GroupingFunc *) node;
*************** get_agg_combine_expr(Node *node, deparse
*** 9001,9010 ****
Aggref *aggref;
Aggref *original_aggref = private;
! if (!IsA(node, Aggref))
elog(ERROR, "combining Aggref does not point to an Aggref");
- aggref = (Aggref *) node;
get_agg_expr(aggref, context, original_aggref);
}
--- 9009,9026 ----
Aggref *aggref;
Aggref *original_aggref = private;
! if (IsA(node, Aggref))
! aggref = (Aggref *) node;
! else if (IsA(node, GroupedVar))
! {
! GroupedVar *gvar = castNode(GroupedVar, node);
!
! aggref = gvar->agg_partial;
! original_aggref = castNode(Aggref, gvar->gvexpr);
! }
! else
elog(ERROR, "combining Aggref does not point to an Aggref");
get_agg_expr(aggref, context, original_aggref);
}
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
new file mode 100644
index a35b93b..78e24ea
*** a/src/backend/utils/adt/selfuncs.c
--- b/src/backend/utils/adt/selfuncs.c
***************
*** 114,119 ****
--- 114,120 ----
#include "catalog/pg_statistic_ext.h"
#include "catalog/pg_type.h"
#include "executor/executor.h"
+ #include "executor/nodeAgg.h"
#include "mb/pg_wchar.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
*************** estimate_hash_bucketsize(PlannerInfo *ro
*** 3705,3710 ****
--- 3706,3744 ----
return (Selectivity) estfract;
}
+ /*
+ * estimate_hashagg_tablesize
+ * estimate the number of bytes that a hash aggregate hashtable will
+ * require based on the agg_costs, path width and dNumGroups.
+ *
+ * XXX this may be over-estimating the size now that hashagg knows to omit
+ * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
+ * grouping columns not in the hashed set are counted here even though hashagg
+ * won't store them. Is this a problem?
+ */
+ Size
+ estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
+ double dNumGroups)
+ {
+ Size hashentrysize;
+
+ /* Estimate per-hash-entry space at tuple width... */
+ hashentrysize = MAXALIGN(path->pathtarget->width) +
+ MAXALIGN(SizeofMinimalTupleHeader);
+
+ /* plus space for pass-by-ref transition values... */
+ hashentrysize += agg_costs->transitionSpace;
+ /* plus the per-hash-entry overhead */
+ hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
+
+ /*
+ * Note that this disregards the effect of fill-factor and growth policy
+ * of the hash-table. That's probably ok, given default the default
+ * fill-factor is relatively high. It'd be hard to meaningfully factor in
+ * "double-in-size" growth policies here.
+ */
+ return hashentrysize * dNumGroups;
+ }
/*-------------------------------------------------------------------------
*
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
new file mode 100644
index f59d719..ba1eac8
*** a/src/include/nodes/nodes.h
--- b/src/include/nodes/nodes.h
*************** typedef enum NodeTag
*** 218,223 ****
--- 218,224 ----
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_GroupedPathInfo,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
*************** typedef enum NodeTag
*** 258,267 ****
--- 259,270 ----
T_PathTarget,
T_RestrictInfo,
T_PlaceHolderVar,
+ T_GroupedVar,
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PartitionedChildRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
new file mode 100644
index 7a8e2fd..103ed14
*** a/src/include/nodes/relation.h
--- b/src/include/nodes/relation.h
*************** typedef struct PlannerInfo
*** 256,261 ****
--- 256,263 ----
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
*************** typedef struct PlannerInfo
*** 401,406 ****
--- 403,410 ----
* direct_lateral_relids - rels this rel has direct LATERAL references to
* lateral_relids - required outer rels for LATERAL, as a Relids set
* (includes both direct and indirect lateral references)
+ * gpi - GroupedPathInfo if the relation can produce grouped paths, NULL
+ * otherwise.
*
* If the relation is a base relation it will have these fields set:
*
*************** typedef struct RelOptInfo
*** 548,553 ****
--- 552,560 ----
Relids direct_lateral_relids; /* rels directly laterally referenced */
Relids lateral_relids; /* minimum parameterization of rel */
+ /* Information needed to produce grouped paths. */
+ struct GroupedPathInfo *gpi;
+
/* information about a base rel (not set for join rels!) */
Index relid;
Oid reltablespace; /* containing tablespace */
*************** typedef struct ParamPathInfo
*** 913,918 ****
--- 920,947 ----
List *ppi_clauses; /* join clauses available from outer rels */
} ParamPathInfo;
+ /*
+ * GroupedPathInfo
+ *
+ * If RelOptInfo points to this structure, grouped paths can be created for
+ * it.
+ *
+ * "target" will be used as pathtarget of grouped paths produced by this
+ * relation. Grouped path is either a result of aggregation of the relation
+ * that owns this structure or, if the owning relation is a join, a join path
+ * whose one side is a grouped path and the other is a plain (i.e. not
+ * grouped) one. (Two grouped paths cannot be joined in general because
+ * grouping of one side of the join essentially reduces occurrence of groups
+ * of the other side in the input of the final aggregation.)
+ */
+ typedef struct GroupedPathInfo
+ {
+ NodeTag type;
+
+ PathTarget *target; /* output of grouped paths. */
+ List *pathlist; /* List of grouped paths. */
+ List *partial_pathlist; /* List of partial grouped paths. */
+ } GroupedPathInfo;
/*
* Type "Path" is used as-is for sequential-scan paths, as well as some other
*************** typedef struct PlaceHolderVar
*** 1852,1857 ****
--- 1881,1919 ----
Index phlevelsup; /* > 0 if PHV belongs to outer query */
} PlaceHolderVar;
+
+ /*
+ * Similar to the concept of PlaceHolderVar, we treat aggregates and grouping
+ * columns as special variables if grouping is possible below the top-level
+ * join. The reason is that aggregates having start as the argument can be
+ * evaluated at various places in the join tree (i.e. cannot be assigned to
+ * target list of exactly one relation). Also this concept seems to be less
+ * invasive than adding the grouped vars to reltarget (in which case
+ * attr_needed and attr_widths arrays of RelOptInfo) would also need
+ * additional changes.
+ *
+ * gvexpr is a pointer to gvexpr field of the corresponding instance
+ * GroupedVarInfo. It's there for the sake of exprType(), exprCollation(),
+ * etc.
+ *
+ * agg_partial also points to the corresponding field of GroupedVarInfo if the
+ * GroupedVar is in the target of a parent relation (RELOPT_BASEREL). However
+ * within a child relation's (RELOPT_OTHER_MEMBER_REL) target it points to a
+ * copy which has argument expressions translated, so they no longer reference
+ * the parent.
+ *
+ * XXX Currently we only create GroupedVar for aggregates, but sometime we can
+ * do it for grouping keys as well. That would allow grouping below the
+ * top-level join by keys other than plain Var.
+ */
+ typedef struct GroupedVar
+ {
+ Expr xpr;
+ Expr *gvexpr; /* the represented expression */
+ Aggref *agg_partial; /* partial aggregate if gvexpr is aggregate */
+ Index gvid; /* GroupedVarInfo */
+ } GroupedVar;
+
/*
* "Special join" info.
*
*************** typedef struct PlaceHolderInfo
*** 2067,2072 ****
--- 2129,2150 ----
} PlaceHolderInfo;
/*
+ * Likewise, GroupedVarInfo exists for each distinct GroupedVar.
+ */
+ typedef struct GroupedVarInfo
+ {
+ NodeTag type;
+
+ Index gvid; /* GroupedVar.gvid */
+ Expr *gvexpr; /* the represented expression. */
+ Aggref *agg_partial; /* if gvexpr is aggregate, agg_partial is
+ * the corresponding partial aggregate */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+ int32 gv_width; /* estimated width of the expression */
+ } GroupedVarInfo;
+
+ /*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
* path, the list is stored into root->minmax_aggs for use during setrefs.c.
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
new file mode 100644
index 77bc770..abc2ac1
*** a/src/include/optimizer/pathnode.h
--- b/src/include/optimizer/pathnode.h
*************** extern int compare_path_costs(Path *path
*** 25,37 ****
extern int compare_fractional_path_costs(Path *path1, Path *path2,
double fraction);
extern void set_cheapest(RelOptInfo *parent_rel);
! extern void add_path(RelOptInfo *parent_rel, Path *new_path);
extern bool add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer);
! extern void add_partial_path(RelOptInfo *parent_rel, Path *new_path);
extern bool add_partial_path_precheck(RelOptInfo *parent_rel,
! Cost total_cost, List *pathkeys);
extern Path *create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
Relids required_outer, int parallel_workers);
--- 25,39 ----
extern int compare_fractional_path_costs(Path *path1, Path *path2,
double fraction);
extern void set_cheapest(RelOptInfo *parent_rel);
! extern void add_path(RelOptInfo *parent_rel, Path *new_path, bool grouped);
extern bool add_path_precheck(RelOptInfo *parent_rel,
Cost startup_cost, Cost total_cost,
! List *pathkeys, Relids required_outer, bool grouped);
! extern void add_partial_path(RelOptInfo *parent_rel, Path *new_path,
! bool grouped);
extern bool add_partial_path_precheck(RelOptInfo *parent_rel,
! Cost total_cost, List *pathkeys,
! bool grouped);
extern Path *create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
Relids required_outer, int parallel_workers);
*************** extern NestPath *create_nestloop_path(Pl
*** 124,130 ****
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer);
extern MergePath *create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
--- 126,133 ----
Path *inner_path,
List *restrict_clauses,
List *pathkeys,
! Relids required_outer,
! PathTarget *target);
extern MergePath *create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
*************** extern MergePath *create_mergejoin_path(
*** 138,144 ****
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys);
extern HashPath *create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
--- 141,148 ----
Relids required_outer,
List *mergeclauses,
List *outersortkeys,
! List *innersortkeys,
! PathTarget *target);
extern HashPath *create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
*************** extern HashPath *create_hashjoin_path(Pl
*** 149,155 ****
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses);
extern ProjectionPath *create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
--- 153,160 ----
Path *inner_path,
List *restrict_clauses,
Relids required_outer,
! List *hashclauses,
! PathTarget *target);
extern ProjectionPath *create_projection_path(PlannerInfo *root,
RelOptInfo *rel,
*************** extern AggPath *create_agg_path(PlannerI
*** 190,195 ****
--- 195,214 ----
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+ extern AggPath *create_partial_agg_sorted_path(PlannerInfo *root,
+ Path *subpath,
+ bool first_call,
+ List **group_clauses,
+ List **group_exprs,
+ List **agg_exprs,
+ double input_rows);
+ extern AggPath *create_partial_agg_hashed_path(PlannerInfo *root,
+ Path *subpath,
+ bool first_call,
+ List **group_clauses,
+ List **group_exprs,
+ List **agg_exprs,
+ double input_rows);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
*************** extern ParamPathInfo *get_joinrel_paramp
*** 285,289 ****
--- 304,309 ----
List **restrict_clauses);
extern ParamPathInfo *get_appendrel_parampathinfo(RelOptInfo *appendrel,
Relids required_outer);
+ extern void prepare_rel_for_grouping(PlannerInfo *root, RelOptInfo *rel);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
new file mode 100644
index 25fe78c..38967da
*** a/src/include/optimizer/paths.h
--- b/src/include/optimizer/paths.h
*************** extern void set_dummy_rel_pathlist(RelOp
*** 53,59 ****
extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
List *initial_rels);
! extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
--- 53,64 ----
extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
List *initial_rels);
! extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
! bool grouped);
!
! extern void create_grouped_path(PlannerInfo *root, RelOptInfo *rel,
! Path *subpath, bool precheck, bool partial,
! AggStrategy aggstrategy);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
*************** extern void debug_print_rel(PlannerInfo
*** 67,73 ****
* indxpath.c
* routines to generate index paths
*/
! extern void create_index_paths(PlannerInfo *root, RelOptInfo *rel);
extern bool relation_has_unique_index_for(PlannerInfo *root, RelOptInfo *rel,
List *restrictlist,
List *exprlist, List *oprlist);
--- 72,79 ----
* indxpath.c
* routines to generate index paths
*/
! extern void create_index_paths(PlannerInfo *root, RelOptInfo *rel,
! bool grouped);
extern bool relation_has_unique_index_for(PlannerInfo *root, RelOptInfo *rel,
List *restrictlist,
List *exprlist, List *oprlist);
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
new file mode 100644
index 5df68a2..07bc4c0
*** a/src/include/optimizer/planmain.h
--- b/src/include/optimizer/planmain.h
*************** extern int join_collapse_limit;
*** 74,80 ****
extern void add_base_rels_to_query(PlannerInfo *root, Node *jtnode);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
! Relids where_needed, bool create_new_ph);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
--- 74,82 ----
extern void add_base_rels_to_query(PlannerInfo *root, Node *jtnode);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
! Relids where_needed, bool create_new_ph);
! extern void add_grouping_info_to_base_rels(PlannerInfo *root);
! extern void add_grouped_vars_to_rels(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
new file mode 100644
index ccb93d8..ddea03c
*** a/src/include/optimizer/tlist.h
--- b/src/include/optimizer/tlist.h
*************** extern Node *get_sortgroupclause_expr(So
*** 41,46 ****
--- 41,49 ----
List *targetList);
extern List *get_sortgrouplist_exprs(List *sgClauses,
List *targetList);
+ extern void get_grouping_expressions(PlannerInfo *root, PathTarget *target,
+ List **grouping_clauses,
+ List **grouping_exprs, List **agg_exprs);
extern SortGroupClause *get_sortgroupref_clause(Index sortref,
List *clauses);
*************** extern void split_pathtarget_at_srfs(Pla
*** 65,70 ****
--- 68,84 ----
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+ /* TODO Find the best location (position and in some cases even file) for the
+ * following ones. */
+ extern List *restore_grouping_expressions(PlannerInfo *root, List *src);
+ extern List *add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *aggregates, RelOptInfo *rel);
+ extern Index get_expr_sortgroupref(PlannerInfo *root, Expr *expr);
+ /* TODO Move definition from initsplan.c to tlist.c. */
+ extern PathTarget *create_grouped_target(PlannerInfo *root, RelOptInfo *rel,
+ Relids rel_agg_attrs,
+ List *rel_agg_vars);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
diff --git a/src/include/utils/selfuncs.h b/src/include/utils/selfuncs.h
new file mode 100644
index 9f9d2dc..e05e6f6
*** a/src/include/utils/selfuncs.h
--- b/src/include/utils/selfuncs.h
*************** extern double estimate_num_groups(Planne
*** 206,211 ****
--- 206,214 ----
extern Selectivity estimate_hash_bucketsize(PlannerInfo *root, Node *hashkey,
double nbuckets);
+ extern Size estimate_hashagg_tablesize(Path *path,
+ const AggClauseCosts *agg_costs,
+ double dNumGroups);
extern List *deconstruct_indexquals(IndexPath *path);
extern void genericcostestimate(PlannerInfo *root, IndexPath *path,
Re: WIP: Aggregation push-down
Antonin Houska <ah@cybertec.at> wrote:
Antonin Houska <ah@cybertec.at> wrote:
This is a new version of the patch I presented in [1].
Rebased.
cat .git/refs/heads/master
b9a3ef55b253d885081c2d0e9dc45802cab71c7b
This is another version of the patch.
Besides other changes, it enables the aggregation push-down for postgres_fdw,
although only for aggregates whose transient aggregation state is equal to the
output type. For other aggregates (like avg()) the remote nodes will have to
return the transient state value in an appropriate form (maybe bytea type),
which does not depend on PG version.
shard.tgz demonstrates the typical postgres_fdw use case. One query shows base
scans of base relation's partitions being pushed to shard nodes, the other
pushes down a join and performs aggregation of the join result on the remote
node. Of course, the query can only references one particular partition, until
the "partition-wise join" [1]https://commitfest.postgresql.org/14/994/ patch gets committed and merged with this my
patch.
One thing I'm not sure about is whether the patch should remove
GetForeignUpperPaths function from FdwRoutine, which it essentially makes
obsolete. Or should it only be deprecated so far? I understand that
deprecation is important for "ordinary SQL users", but FdwRoutine is an
interface for extension developers, so the policy might be different.
[1]: https://commitfest.postgresql.org/14/994/
Any feedback is appreciated.
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de, http://www.cybertec.at
Attachments:
agg_pushdown_v3.tgzapplication/x-gzipDownload
� 5��Y �<�r�F�~��b�����!
�[T�*Ffb��V��}b���D J������3 ����S�([$�������svkx�m�Gq� ]��^���]�v��`��}���O������nt�WF��z]s�w��a0��b �|���.N�P�k�>]�mo��7z�9�~�c�-��h�}
�v��gv#z�� �:j�� R��7���?{g���<%e?�#�@ �z�j z}6r{@�`O��W��G S~R�������l�/��#O��{�@7����� �[.?o����g�a��7��7��5��� ��� s�7���� �z;p��L�d��s���}{^����?��{������Y���]�x���N�?���fvz��f���s��� �gA���� ���]�� �-��_��-=��
t1�x�� \��
L�2��N�1� ��_��� Z������5O��c�x������A�0���,�psL��^O3�}��Z&m2����k]���s~1� ���N.��AS���=���z�lwI���D6�*c����Yx_
�����|� �HMk�^� ��U��SI�[�v����(�g;�t,6!MS j&d9!�iM�$!?��wv�e�OQ��r�`��%L<��= '�FKL�A��o��HK����}�6�������H�Q_�����F
g�D�o�(�l��uq%+\$�n����C� �dL"�� E��e���rV�{��`R���D+u�q���uH�u`���ru4���1�����dt#%J{�Tp(�a��ld,)2���������v�0��s��!�@ub'_��������Lkg��I �k�o�~�R������W0P����R�z�j���p�Y��b] ��5�3�����1g�y�:}G]H> 8HJ�U�����E��bJ
/����K�I���WI7++�������0���Tv{�du�X�h�����`#UxER
�d������� C��} z���l�)|Q4�/Cwd:z���]gh/���V�
`
����S�aL( �6 �rS�pP���(/��GCL+4�U[c����&_�\���&� [�T����$ e����� �� @�Z,G����j|{?�I�6^0��2Y��[� � B� Ci]�g���Oh��>� U1V]|�E*�S�L�T��
�� ���A���&��]���^�3.���k5O�W�x��=) �?��Z
���Z��A\�WXO
�e��X h�����Y�2�� Z��o�� L^*��������P� �w4� ������ �Z����T����|i��D�����=�������.��'\����(F���cq�4�YrL�3{�&�)Q%�� �so<� �
{�| V`����o;�'u��
n��X#��A.8�� VB��X����,|+�����
���X�`���G�� �� !�\�����V.s +�~���Ar
�GzW�3�� K�/�XF�Z�/�_j5^��gp� M�!��|���v6O"��O���#�I\N��@n M�# �t�P,���o�r�v{_�p�L�(J����](�-rF�L�(�` ��x����75��(����[��/�C<�� N
+�0�5s4P�]�!��n*L��y*�J�z?�X)�z��q��VA����J�@��x9�a��-�4�z�
J�u�~6�M�a9�V��� �����(�-���l6��hor�5H����%���� >��7��R�F��.i�%���x����3p: K7]�tj�Y\]8� )�� �@��@���I�+J��$J��[_��i�%1Fj����<�����l����7��^,�moD>]��C\��E��&��ByS1l%�s�hGM�- �sS�o; =��%kX�����+���m�@V ���Jj�PI��4�������� ���� ��!R��Y�����M����g�8aod%���
���:�N�������+�>���au����U:&����M��U&�$�H�[�{=���n_3-�Phmu�\[�~u5s{���"� 6/�J��g�j"
�f�����~��y;y{;���^�xs��F�h��4<��}�a��� s4mu<D�iP0i���e5n��w1� ;DR&�(�9�j�I�)+9{����������5�N-�4�� rwk�f�S=�|nU++�+��I�����P#�����yOK�O&��d��Zf��A����5���vr1 _��fz�B���-�������@�����/� i%�g?A���U�_E�����'( M�Q*���T��\y����� ���������O?��,J��k��m��Ub���Y���P�>��)�����I�;2�l�;$Z�������� �� X��(����T�T��"V��ux�X��*,�'l2O^Z%c���U��[U&��/k��V�- �e��#�2?UL��e�
��J�L����ux ��`���
)���j}ghn�32 C7���
�.R����D!D~��Q��z��+�!?-�[A/��R -����n2~;���_�W]I��P��� 4���GN��������Z��.
������
j��wB7 P���)���| !X_�gY�.Fe���� @����BO�0��C��Z�2,_^�g��GMi�{|\ 9"�Q��d���pV�P�[������ ��!���� �����Zm$�F�e�����#5��kw:�H .,�B#� ��d���t�b'r����M���{�)�N�([\c� !1j������������z��bK�D�oyb{��r�F<�� �9k;j�@y0f�$ �z��X���������O���Y��]��}5���vMC��}�� SW��R�Z;��������-a��e�0b6�V��_v G��4]oA�?���a�:�����c��d����Y;�m���\�l*�a�pD%��e=ia6yK��q����M~QL��>�v� _��3�����~_���A�?��\�ZD�%a��U4������hG���9���=O?�
�t,/�V��O��l��#X �����l�Y��xft�����Di�������{�����b������ ��mE��m��:a�-���d��e8~��� �I�V�^��;�+g�x�lQ��d�������N�7 -x��X�jX�m��/�Z�/!�C���.����j��rY��4�(k��h���*�.��
E`/Q�F&kM��|��K6�� �����{���?��� ��Q�I.��)j�<�� j�Q��]��dr�$ Jw/��_�-�{�)C Qn> ��?M�&�V����[;��C����M��*��N��u�����mpsX+�*�:1V��� ��s����������Z)5�%�]y?'�s3}��%s��X� ��4 b�r��rZ�����)����w�� �zU���5
�CX���k,L I
������O�����W�_g},�������r���J��O�l�� x�~m ���P:7Z�6<Hb
C@��s��:��D�S_�,)�.�+'�q��s��tu�C�����{�x����y9"L��(6s�9Nk�������4[�6����� q�v�� ���,r������{F>���tg�E�� ���p����@�DV�
0�T���y�,�����l��S� $rb�)�'�����w0�����g��`$!�� ��<b�"� ��)��lS��r������2-���C�L�A�Z �Z����3L��b��SJ��a!3����T�|�H,�7H����
��!�_>]���r~����\�������d����O�V��$)���IpY�H�� '��>~�&�Ij��f��%4 �Z���&@��!; ��"��w��qj�.�������i:*�V�� �L(��I �6���p0{N�-�*f�0Y��DV|�k)x S}��Zm=X�|nz�@��
(s��l���9�Iq ��5�~y)a��<z�b�!�DIdZ��&X������������/v���u����31�g���Z����ZO����b
GI�����N �2��Z��J0PT7^�m �T������u
�"��Ft �����$�T������+��\*! ���dK�?kaB�
��~���'u�� ������px-c� �a�`����h���FFO��G����y�����7����/��y>b��+�J��>W>|HK�%�R��i�E�ma��|!)���.�;C*�H} u"NQ�Z@����(?�>��V������D?
O��[ <��9^�y`�N,�"��;�� v1~?��\W ��X���No(�!�ra�6�� /!�
�p]�l���x�
#0�v..������3#4�f@�96�:4;B�B���NF ${ZF�.���h��tR��& ��L�/P�^��#� �R� ����PT�dOk�Yd�&�����
�� @�MMn+ ;�'� ��H�t������* *���S�/�� $��bQ ��A��7� �c�M�.��� z����B��8���j&�q:I a�� 6� �1K/(��8;~(�}�������i�?���I.�"��"����B�C�9I>��i�J����@�� ��S�D*���JyoE���n�NE�S��R�R jQ�C ��zA&� o�n � �I8 ��:��(���J�#�����O=5�M A;�)|�
��'�^�p#���b�����?,|C~8��Q7��.��v�gSJ:u����t�ci]��?u�m��E�q43���z�2��!�����"NI/�9T
��8�"%�Tt��q�(�1X��C��[ hlOi��;�0�s;������|�� .�`��V�M /_�D�R�.f���h�2�����m�m�
��2����JS���(�Vj��;l��������yC��@�X�����=�]1;w]
��y��H��<�\bvi��4Z��$��2��R ~�K%��=�M���c�DV�F4q����y�. ��]���(��N�#�)�N���M��a@����@�rH���+�+1�* N��.���Z��V����&y�8*������f���0.�<LY�������
D���w�����������(� �@���|�@(��@_����@:�f�nr7��\� /!]�D�;����L 3�Wc��a��j���?�`h<(��9I� ��T����/���,��� 'DL���p2s~
M%��N� !D�D�������E�]Me�D6.���p ^< @���R���� p����t�|�j�����3H�� �0��z��\�2����T.����(*�)��5 H���
�=4���:����;�_T^e�UA�2���� r���yK����\���q������2�RAI����QoyVO3��(�MR��{{��K��cmA+�� +.|>� ���$�Bs0l�h[%�?U�:��Z����$���"R�����,����U�� ����e�������g��@b�VW���������*e1-��Y\S���VHWODk����>?�KC�5imr���
���H����-���p�o
���+�N�-0� d&Y�D5��p +z��j�����j�K�E.=�����(F�y�ig�U�4Q \"�l����4��5��!L�(=���09��� 6�L���a��"��8Da��^�������s�� ���;�`�3���q`v�o���5�����C�
G����s` ���Vh���<������%���?~���=���h������w�����O��R���g���,� ���� ����� 9
�=���#�1����M����6�.���/1���5�hNe�9f�����[Z_�n,| f7ej -w���i+'��� -�Q�"\�� ��zX)��2e���������g� !� '����G�� ��3
���te�b�h�CEx ��C�w�����l
(�"[H����sq��^�������,�����)���Sa`5qdh*�$�]^ �~�?�������y���7{������
���OA{�w�lY�K�N��8�w ��vff�}���%v������;������ �R���vfr�xl�B�P���J� z��$���pY`i4H�d�_����+����P #c's4��|=���wXN��f�q�!�j�lF7��� �?�<�����}l���'�<g�G�P[EMk���cg������z�)�B!���������d^d��P5 Z��1���_�YDM��#���\���Ul�BjP�Et�U��i�0����g`�W?�[��F3��M����� �?���l��qa�s;�����f��oE��i��5��)ax��
�:��� x�,�o���n�`_<�� e��7���h��b��c�O�D�^��Yx����?G~��K\�6[�in�h�����\�wZ����h������/d0�q[Ce���P���h" 6;o7�d!x6���*�2c=7]+@=���Q��'gx]������bp���>"��H#�� 65�G��@���d6�Awt���q"���50�F> ��^�m���� D����mj]�"�Cl��(c�%��3$ %���}L�Rd^�v�O/�^i� ==�7f&��#����[1��Kp+�u���l�>
� }L���A��3{��U+�w����Z@����|Q65����FI�i:�APZd�)��}Oau:�H���:"1���>c
l�f6y����<���M������� ������r+���zK������(��F���i��� ��V�
�
��}�;u �� �x�
w�=l���+iH�tcZ,�=�zp`^�{�����]���/��`�,M�����?z>�$���V'J8�3 �e{G������j���+# ��_��Y���XT�"��A� �/��Np(����~�q�/�\�� Z`�fw����!b2���� F�����&BhE5�D ���QL��W:��OK����
/������J�8���j�bT����?�hg��Q�8;]�S���~�DM{2>K��� �����}�?��J\w��5D#3��!?[nS9�#�>���`��p����*�:K�+�BZq�;�����H�&? nk��O�8�X1t>'9�2K�}j$ �4[�Lm�K�]�"��N3�[�F�Q�P�����GB�R�I�Lt/��8'���$�^�f�+ ��U�r�a# ]��2��~;����9C� �� +�_��>�Q���9�Fo��X�1X���$�.��c�z ���z+���,�$�l}<rE ���?�$$F������L�Z8;�c~��5��"�=g �$l�Ye��6
g@�uB��������l�;�O�=w�����^'��<k�f�f$~w��\���*;�H�����nb����8��@�4���G�I"4�q��^���;��:�������|����P��>������6����u_-� R�! ��"#�X��.��*�P� J ��l�E�D/��& Ao��@C�.>]�5� [kX��9*�p�7 �+������|�AY:�D�w�~H�v������+&�k�{
>������s�'�A,��1��oe ��<Y�W�O�����Ho�W-�Bh�
^Y��Z�[B���q*�W�z ZY
FP��;����Ezo�7���r�M�/�I\#������4S����D���xB�u5^�!�������w�g�|{L�2�xi��f�c0 I1x�I��=�P���-� � ]YR
�1�+uL��M��kb�����!BG�cLE\�V3V�� �������u��+�{{Z^�g$v��QP 9 1F��{Ys��Zlj�WA�����?!�_:41���Uqdi��� �o�jS�Ej�������c�z�v��v�TG^~�!���Oj{w���������:D?�w��8q�����b�����`Tg���5/���X���j����D/�_�]�^��R��J�P� ��:�Qt��5��P4�P���lZl���c�����B>#/E���<�n�������F�w;����e\\�Z~��]�+�=��|�*�s�����l��;����{;��}7 f��(`��P��?ln����W� Z�w�F��c�]#�����M��]��p �;5��d &R���p���
�%�)��mK]�F�������<K���������,�$�� -��_}H��#2 ��>�-�f�w[a��%�7f8f����h����#
��T�Rd�T����5����Mo��;���j��R��8���?Sf }&�������W��)�P�XK 9�8��I�������B�B�Z�������S� ��J!��_Wc7,��K:�{_��rS &��� n� CX��~ �$���>yG _�+���W�������IW�� Y������A#�<�&�V��4%�����_;�-��������m~S�X�L���i����xH�G?&�����V�Ctw��]��w��z�|�������2O������oU,^�����p0i7�F�� ��C5�*:� _�{^~����� ���6� R���S��{J����#� ���G ����� 2r��}:I6�<;}�:����c������H������lD! ��A u�P������k�����a��,���(� 2�]�L���@�uz�V'�#�����l H!�@��
��k��|J� (�A���D��� �8p��cQR���z�Q-� 9O�a�� fuH����)�0�� ���@]��6�J�=F�G�
nP�!��|0��,$���d��e�����)��,��p�������O��p�M�e|M7 �-���wL�T N�t:�#L6&c������� \�&��t�@����D]��'�M��^o�36^�� ��E 8���6E���)^��� y���BH-R��*���������ZjI�������
������a�������&��P6����,Fn��?��*B�$���lK1}O/��5 /�s�A��I����BX�����H}�r���|���d����(�{�<C�� P��Y��Em�� O���g~� R��������j�[�I� z�����.�maf��Y��s��x�)l�����A�01A@���OO�<=4� �������;��� �!�2 �<)yY�l�-� ��a��s�rfkN-����'6@��Nw����TQ��9� g�hcL����`:
0V�� �y�YlP�������a��f�u~J��dI3�i���C�~EC ���53���_��$��v����������L�|`�e^�f����v� ����Dr(y��� rb��)� ��j{�����"V���������.��f� ��^L4S���z[������?�~�Uw��{�}��g����������?�����O�=?*t��7g����� C���!<��_ w�g��(��8,�) .Y�
�+���� �1��%����I����� ������@2��y�/��� b�8m v�OO�������a<
h��E�g|�,�z�\PH$x�18 "���8� �tn�C1lH�, �S�c
N����k+����|��8�<�� 8D��O�Hvp�� t�tRu�� ?�!h�%%�8=_'��BR"�<�n���e�6KgE�~ ��� (i���E(L��L �� aK8VDT��)����}�{99���W���s/.,X�����3 T
��x����7��p���s;];�"���������'�9��e8��
�}X(��R����y��/r�������,�z��?���|{ ��[�{����:F��NC�F���w�yVR�1���y l���k��� =�P5d0{������9��-`����B��mt7%xZ��r��l�"�3`�K3C<p��T��Bt�Q�7a���N4NcSpb0�i�%
�3t���89�:R=��J�]X`^�4N��Wo���*v8����� u Se'�p� ]�G���c���So[�:/�3`��F<�U}�|�� �a6���K�����/�����oJ`1�9�%������d�^�#u�d�?�x�/^�x�����R��� 8d�Ag��;!��3��+K>�s�Z�R��Fp,����js���0���^F)�J����c[�a�t�*Y6dwUixW �������a��H�$w { t(�:C�"��d'A�9�T�,��A����}���m2,�:,����Rh�K7~�]d���x;�A�������n���?h�2��}������6/�j �6�M ���L�����b�B�HLDR[� ���5�`F��'~ �����MNh�`��O5O�|� u~(��K�V��J�~�"I*c���
��G��/�nv�~� �m. �K .���������������kf�����r#Z�v��Ah%�l �*lI1*�3��
��V����ZPL{7� (��`,�T��v�5�7�W��t�s`!���g-o�&�<�]��(#g�P|�[:V�^" _h2�T�HC g�R�2��sp� �����H&f��@�ns��I�sv��$�t��y�S�%��n lR�.���HGPD*_2a��9(��l ���B����y��@2;iU� ���1�- �Rr
LD�5� �3��~1c�-|���?'�
QF��!�d�A��G����������k �-�^�����}��s������M[��jm����%L�D9��8�d�n2� �F
��� /�lz��G����#���@C.!>N}�|�J�R����
"���[����X��D
b8�T�* ��R'��BvX�IWk[��B~H )����&\Z
~�{E _���g��u���� ��+��j�� �Y���a��QO�W�"��
�y��z����su��T���?�;�<��NP�O,9�u����1�m��}G�v��|�����7�
�(�e
m�� 0�#{8�\�����l�"h:|V �]��+��z�F(��q�=���� �������ez��VkM��� j#:�:/L���44>k��q�E�3������z�q/���d�:0ILfc���� �h��0 �"�!D����`�).�vok������.),���p$��3�@���.� ���;����S��=���83�����|Vt�*���RI ��5�]�
���� �������'8�$\��w�+����V�h� �~���V@����D �8) ���F�)L5@��`�J���
Z ��q ��U� � >J2��x+���$S~dq2�>�=���Cw�}�a�(�}'{��k�e;s�g����' ~���}���q�w��� ZXc?��� �S[�7�x3�. �y�<4�"Z��me} �/�5��"�������9�}�{]r5�P�����'��Ne��U�~T����E��&�� � ���?�4Hl��ajc����e��p�N2
���A��u���1^;�JG�O ��������ht\U�G���|�n���"46� w��C\��}:��p����:������8p�
e�Y&-���R#��1���8��u������N�\�O��h?P3���Lq �x�EW�;���g�k7�Nlw ������*D�)cY} ������~��C!�7���!q�J����*&����%E���)�!p.��+� =)�!/h����z:DIU�ML���M_�vM�da���� o6 ?���Xou��N��%X�����N�+�� ��]( A|�r���n[ O��:��vM���������8�h��mD�1'Z�HA���S] �DKF
� ~(��� .TG�*�D49"��<B*G&����x|����$���B�%���@C� bY��<96�]�C�H F�C)I���y �3� �������p�-�j���z���V��E(��M3mm�����;��v�r�������ic���u
E~� �����E�j7Z���\�/z� ��LR7oZG6.E �~�E���6���P,��� 79�P� �3����@�JA<�` lr"�gH[!� ��@2<����m������&H9G��%���<|�2e0`� �� ?�}+0�@ B(��)�.��
�c� Z6w�j��C�5tF����f�Vi'x���P�D����!���z
�\�Oz,P��-"q�Pi(������ ���,�L� �4�:�d�=L?J��F�R�c�A-3����t��kd�q�s��F� �a3�?6���T_�7o }X)���?�w:�F�w�t(�WT$������d�R�"G, W��� �z~H �����������4l ���}8��j!
�D�R����QY�B_pN��66^��;��\*�> ��$���3��eu����K�
k�F��2;�y�s*=@��P�� ���sAEHVX�
rmb 3�Zg�Uij��j���� .�+k7i�MS|���w��4 mbn��9��Mi���X�{��@��:�L�� ?�Y�J���������;�� cH��wH�R� �C���I�o^���G������F~Pv e����*�d���tX��&�gfOo����Uf� [�,��d�K4�&�[Z���s���]��W�gn�e��M� �e�2���L5��$�c � �Yn��d� ���w�$�{G3���^&���i1�c�=��KLd2M�Bm�B��K
���� ��#��a�[�8 �,el5j�ns��f6g�2E�%��#�Mr {N��4_�����z�[1|���>���<0��O�� �I�5B�d'��f���G]�����F�l&����� F�5F_&�7� JJ��MP�a[q �gXBn�� M���z��>�M/x�T�|��K��q����,[XK�2[�N3>���mb��nl�� Cb�����=�1�f ��;K ����b�>)o�d�l����<�R7^��U�<� �����6Oq�"�l"r��j�4� � �L(*�st�i3���5�c���q�����j��j�? �fN�mW�p�M�I?m4�'�N���������� ������� Ur�����[�A����0���� ���x�� D����"�y�����D��cG6������~K]��'uUeT���,�a�^"9���:�y�>�� ��[�{F��t~9 � �G:�� ��������H�{����{s��!r6���� e!c�jpo�9h��� H=��%P�!���&]�F� �)���S���N�������yl�N�Q� ��pO�������>
��� �6��p�d�<�&�F���v������o^S��^ ? �����4zK}Z�~?V ������A�� �
�� s�� ����2�Y� ,��� �� H@�b�Bs3�#�7����H&� y� �W3� ��H.�p�^:�1�P��\}4?��� qo8X%]��FE��� `���6�Y��H�x�K��w������ f6��&����Z�m����<������c
X ���'I ��(�� �jL�:�KFD��w��H|'m���~`8%�S�d��9x��z��O>`�juy���
x��E| ��D��'���H�A�Dpf��1�S
�
�&����]i���3��M�P��4����}i��MP �_�a`]#zMFbe cLuP��"� z:��(lY<�'q ]iaN[��{�����7p�`��Zv *oq��F�t��6�0
��|�����V7}� ��<�S�mQ�^ �|�rH��)i�u���e���f�������lI�� � j �#&��f}��]ABs�]�(����@��5L�c�L����!_&����(�c@ &�B�������8�.����#�������������{���������� �C�0K�*��s�'�E �<�����w����#� o�t~R��+��O�� ���:p���KU������@N��;�*���5B�p:G��?�y�|���K�`J0�����9j�� ���!��}�}� {����W�3�= j0a��s�E��{��)���h���(����t�+l8��Z��HG��B�����]Y:Y�%�6�,EW�: �@�<�H n3 �*���B�!H!�)Eqlt&���tG@�O]~�2��%M-O\pic~�{�E�Y+���C �E4�c���A��#�e8��Z�`+���
�$s0�`���� f\k�A\vA�h %s���������8�U�*��d� �`��)��.�Q��w�^���{�Y�/:8 �� AB��C��"�;������CU�L�5�
<j�T�)�(mX��Q������b��2j|hN���� ��C2�J�#���jm:���9� ��?t� �$
C�6E���[k�A�S�{��0p]1�Y�@s���%'�����L� �yt�F ��/y��%f3^���i]S��:��4��� h�R��`_!�������Q/�P��� �f���#~�/���F6�r���F�Y���� 9�sYWB.���fAxZBh��A�/&:���'�v����9 lI����K���u$��R�5�R��`���p b� F�M9R�gp]�h;���HUr4�r�
����j�
d�u�V��j(h�=��J1o; z 98N���k���z�W�^
����!BU��%[�c(:�)�x����B�� �)��Ju��0ad%{$p�1�B����j�Q���WSs��(L<qQ��������\�, 4�A��L7?��f��"�n��>&��>�mj�P ���>��%��LQ����T0 "XiX����� �_��x�e�)� P�^!!��1t�r���`$�
��SJ$���������������������@IU���6�q��8 �%���M/5�G��1����y]Hs�A����FyC��<erQiD��:�f �D�f)��L��}�� wu���N�:����c�I�{����^���K� �%V vR�W�f�zb��f � >+E@���Po�����Xw�t�?�\V�F�}�L��`-d�� ���������2k���Q��jv}dJ���`���E�G$[�)��+�yP4T� 5��
0WJ�f3L�%h��� ��p�!��4\��*�Oa�CFU)�[t��[0���� ���T�
����G��0{}$����B�^�wg�F���rMD��
�8,�� ��.��1w!���������sz��?�\W�e��m���[�Zg`{�7M��/���N0��l
�&A�D �)i�E=))83~g }�1}��2����L�����A�^}\:vp8�|�N)(��P�rST������;��p���jH�����������������<Y���b� (g��*�#S�IC��{C��4U5 ;��p�)X[���[Z��� �p������z�$����W ���|]~�!�W h��4V���� 4�� �X���B %�L���$P�=�&!�����
��� 1���j���L=;�g��p��������,:�����3��
Z���(! QO�t������t�AR ������&��@*��=T����JL���^�X�������TWUt��� ��Eq�`��x������+� H��P@�j6[liNu�t ����u����~�XQ8���+��j�^��OB�<H[/
�3B��,����R(��\&?{�Vw$2��q�� �H:����H����7���d.O#�qI
�����}��Dz��|�1,0+�:�^��gOO�( �>?� >��J��(��G$l��*�J_���u�\�F�����9������f��~p����$?��@�$$�N�����8��u���r�\y*���,�N1�,]@������������!>z@��R�z��{���(��
��L> ��s�O�����`��� ���v F��5��(�i�R��F; hQ�����nc�M'��j>� ~���� �z!��� ���B ��Z]*W�.����9�`�v��g��[d���W[�q�^t�8��>�B< x>� K ~$og��i~�7+�(dL�d�K7N(p�K��~O����6�,�-x����� �����p��.!���h�J���>���H�b<��p�:������%��si�V��j�P��z��-,5a��
���� �����|a���Z�Fr<�����a����A��� ���GA��<9N�`��a5��� ��x0�����g*�����`p�A/�N 5' ���e�,���'{��\Je'���9�QQN�6h�m��3����1+rQ ��U�q�! �<����VC�O�G@����O�;rv�)�v��� x�h`@
������]Y>w!�}x��+T5R��� I�!d���!h%�0w � `=8Y�JJ��q�������n�O���5]������}�� �g ��K�ps��#L��#�_�?� ����Q����i�SP��Mx`(�����|�����0��]�j�-:��������[-���ra8��|K�+L�"l�7�h�G]UmP�$���c�pP��i�{��
j�xW��sL��S�����2�5a#O�D]a�'9>U7ux������% Mja�p� EGoP19I�%)w�7=
���3��E�:�v��v[����9m���� �.Q��W$�(A����������78�a7�z� ��a����R) �����d�"* WL�Yu��s��o����pC0/��r��h�p�Y
�&������D��q |���P@6����� 4����[0���Sm�jV �O��F�; �&i��UE�U�*��0�HMYB� %�<��
�� 5���+
�=�J��b��)w@*��V+9Nw �����V����,�����^���y}����<a�Tf ��������v��'�V5O�.+y�6+� �#��&M� f��*�����7����Av%RmWt�
���%S����c���E�u_|c�'� �X��zG�S��I���Y��.]���q�*}��3��:E� o�����}�;���bMrx������
\���!�y�>hN;�d�V��������@�W� t��#\[��U ���� �m����x��{MW"�dz��� K�3�#���>=�O��5����e���Hp�U�������}~����[���� ��:I���r�.D� 9��C<�G& ���J����@1\�~/ Y� �����J:��uq�i�1��\��{�7l
h�`�A+�`-�Dr�������������&�B ��#���wC1�plO������� ��A�������I�P(�c����&}A�a����a��m@��
=R�Cn �]� ���I� ����� ����d��?t �I~�)M��:o�G�N�Q��k0��AW�F��G ������f���S����:�^������V�O�o7>��[�VC��o}��u��LnOZ�f{4h�:����nw[�N��z����V�������p�O'��~�����:�����{�?��[�^ ����@m�����jCF��'��O�v�S2���/�����kF �� ��`|2=7Ac���+E�3� O������tzM�RT�F���� �~L7?d�tv�� ��@�^O] zm���^���
��t
T�D�6�� �� �W����X�juW���U 4@ ��p�i�I����X��~��g<���-���<� '�%X���z���������>���
���JyF6��p��U�O�I;4��`t�>���W���Z� � �bz ��9��2������/���]c������x��} 3���!���q�+���~lA��05R�=^� �cxj���A�����p(>�]����-M�;����Z|�jFq�9��k���b���V35�_�� {
)���m7�j1���:v1��x ����W.� ��>�VY���:�v���g��'��� v�����_�u,%<�+R�O�
���O��j�����R`p�]o���=m������n�@���]�����^�
_(p+�#��/,M���:��q"92�2�|YS��"H�
m�OI����-��v� q������8l8��Q�[���!�6��&�h)���F�qw����\����%+���"V/?J�s��� "v�U�����5 6�~?� '�&�����k�3a�!���j������ ��A�mX�u���Z7��B-W� �2��
�����)4�6���4�h�-
�+@;\�T'G���\qN`� � c��W_E�t5�9#�4��?+����R��:mbG��y�!�7h��A�e�N(tD��v���P���G��
j���J'��;�Z��| ��
&B���D��X&�Bfb4��� I����!,�+���V+ ������Jk 1�p����j{q����|�/��ph���Q, � ������i_ ��N�Y����.��:�[of����M��F�����u��ku��!� SG�`;@���i���
t� ����#�3�bt���bF�ryt�6��l�X��p)�����Y�`��� fl� �~��������x���*�u�5H1�����������!�o�2�i����Q.}���X��}i��H�X�9�GWIs5Yq��CL�`Z�9_�3�dEw(��e��d�Y��N��Xa1���!��k��o=���;���4��X����z�~�z ��l�]YN,��g���<{��>�De��2�U��&�W���0M��n���O�����C��a��1X����["|�(��^��>[��@��n� W����i
Zj
Z#qE�{��a���f�S� ���$���������� i�a��b�N�&8��AuL��R���������#
� j)ep����O7�R��&�b]���^B����
��:��BD�d �67R����Pu�� �(x��=��1w�PPL�:M�� ��|���� V���XXuj7�m�/jV����H!h��_
�����Es�
j|Md_V-��� �\C+�IZy�:Doq%���D��V�Jj���hx5�o�ui����W�~�!�_%�� g�� �����r�s��|u'/��Y�n F6��Mr������c�K��RL���f'V�nug��]u[���GZ"��3o��Y&J�Pz�\i�:�V�>�LU�ou}���^+��Q����:��Z���E�&�����c���d]~k���h�;]E�n�V�
D��[���$EB�� �+;elk�}x��� ���~8T�Y�����K_.���4c|��b��v�����O�b\�I�,�
�*k�
�B�\��c���k��'P�^r�S�ex
�������!��
'�4><D��5����]+(J 9�%��'G���F�rM�� 9;�si_�K72��]�J��q#�������#�W�}�� V��v�>�y�������� �6�dU\1\�N�;�;�^�����z���o �W+���� ]�����V�Y��
CW����\�������u�f��o��]�?�]%�@��6;�Fk�&�>������� ���Kq��DWF�t�v����y�e�Y�d9��R����O�������/������� �b�q��Yz��~;�f����!0���B��f�|�D�O�����8���������Y�O�����?h~���8���HKqk��;�0�r�f�=<��p"X���2�a&� �r�E�i��=��W�!��( �D�" ���3��� �hdUZ� �����o��Y��!����rZ3W8:����WD����NZ��u,#�&d8��'T������r�{�q�[M�Ih��.�D�DT �3�;@|_n�9<8g���A`��p�P�fb�R-"�'Tia��mNP���M����n�]����������8 EP^4'@�d�t�!.���"��4�q ��d�e�=��j�����F�vUjt�3�t����������[���T�E[�q7���2�7xx!K.��a/��Z�J���Xn��~ `��a����W�$m�� � ���=��� S��:�� �9[H���e���,ib��������MLs�M����=�@*}����>�<QP���z6A������=]��7����P<�bb�G S�p"����JU�&u�����Q!4����Gw�J�W��([���X����O&�U�����g���llHF\ZL� ��i�M�t�mO� VE��:T��(���<��)�L�q��DXh_Q$ E���7L���]Om�~q{�����l7��q:d~���j �@��G G=�(���|X%+{�V�s��_���M� ��N[�l�� �� ^c�����k�R��1{O�����H��� �F������ ���N���U��*�#���^�o�\�������i9O������o��l}��O<���`��(�v�o�� ���_��%��=���n��4C��G L�#j��x���~����a��
n�������:�z��t�c-4����@�cx�7>��q��-=����V��6�u����H�A����>�I�G��V�;�����?�7_u�����r7�u��{�W�bS`�{���J e�#�z�)��;�Sl�*�o?M��^���t�d��8����e���>�,��{� V��n
�V�����J��"�,�D��/����� � ���[S����9���Q X\R:u0���z����O9� �����.�of�:>������I����Qk:��jD�nw2�m���Nx�MMmy b� �M_�U�B8@^<A~GF�>�� %��b���t�� �+=�p�".����Vq����s���a�:i4z�i�� Ws��m%��M}���@�l���x1[.�����f1GB)xm�� c�K��H�/q�)�W��{�U����D����@ �q��P��u�K-x0� ���[?��W��_�1��Q
��v���'�N��W����mN�z�����Mn�3'����4�� (:+X�H *�Tu����r��;R 3�:��B����� ~C�����r��{��A�Y��0\�9�!���r�=������f�����4��1io��
v���a���_f��t=�$�IZ�kX ������hU�L ��X��}����tl����$::'yX��*����N�W�x> ��1}p�E4�[c��_ �!Z.��� ��n�^���]���J����^D���� �b�� ,I}��>'+s��������+5�N���� #�W�R���"54���EP�AlK������VS��}H�FP���Y"��w��[��
?�}�����
��X�Qf�g~j���d��X��+Q��TM_5���.�P��e������b�]��5�HJb��L|X�}��{�#7������3RT �����f3��R�l��4'�F(����~����w-1�`T� �x����M�zK8l+ 0l%@�y�6A�-P�ZzZxHQ��� �?��_�:D/���5Wa�VS ��5fXqR�-a����dK�
��>��h)���AOM#�e� ��b_j6�\��@�a�y�n��:B�R'�)�\Ro�F F���}�1W��
�����;�n1�j7��fV�4��X6�Jb����C����3�i��^�b���/���5�� q��V0�'�j��%�?�c��G6���Y��b����B������ ��d� �r��m� ~� ����, �����5yj�4C5D�� ���]��h]j\*���o &]�r�i'� 7=�kq
�K�X��=|i�ue�������7 5�&�8t�����7r��6{m5�^(J����4�,���+�����j�����~s]����F�;�'��
7M5� 4T]�3�����B�I�n������GjC�G���|�V�|_���u6��Q����*@q�&��=R��i������%�?�k���S|���l)Q��� �?����� 7��d�g$��.Q�=H���L& m$%��$i_ ��A0|�q���;"Qt:=�����%�u������R/���q�L����4;/��&���0��(/cP��2��[� �X}8������� �J�����Ci �������N������r���N3���w�]1~~���<]�r/��8C���v��.��fo��r�a�o�<<����������9T<79���S!D��P�l =��=p�����������
;J&
;E�������x���B,�N�G?�> �Y:_�d�^gk0��j��9yj��6E/-�ZBm1S��o�Q�7�%�G�A�\{"e�
����P�@���>!��f���:�[J56�dc�@����I/w~������Q�cs��� [m��Rd ���'��@��gJ�:�g�t�(��Ulz
+0������h� z��� ���4���Y�4�X�l��P��,�� ���u��y�����NJ�d+��H����r~�� x��XA%}�����^���PS�T�E����, ��_�H�I��A
�����9V6��l�^��N= ?�s����Y��?�p+��I��T� �\��tk0�v3�c>�R�m�{�*�5�{�����H��N�5�dw��*n������L�l#u"�Sol�����-#A��[4
��2��aR��vS����<���:N
|��C6��u6� �r ��l���ML���C%�&�t�3��z�m�%����bJ<�����Z��:���T���%�Y:���f�<]o�� !��r���c9T��� �*�l�.D`�5#4*�cm*SD3��OB4
��� D��b��;����,\� ���A���/�,k�J!4R5�7�tC6��� A��XWG�%5�I� 1J�!���P����
3���xc��'�v ��*)���^�� ����>C<���!�D�Os���l}��Ew�h���nUC���� |����e��0Vq#(��AmiXV���4Q+�1��7��U�� ]#�S��D1�mt[f���e�n� KWo![w�C�h�-�)�I�krJ��C�'��a4Q��� p�W-H@���N/����~�������2 <�T+^���{K���6x �{A�`���� cH�UK�7�^� ��c �:1 Gv���U��na���.h=��}�I��[����'�t\�i= ��������k:M�y
e���<R�\���� �� {���d�Pt@:=��CZ}s�����l?�����F�RVk�U���D u��Z�~��E�7bH�^�(�OkY��>�@E����b��d�Bn�!���<'u� �� ��b#���Q�Wrx��@����{ `� ��uz\��6s�K�`�8���=�����9 �w��� �v�q�g|<�z�.,
FI_�\=��8]� ���qz�Uv�h��{���*��(����Cx����)���(�[���3�Ki�7G�2��E b��b �� i�^�,��:�(i���5\P�1�LK����sd��7��@\]����>,��S�aa��:�H��(���'OH ^Q����S��Y�P���l�&������� ����[����u�e�ndd+�;/+�z��� �����1��6��;��-z��X'jU��� _�(�am���d�/Ta;�H����,R~B�2�;�/�6z�wS��`Z>TM���Q5x���g�9������W���}���wG������������JZg�xw ��fu�U��w���_���#��l7� �V�r�����mh�v����s
u�s�[������0 ���"��>K��?��� �e��Y�\^�8��7
��/~�~��>�vo'<���Q<�4���x �u��d��w��\���:[D��� �+�6�fQ�A�X��a_q�*Z'�t)MV���Sz1�Y��|E�����)Rv�k�C�ui����b������h �h]�i����F/(R#�b��� CF��{V�syT��n4�j �7p;�
�q>S2��6}x�y��I�|��g�5��@y���p���k� N�8V
�{F�Q7����"�q�Pw�<m�����z��Sd���m������*W2�� �J��������C%�D��P����
��0�U�[��W�s�x8l���#����-S�K,\E�c�s�PG�z�]G����Q�����D��F� ���d�1�s�/i�^�'�G�Q<j����v�Zr
�' V� ~1��GZ�]�]�P����:=�L�P 4\M��jK��~z5k�� �t:�0#4`@F���~Y�g�SkR?� ��yr��B�C}� LO�O;��pW���� �� ����Q�Y����U+�$M��g tG�2?��
�2�$�jTp��Y�2�Q�8�`)2�J[�'�?�MbWG]Y�� ]�G�zY����^jY5'�_���M�����P������(�>����+p�e�RjK`���&�.T���T4�������xC������H���
4�1_1�B��l9�o������{g �W���q:hM�N����NF�6�������f>sAX��'���%:cB��>[�v/���o�� ����w��V?�G[DG�N�=V��=�6�/��� ��|��������|~��9]�c��%t8���)��*�!�
��y��W �]V����)q����B�l�^5�������rna�7��f5��Q�[6-���2��n�?D���g��|9��y�&���4Q�`�h�Z���fw�@�p&^��& x���?����N�O�N���4{�~�O�Vo0h�)��f�X��6��m|�3����JmH��z��h��u:�N����:�� �4�O�~��UO��a �N����3��u����
l��b�������A��W|���~s��St+{R���lW;%+���� ��4�*�-�l7�����U�>����3�����H]tF��P���|�P��L�P+���\=z��m���t����
5(�a��h��+ ������ aG��w���A�����GR ����f�$�� r�������4Y�C�rA����DOOO#�Z �>����J��'r���PL��i ���69^���G� ����*��?���������l���}p�!���}�������-�����u����~� ���L��� ��u��m'�k�Y��O��7d �m��<����?a)�0}��� 0k���7� �T1GO|o�g����^�������Ic������ �������{��\�;zx����CbO�?���V �,�U��B4�L*�n�L��5���}[p�� +hA��l���P���������o��~G�� 3g_��o�w��1�vr�)��a��|M����G�#����z2�k�h��-����@-�b������ :���d29 ��$���B��F���-2�.M��������y2W���W04��8�����[�>�?I��G��f��H���
�({t o��F��� �7�����?/)t�lkw0����Z�9�:2�F4�J:�����~ u���2��W�:M���q�!�W$-�
++~�&�qs�h� G�f�:k��qU�r���{X�y<�o���J�1����=��7��\�����0�7kFK}�c�
J1� ��2������i���t|�]P~9�6����'���72�r��p�8���i�����^���L�����'�w���L�n�D�-�v�N��������8i4F'�d0�Q6T�[ Z����PAM�8��,A8��� wGNf�yz>�~9Z?
i�W�c�0�c< �����������{X�'O���_�~�n��������Q�0r�����<;��h5�����E�t�N)5DE�(C����u����0D0���}���������J��p��+p �{'�
��q$�������z� 8[��Ui���&�������*.?XoR�A�L8��)�`�6��_�{�� � ���oW�l���6`4�&&%�n j-�<I��W���:3%��z��Y�:���@��b* ���Uj�:���f^5���N�Z`��m�x,���
7] r���j�
��V������ m?�D�'�_P�F�
CZ9�D�J|XfX��t�1�:-�����x��{�Ad:p ��.��G��?��&W� �>���0�`5��fz��K���"��� �dS�.�2�(%��AQV� eY�R��K-5���������� �����q�>U[=������;WG�<6
�(/��&M� � lE���eJ�!Qx�H6���� �!��� ��`����+t1J�Q?l��tY�Y}������B�ti����@,�����A�������~�y �$7���i����$��P4t�^C2�:F���� t�a1j*8i�Qj�ME�= yu|}��)�L�%H��J���}�)�v��qtH�I���� �����eh,*5/ @����]2�����
kP��w������I�?��q��yqy��x8�� ob�����<.'�.������33^m� ,�d�#�O������l���Y�������Fd9x�D����]��s��6����'�d@�H��t�L���c$1H6H�`�3z�I��n�qd�������A��M:��" 7u��_ci�a� s�M�X�?X�8��S'�=��=�FR�:@Me��u�Y� 6��D�^�fL�>$3"�
���q� f���A��� � ���T� 5u��� ��lvz�&��~.h���7�b��v6�S�,���$�S��f� W?=� 5�_M[�g���.�]���;��_���Su�����'�!%-������.M ����21���rA (&��� �d@�jI_�X]
Of\hp?l2��%�-` Ws�@��q4k��������]
+��e���?��j��2���[�Y ���v �QfK� R�6�"Bga*���Lc������GJ� 4��/�\3�MO*e,�W�.%4A�����}� ���Lgp6E@����x]������!�S��K>,K���Az�r�q����U�V�.A��LF���' :�8��������� }�<W�3xG��GZ��L M,=[�c�,����Z�,�m�� @H��#�L�V�����`/!�F|�a��Dwy,��G����/�a�.����d`LZ�����J���R��B����<�]�e�@�V���R����]�$^e�C�t��9e��-7[�"_d"�=lJZQ�H�bo�|P�-�/����fF���8��aV+^]d��|br�t�|G�����#��3d�
�G��� ��|b��J�B��?����G���0�h
>'����� ��`��oNTk����@ �*�%�%���w��X~VS�$z���d`���b/s����l b���d� |����Q�Uw��&�����8n-<dj�� �� M��v59_P���N(2�_9��� 2L�����f������D%�`-�2� -���� J������v@*j�����9��p��0��#�x
m����A.�-��E)��,|�:"�I$�&�a Y R��c ��������q�����G�o#q.p���ql��>���mo/�������7N��X�(��y0�� �rZ
Ye�X6�P|�7��d������vO����AHd�o��o�~��R88���R��#ZGZ��(n5]_�n��V{���eL�V�v�j����r������Vd����\_T�Sz�<�����8(y��xd"'�7��� ���D�R
m�� �3uc3�<�� 8@�� ���L�,����� �� 9���eC����`F��1��ZC��q��k�,�������������@\����<mH��� �7�&uA �V��H����6�����H����jl'u�(��N
��)�oJ cA�
�,���ia��#�, -7�(M>O}G+��3Cdk��d� �3]h^��}���{�c� nNz���|�[�)+��(/���4zbj!� h?��Sy�l6k� ��<� J�UQ�o����KWy7���p!['�|�(4 �c(t1�\}w" ��D�:�MLb�Z_$jm��Z����O�=! D��}� 6 ��L�}?g����X��� �0��0�BY���<�����dt���Ql.zi�O��y|v�5�U��C�A Q�����Z���f�d��/)
R� ����z�����\�Q?$sp,=�'y��"�����u����Um�8�L�8��|��x�Of+B�Z2�����<�� �t��� ��+r���d/ �t��� uze�dw������W$;V����l�a��D�)&�����>�^_} ��.����/�A����q��;�� Y�:V�6�P��Y���v?nu:-A�^��z����JI�$y m��G�� �M�T��.�� �J�~��l���koJ ����{9V���F�
���8�j9p2s� ����H1;�u J�J����.f�m��s�����f��!�0����+}��`DP�w,k��>o�Z��0�L���s�������R��*a;�>j�"�L��- ��������=Z�|�z "��tw��c��/�~�(��e�9�g (�I�2�?}Hfs�� 9�' j�Tk��fK�!M�{`j/�HV� ���5G ��g�v
�����&��r#z_��>x^^��4
��mv��,`���Qrmd�
N����gk�X%
��J��E
�86#�%�%R ����8.�'��A2����~���-}��s��=��qL�n��aF?j�Ob]�4H4��hf�������>E�K1�#.��X�r��PYES Y�k��O��!P���ln�P��I���6����u��]u<��C����z��~��N�uJ�I::��|��
,(�E�K�x,�8hT8��F�.HZ�!�[ )�J�n'�ct���FC����M%�HC��+� �[8��3�5��`��(�]&�K�� ����� ,^�YS��5h�k��-���������C�X�-���^���4���.��T � � #�>m-���a��xD�H��;(6{(p����V�7n��tr�>�i�^_���+h��n���(��M�����HD�/�$�D�$_i��E�����=.�i��N�r���N� k]�O/��HF�5eU/3W����}?Z�0����B�����!�
��A���di����%8��]�.�0�����P4� :��q�ipm �iT����(E+�IN��<�������[��� y#�;��)������4<&�� �������O�� Ug��b7�}DTrQ�����\�Iy !���\\�E�J� Uk[H� �6���%��5�D �%��Sz
K i[�� ���'��C��X �"!�]��At�����������"\'�A[_4�NNb�7Rbn���BN����BQRli�aX�����u������qd o�������n | ���|���|�v�W$+#�����-�1�|"-�a (SDK��W�~\Xoo�����oVJ.t�p��7��:�2N���J��P�7�r���cp���4YOB����&�I$��W�S*n>�R��E���~I$��@Ap@����?��y����l����1�U.B}�7��.���
.�5M�~�h w��'�.�k�����m������
�2t�4|�o � �}���FQ�qv�����)�*��d�3h�0��p�J;�������l�?���:/yz�Ix�#�a��9���\q��b�8|��C ��� ���#���E>N `�6l.�V���`4���Q�E�'@�;���q\1��<�>b��y���G)��8��4��J/���I9$z���+�*�.7.���D �}���>��w���:����7��^��3��Z.�� �I�� ���<;����q���F q���^j���f��D ���.~�H�c��8�.as cks��F!�Cqh ����_��;��qq�����N��)xK����?����Y�W�����&y�H� �W�������&������]�
�z���e:�W(B�
��E���,g��N�� �Qj\G�:?�yU�T��,�e�;Z�A7n��-}G?�} "�l���)�S���f�������x��c��t���r {�t�9&�B��A��/@�^����5o�T$^|U<����G�b
�Z'I���- n��a�*?q|�"�j��d���b�^�|*F.�,U}��
��h���E�������A� X �����j�5I��������t�@����&���K���Z����a~4����"] �k46I� �6�� ���a�?0�xR�(a��>kJ�Z�o� >K�U����~<���� ���3��v� �\��l #�� �4�����L��Ge�I��L�9S����/�9c4vq�
(���������������gO_y��Pb�&����,T��A�V����h�uz���������s���,-�� ����
=���v ����Wa����^��� z>��R%�jR��Q]7?�Wo�Rc��w�c�+X��(?T� �� ���v�� �&Z*���HgI�
��yZ��:+8P���b�������3k�ox��� ��F7��������"����hrI�=W.���b��}����1{0R 1un�f�v6��:Z��������b�a� [N�V�;���&��l?����x�W��4���"Y&�&R� ����Y��* � � ��I)��G y� ����� �H�������)H-+����|���Rg>�0&���b�v�>0�����e�X��Z��_���� ��M�SAO�i��}�o���@ ���&^�P !L$�D0��a
��Oi+��w�����u0������
�j�Gm/��z��c�Uy�?�|�1��~'>Q�Z�} � &��i�^(����H�D�w�@_���$����K1E@<������Mv�i�C�.�-Xxus�&�G@��%����GM� G�:G�p�y��\�)s]���2��m�/��N������ =�i������^��?A_� �f� �?�����v�O��<R��k����"�:w�D�~kM` NK����{ ����!&��"#�o���9n�\�� �A�:��J�~- �E���L/w��eD����O������J>��H�!�H�x-�Hvw���,���r��.�+����"�`��!t'h��&��' &�����w��M�9�{~��v���"��t*��(����� \� �k0�3���d�M-��+'M�#��A��oQ�4X-D0���s����� V FPR� �^��j�� :��b2� �������[0��"�u"5~��r�It�NSA�v�����2=��o��&������@�v�
F�o��7} @�Z]ix�������Pw������ �(!!����4�ejW�,v��.��/��0�����C�� |� �p���!}*��)������:t������Pd���u4 sC�����IL�������
��Y���r> (����m�'�����/���O��c�;�kk-��|X��
�����K3PaZ�Q�}������Z�T�
!ex���H���K�k
�����b� u7$�.D�j@�rT%*����~�����B�=R������T �\��+Xe�RJ�U� )?T+>m��3i���9�!�]Q ��tu5��E���@QmI$�$�$~yd)sME���R U�BW��,���v�KaqE�vI�k�0Z��Z��m�I(�*�_I&����""�Zf����� �]x�� �2����p;�V@3�8����\~u3n n%�����*�f���G�n�1�z��Pu��~wf��� +�w���-�������@�Ud� Q+ +f_$�e�t�;�A ��e�1,����Y�}�2P�M��p��}����HGh��p�: �=��YD#P�fWD�Q����f�1*��M���2��<}������O �����_������(�����������N.��>�k�q�c/o�����/�����1Z0T �������� ~������y���8�7�}~�J�!�/A�I�����/C=�_�}Wv�'�# ���� �
5�DP� ��U-<�D�t���|N2��:�
�^s���a� ����,N�:� �� �����
8"1��<�w��0���m����&� ���B� �=����%���
:�i�<� ������w���?h�l o���|���"����q��O��N#�l.>�fc���������V�b�-C�� �'|jf~l�c ��y^D�D.!C��<�o�+�� �bU��P�v�L�
��X0���7}� ^������h.�� /U\�c�O��+����c�� kr:Yi�,�R�=�O�� e���vA�*>*��5D�?��F!Qz����B��JZ�tXW�E�(J�� ��R�)V)DDFU�7�H��8.nu$��,�
�v��1E�G}�C�6�m�R�HQ�Lv ��L�_���� Zf0Rw�� ������"\�Er��� ?���S� }���t������}��S��~�j�����m �
"���������W���#��z
� P�S�Z!��5c�R>' C�N�TL7���5� �8�J��V�i:!�_�'��G'���p!��V�u /�]E��G,�']����v7��K��
]�G��EK%+����gY�� @���IJ`r�7:�;�N�����[~V���%���������'�[;�^��*�zD���Wcx[�� �����d���]O"��e������������b��9 ��D��=� ��@yZ�"���~�IV�>�qQ`��I"�+D�u>�O����n�RI�h"��uT�'�����~�E ���p�s�'�����D �����s(�x��������������fr`��eY��n��,DugP� |r�c�Z�����
���U�A,��������]}# ;�,��x�X�^LI�ev^ PzuH����������b M�v�[��Z���)~5,�{F� ��������IX3���S�g��]Z��QT���;��_h �>�/ U�R�{E�a`��ZuG�y���!�����V���rZ �wL=��a3,��O%mx�� �p����2����$������&��[��-6�}�6y:W�I��� ����,D�w���TVR���u��������s P"��VY 2�����X1�Y�K
�8���$0Q>�p���\��K-���gW�9�0�B~���2BAcy����l<u �*�x��o�S.s��o�����e* HW��{Q�\5i���L���PJ�` �d��Uu��i�m
�n��O���r��R���W��E<JZ�������^�:b������|~A2y�d� py��m��9�7�J�]G�}���+�E�7����C-L_U���[KE�-e����z;����~W�t }l�@������
��id"t
���Y�i<�2(��mO�� +7�0�/��y
�"���'S!��L������eJh�TC���rE��&Z���q���@m�w�s�f?�w�)����N��<$��H�z�**,(u�PnBJ��y
��*� �y#Z��j�.yyQ��8Z�1�!���!E�j'O��QB����Nt� ��4A �}\ ����������������(���������g9���|\h�� ���7d�#���J��|I�H yQ+������I�8z�^�*7� ���`����8�ikvdw��D��%��E�\�8�B<`�O����=5Q����z�pT$���qU���qUC:T����W��J7^�����Jv�M:h�vu���p5[����|W\q�[� !�0���s���[�z����<��X���T���T����A����=B^ Nce1u��P�~e�OZ:Bv���[����� ^T��H��yP��"U��v�������3r��;d�.�lx�FG� ���D�s�}IKI
ve�> j4�% G���>���(_��'������� �.�� Q���5��A��Q���a��� �zT�>e�����U`wx;���������X�p������\������z�d�� +b�� f�R (e�2� ����a�]�L��0gI�P{���o��E ��^�����R[�.$��o�W���@��fu����:�MimO�'8����|�m�|��T�<��')UU�u��%�7��H�R����%��L��h�Q?,������y��u�
��P���Z�oe��D>B�� ���w����u��F��v���:i��u�� {i����n-3��,NjH��Q}����-�����9">��*����
���z�J��
����z��_�u�-��eG�8���� �J��R�����-�@�����Q#���n�S�{Z�<UjO�ig������j��'f�E�\;�f��*�\���j��l��MV��jm����k�Uo� ���b�w��C�
��B�.��E
Ui�h Zqo0�\���5t�n�jf�:�E�_�s��t����`�����?m���X_l���x�Z������������vj)eJJ��R���5�
�B?����j'!T�j��Z!����K��0 �G�R�?O���<���P����z����?B��nK��;��*��*�*"�d�������r ����@l����b�f��B�����\�Y_�^T����j�u<�5�s� �hA�Q �#`�(s�����t9 �.O��%C(uR' ��j������O@
��QP����v����l� �!�*�W �U�ZcW d��(��G�Uk�A��L-S>���\s��+4��� ����>v�0�����w�����v3��������H�/!���-��0�@��b]kcIX@�3����� y�g�8�iPy�@�+�5��G �+������(z:W�\u�t�1@Z(���e�f� }H�X=��E{���/�����2S�nUi) �X�������7����/ ��2bE,����� �c/
vSb���-t�!e� )��85�9�>?�S�� � �JS2��^rC-��t'�?��������� �G7� ������2^�/+�����z����U�`��:h6�AsxyT�Yr%���j5 �}�Z����I2�Y|w�z?i��{�}�!g��N�{'�{�@@;��)��!e(�>� � ����v����l�]`�PxX�m��8a�G�d �e>��50W|�z��]J ����{�/X/[�b��Z-!>[R�N��S���������������t��'����p�d��Wg��|��G��B�]��^7JG,<����xjn����::&O.D�$�;m>6Za�-��o�.4�v����4�r
�3s8`�k�K� �@����@����vs{�w�p�F�����<��7]� ��C���� �I��j��
!/��c%�� �l� ��c������5Y8���w ��{�&�iM��r�R`��3���`��iC�r���0j�!�d���\���\C�q��T��� �d���lf�\pV����������-W�M:����2� �XG�����aUA.z��t�Y������� ����+���C�*��2����������2q�N�+��}��6s�T��9d���n�e���5�����>}�Y�u�&�������n�#������ ���|)$�/3����7�VB�������X#|�\���d����h` r�:�~��t�R��S�����dg}�r�������ZU��Fa@S�x�pD��a��1 ������Hg#W�
�/Og)F �Z��.��G.]?b� �V����E�a����@5���O��-��R
4b��E�d��9����%�� �J���N�� �^d��3�>��l�l��9� =��_��+
|��[@ ����U%��3���!����t���V9�+����
y�w+��k/!���N<l��xu}R�I��&�X]��1�:I�|� c ��>����q2y�4�G�j3[�w���>����e<���S���<��E��Z5��]����a��n}���> 00� ���d^K�@����0����J2�7 ��*<{��%��?�}������������-������~�� ^z�1l��~��l���������u���^O�a 3 �!W��'��km�N; v Mf��[�f���a��Sl��W/����T���5 �����]<�����F�h�.�� q{s����7���<���N6`D�������|���+ �/������� l�� mw�s�V�.��b�R�gv�o���� ���v/xR�:���ST�O��B�s���Y#m�A����c�&���mnT��� �K���>�]����Z��m�f��4�� ���~(��io����[Zw����Ti�a/J���b"
<��h�,T!�R�8������.8 ����G�U���W�<�d^ 0_���^�=����"4K�������V��]pU� ��f^���f< �m��������/^Bf4���Y>Vt�\Xc�\�w�^Zc����7bP���R
��?�A���
X:���q��P���$����'�T�"���C��u���J��k.�}�T��X�csq�8S���[�NG���m/� m�����������o��x�r���{��e�\�(�>�a�b/��=��~� ��R����T��\I �;��k"�j?=D��D�w[��k{��kx��x�}����F��_u$j b��u����1�
�� l)q���Z�]z�eyf��=y��D�_��E)���-e��6�G������
���d;o�_ ��T���.��� W�n;
DYp���P�����&<UD�lHo�0 P��b�V������8aL�X�*�i4�d��8����V����kKRkKku����8�&�����Rx?R�!,���82��#� �*�����Fr��D�:�� ��(���YT~���Y�k�U"���*
��p�\?�=���Vv�t����Z���-]��kl������z���������*��o�����t�.8��W�P�����g����,��������u�����El�����%�{�Z����FMq_�
�j�������=���SP���mGX�l����rb�)(
Q��
��"��vO(
/�� E�/��b� ���T$���<��q�n����u� y�.�w0�d 1 �Q]���@� ���z��� ~�6�mq��W/SC�: ��8i)f0���|'�hCaD[��"� �����agC�������� ��m| ���e"��,1m('����MdJb�%�/��E`B���S�s�����t�4pe`�K�Md�x�v�K8�����+��6�m/�}$j�j��ho8�^rD#�R���o�k� ��'>�c��f/n�[>p�mO�c�\�8���V�;P����V���L l�������.���kf�N����]k���,t |*�t����5G�0z����y�2�A�p����tNH ,h�����e������������B� �-� ����<f�6@��7R�h ����;j� P �*��*Y���Rx����o>���%{Q��V\o�s�'dZ�N�j�M�+|b�����%X��[�1`��G�DM���C���w=��ff��2=�e�����_f�{��O��l����Q�:h�:�l�o���������4�W�}���Zm��x��)�d�Nu7���
���/bN��A�=�{�"�B�$[]4�e��� ~a��f��<�!��d(a��YJmK5c�cF���%rgxN�A����~�������w1 �A[�-��������aTj� rU����U��Q�|�6���,��b��0���2�5ekW���P�����@�����Y�X.�Y8�#Cb���8�c,�;T�<xC�����'L������������� 9?#��q�SP��#b��2�"��&D���G���}��G���J�%�%���U���T��6��0'U�r�0�KEm.��%�����Nm���u7��9�#HB�+�) G���n� K���J^�&������%���X&��eb�(���S �X�E�����d\�RVD���a�x��5(�*%b�T��
�j�R.Z�$�S8G���D�\F�M�WX����6pE�g����1)e�$s� ��)PJ{ �� A�l�q�*�j�f!�����M�3%~���3��sp�X�3LS�����(��7�������,��SN8X�7bld.���k��zAw��h�
�`[���2M >�:r2����'��3C ����H��Wq���d6O�1R_��q�L6�x_�� �}y���5%i1@�Z{� uf/�l��F����)����#[Q�kfL�\^V���H��o*:`��y8[N���I��}�� B��!��a�QX�\@ ��t���ji� �W H)�d�n����:,��r��g���{"����L#��PS����� ��Z���h]R[C� �G�w ���0�(�Ng��d~H��4�Q��FE2qe.��������xS��<i� ���J���
?��O��� �K�k����5��l���#�g��@8���I�~�J� �u���Gl���'u/O3�y�r3��q�a����ag�4
Q*���x�����^=-�>{�Hy�e���,-�<�Ho�H�2M��^��,�a)��mn��3�C����/2�R���n��Sp�� f<Pm����W#O���._��+JI��p��h("$FMu�uB�?V��{��0�?�P�ek�O���]'�)>jcTm����' �� N`�5�,���\#n����&� ��J�}% T���q{������1�u��S��C3@(�g��[&�NM�KN���i71wFq,:c�T��\�N�Q�H�����@r�R��(���D�fG,��?�y����2�=����&
�G��$�x�S�V����&!F����D��:��z��� 6�)Y �f\\'��^K2t����i �[E�W��Oy�����R��d�\� �H�(��IY�{�����wywJ����P�+2��y&���Y��\ �� .������v9W7a�v�N �w ��$b�R]��i�(��j �K�����eI�b1����:��dg�N������t����k�N�t$A8�H�]o�"���1g'�+?�V�3�S5�6�E�v�%��� ���0��Y��g r*�}�>J?��en|(#�D���<M���
8_�����0;[$� pm�E��e/�)db$V�$���~z!v�S����N��u�B�<��j�I�w1������4�P��
���. � xQ���C;G������|���?�<���]X�nO��F���l�hW/r� �� 0��j���������5�/�c$���.��w;����'����m7a��$��l���-��qj�C����v����C7���6���� �^^J������G���V��D�����W]��ru�v�������K��
���0��I�<�"YQX�c�U���p�D D��)Z{!g�N���� �E��� ^�
���{�pi�!�J�r����������}`?�{����� % U�`���LO�"�z���4%ANT
h�f`�������y��gn� �<S �m�����$ ���� ������x����� ���������}�X
E�: ~�-B��.��I.!0KDf���b3��*j�D^%�dfQj������0����`�Ko����p�NY�N`��X����P������������ �7����n��;^���k�E�
z��cB�$�L���^[�q� 5� O��5�K�������c���uw��$f�/���f>�FI?[Nlf< X��� ��#���l1�'k����>����� ����������w���N0uYI��#n�-1ZBb�g��B���<�=:�IK:���7��:�W��a
���&���`�/�5�oH�b�W����u�r�(���(�P(��� ���L��A�d�fb�����tZ��LS���S��u���� ���>O�������&�4a#XG�K`���H:��>��6�u�{������d�H7.5������N�P&����;]zp�������VJOs��!�4 �$g�M��������a��w�p;���SOa�� �[wK6�=��;��4 �
�X����Q�vg��
����#1����m����uzX@���� ^/���M�?�� r �r� ��7@�����@�vp� ����{ s� ��W �C�@��~ �������� ^*��Z�d���O�2e����4�WV�;Sl|��<H#��ee���\� � zh�f��V0����_�( ���v���8���d��yY���N�B���Z�@��/Q����;h*�w ���v����V@�]*�����0���YQ� ��W�N�2�)]s��x3/MBfE�����6�>�aV� ���9�e N}Y�iZ$��gU�f��F� �F�jv��
\����V�>a $��lG�y/�4q�����P��k54�R;��b{<`�����=�j�=� ��m�����g)Ej�D��[�1T��-�hs
� �?,�Q�~��1����+�B�^�=��).�.c1+�%\vr"���'������*��I��s>�e���v�. ��-�zA����#S�� ���:"@q�2�B��j ,,�y�]N)��4�To�4�M� 2 �CT����6��.'�G���k������'�4��
�*U�=�����E9����| �R84���v�A
���_E �t����a_��TK4K����� �0�����b�^K��������^^+��Y�t�b�U/y^�����hG ��J�.����3��b |�����'_oR��v��i����(| W�/���F����Wi�%��%����H���b���h�5���
g��F{���)�h�� ����5��W�hk��D����w��>�Nzm4,l�|� �^��� b��M����<�� R|��Rm�d=5��?�4�o"��n ��B� ? �W���SED�; @ 9�t�N��g2a�Bw���&���MF2������e!�@\�g��{F8"52�v������pA�z�J� � ���!��B+-@` g �B�Nu��"B�B).�s^g��?�Q��h8�Y�#C2>�^ Q�����5�(�kn��������/��J������Y+�%�)Q ����}8���EJUh��D<�X2u4w�"�;��X��l>����b
�L���i��6$����*���`�Bl�/X�k�"��H��VW�����@
�n�)(T�Fa� �[Ygy��[���'��,;O?�k�J�� ��4�`R�n�>.����N���]���
u���]� Ee�����^�)1�����e�P� ? ,"������"�z�uY�yT��uu��� D�.�sy5� V�5���K��_�j�<��su��j���C��1��VE`e�C��.�����Ii(/��w���8P�������`^`�.wT.��@�(���i�d���� �R��B�D�B)&*�Bo�&%�*�7���%2���X���
DR��\�=����v�. �����P^O/���m!��z��]�P�pqiY����j��?��:C]�����o�.���l�J��%SW���(SJ��9�d��$��^�O���U1�,�(����H� *��E������Fj�t#!�.�
f:�a�3���4 >� �T�$ �Vl��(��� r��MC����/� �����������|�����
��: P�W ]p�>��\�V5���DR�t�df9���um&��4W�@g���pY�M���6=��&�+k=�n� 8����`��Rx�1����~�.7fA �����ms����_��I?�D ��|�m~)�5�����( l�x3�P�����P,�-�:��t�:�����eU�(�?@���`�;�8�:>3!���Cx� �tP4%5J��z[P��NL/��� ��l�������ko��N=>.���u����^�����Bs�H��R���\I���YR���E����A���������}�d,.�pP�����_��d���d��vCrWQr(�!4�����aQ���G�����4� �tP.z�&s���i�Z'� >�H��yK�!����[�))Hx�:�����:���&1�t��v�������uK��0����JH|����� �����E�G��E��)�� ��c����<���S��gk7t�?Z���'�@�}��� P�Ng�oL�:���U;�� ��@�8A.�"z 7���r����i:5��(O���� �|[z�t�@���>���8�@ .��� �j���%��~$���A:J���Ht
Q��+Rp��nu|���*���^����7���* �!��j` �)��#-Ws^@P��
�yi+�}��@��/1g#O��d#����;��~��� ��q�<�d��'�����EKd����~����(c��4���3��F�+E�=h�d�� uJ�:L�� P���O_�����;$Z�L �� �U�u�� �9v���.`+US_��N*�p
�k����_������ �_�������x��vn�+����M�H���\ ^��>�/���c#M��J�����~�*"�?we��W9����]��Sn�!Wu�yM1"��M��`��w�����M^�Ci����\�t����������� ����� vu[�Sv��+������SH�}��'�����t9}��6��:���Vj�>���kc �myg��G'35��@�j6��� �<�z�Q�{�n4��xx<������Q �UEM~�:��gu�m�d��IpP�� ����~�3R��D}f�R�)�Z �Wk�����R�*C���G�'���}U��v$� �P�M6��7�B}p�]��)f8?����:;����m����!u��!jtkw `S�~C�:���#� ��d����'y�U���r�})����:=�J�{����F�� ������R���:���^`WW %t=%�g\z �
S�ap��� � &��
� W��������;#��jEn;�G52���$+,���c��@�����*F���2�pc�/ ����}�!�X���7� ��:U ��3XU T�u�a�o� �0�P-��:����RE���
�����K�� ��7�Ix�[��]A'%�&�x�����I ]�� ��u�,����]"�4����i�� 7�Ao0�Lw�W��N�j��[w�N=y��-�oV[� �8�v'n[t �ip�G|W�*�t�Z�qc��������^c�k>FnpB<�}���Q�x� ~s�|P �_�d~���
��j0��H{$�)>���p�\ \F
�vU :5��I�h��O���q=�~k�(��yT�p`&f �D?:^��x$8TI?Z������~��^Ddr��;b!������?h��K������S��_����;%{jY����U��d_m��F�r�������O�
��8�2,�y;YN�VD
n�,�r;��o�Z���n�J��.��) ��sq;����!���m |��c6����p&��� �B6x�h��!Y�{8y�^���
�����)�Bu$
6�(io��'��d29vk y�m-yBM�%f��r����_Wc����4��w����U��i��; ������Z��d���W`\����5e��81V����h�8�����<RS�7� < ��pHm@���#���O?�����
$y<CV|���<z �BP^����M_mx�!�@�����l0�F;\��=��j�qH���;�e=��7�����4 /zPHH-dtfq+lu����<H��BQ����e��]����1�d������,��y]�;����D��G&�>���h?��\4�0�)���+m}q�9��SF��������eQM�:d�G�
"Tm�s%)����z{�o�S=@G��j�3%����L<���~����%C��D��N��=(��s�~��
F7��hL��q�J'�"]' �A�������;-5��b};�=TI��-�Q C�e�Q�2�8A��� m�����d��l�P�����c��{�S�h�,Q��O��O
�(�t�[k�U���F� 5�/�M���5�
h����q�;4�P�s��]�0�s�8������9����VI�)f��O��tG`���1) 3 �&�9u��T� ��V�6��_��WR��
�qe ��!�V�;M��pt<����](�b����K0
a� 2�b�+taf���X] �����4VPb����;��~�;p����d�%�v0�"��
,��c���l�#L��d��^�4�
Z��-����\%P���xM�O=�;��o��_Fm��|�Rx��h�5���BA�Cl�+u���<��
E�����l�������_��|���}������o����!���Ae'?���"���O��&SL{4C[��9���)�|x|q�NO��:Q�"fK�O�lh�X>���v�>� >�h ������������q����fTk��)���1��3���ac��4���.�j�k���ze��>���l>�\p_�Z��� �N�|?G L�y���qG��$[/���n��f9�N>������)����,�35L���4��n��~o���#f<�-�&�����6V�#e�����`f ��DFS�&t��G��;�e�����j��E����s`,#���%0+E�)!=RB��
_cj��P��Z���k[�P� ��F,���+�o �~���(�0F�� �#O�H5 ���M�Q��#�B�E��?�B�ER,�7������ �X��744z���|�+5��Y������H�����'c8��a|�7O���s4�;����w��v��m�`Q4�|���9Z�XT'�$�$��E�f��� � ������Ud�����-�*��?���t������1�#����B��c5< 0��8������'8�P�u��%����z� ��"������C'��:��M� ��S��� Pc6b�R�C���Vx��s6y����(�&0J�a��}QI <� =N1��0�u��;���df�Bu�~�S��*;<�����B L2$������������8-�� �.��AwP�����I�s���-8#�mX�����]�-^����#R+��4���%�X���rT��w��{$6�znpP0�D�������W����:P�A����|���Go^�x7~���� =���iS���Iq�8o;�����Fu�/^�1}�\���� 9/H�� ������h5��6�,�$�Xn
oB��;���hs�����+b���{��e�� �{V��p�;�,a����f=Rj�����?�H���|��\�NK�V��GZN��0W���]^-Z���SLtr����vqH�,������R c���� �k�I (��}u����'��YL���*� [j��1
]$������ f����1����)JE9�A��^��=-e��X�� ������|�<�����y���6t-O� ��� U�����88U0��XRGf�����j�<A�`<O������Z��fFU�E����.�-o�)D��6� ����Lb�"�IM=BP4��
����@<U�u��;�� �"�T���@&��h�r��l<X���2$SQ(����*D8���F��&���=s�B��7�_���Z��}����]�3�h5�A�q��8�
My�.����-'H -f1D�(��p����6/8�tO���"�$���`i7��� �� }�s�����+3���?�s[�i}�$��iA�>����Y��)����{�� ?
;��G��h�N �� 7n�>�>J6���
��9�� ����{R5 'gE
C=��Pfb�>]FChG2E�D�?O?`
�_�Mea�R�NR#��h�>6 `p�������8�������=��:yN �����I�mO��t�x� {FI��{�X|��r�1�����9F0��s��RK���M�{�K����} ������w������y�����N�a�����^����s��w�S����4X���� H�a;p�9�E�����*�?"���xj�EsKq��x m,���^h hf1�A �;�)1f� ,l� ��w���~V�N���{�-d���4�l��
;��x�}��a0�U���c�q�����[J� ����UG����f�"8-�X���{M�
�n��^�y���z����F|�� �+�"����+JS����g{�D�*^IuO�
E��I?������w?b$�����!�����B�D���j
J;����g� _���X3 �O$j�g���:��%� =%B7\��\�� b6���-������~%�)@��-`5�fpI�e����rm��??O.��.I�bS�~�@����������aB�45,����d��� �4. A��!=���x8m�7��~[� i�:k�d�0��l��*Vn/�pwR��F��FS,JHw�7L���G11��[=7�#��^L�/�#([�~���(w��5��N��Y������5��(�$�]�@�V�J �cp ��D���)4-�DAV�=e�pD����=Z�?6g�\�3��Y�85V�T�a (n+[��+�n�\�]�B�t���g �*J]d���:�" Fo��mc��@?.XJ�� ���]`�} �s%�4� �u������� l
2{�e$$�U�Q��� 'd@ �����O����C���4�,�D|�VS=���:^dp3���"�t=�=0E�/&�)V�� )�|*R%�� �o��f��`<}Fl4����6��6����C,��F(^'�����t��[����l�z�����!�=�����>Ji����5=1� ����Y� �G0�+\SI��}� �Y���':�T��������J�?ao�����Zv��5�:�k�;_e�}���������W�J"ok*q5�����W��\���RQ�l��X�
���s !�~���d�P3.�h�T5*�
����RP9�/W�;�ixi��G-��R�G�O
<��H`/I���[=j����������<5{��q�����,o7� �A�!���0
6�"@Yq�Y��=��t����5���N���.���T���}����5�^(a��+9�� ���� 70O?6L�� �Z��-�@1r/� �_��MFa��Cm�qG�]��U+���;�Jr��;.1��.)4 1�i_�F[ �u����? 8���~�!b��
���G8:M��!
�V��b��z�z99���t��^r
%�����sv��f���Y�\�����������wI���g��9_�n�hV��]�����|P<����~�CBs�����'u�'W��� �-�X�Dmu��b�f>jq���t�����}�
���Sx:9UWm�D�6w��z�e�s�,�)��ad���r���&ps����e,��YR�ew[M�3����\��Q�������s�x��xn][P��*�����K\@"�&,�W4����K4���������yY�A ����O���R
�N�%WZQg�H T"��Fw�C����x{iqj����8��n5��| �8���B������q0^�#�Ql��u/N�
���{qv��}��XoY�VG�!�I�@� �c��$������������ B��
�a�Ze�G���W��Ht���O_ =��B
��Q�8�R���U@h�
��2A\&��Y- T-;�e�(;�P�d�d�D\�����w6Er�J� P� ����~us�0�#vj�� �� wnv���9���� $ {�t���> W�� tc��H\��=���mt+�h{����� V|�P8�Q�����S�B� )���������� ���
��� �,{�7 �&��zx��BA� ��; W�{�8G��Ts��� O�B�&������_}�d�26�����z���{�t��M��}�J��p ���5/YZq�p�o ��)� z��JqBn|�Y�zQB �u0B�Q�4~|��-����e)s�9M8.�� r�A� 7;���^'L*%c/�<��ky g�%
>!
I<�R�D��+Q�������K����_*������� ��5�R�w�����~@Jr?W �)��po�
�)����������>uf\8k(tE���6��<�k8�G�< ���W����1�\&���cz����1}T�����1��w����^����#v8N�t"�u�'N�(����~�H W ��P ��Y��1:��b��:>��Ilbs���*R����D#qLxA�1^���q�WN_y��'�C�����`�}�W ����8u��8�+�tlz%����K �UG>��� ���5���NN�2����\vT}�<�womW_�� X5����E���|�q���2�6�o���0]���������a��?�����a�I�����C�WK�z���+���W�,�7}�*`�����v�8�Q
� l�.q�O�I5��� �[���K����������Tn"�y �Z�������� ���g`�]|����
V�C��Q� �?Fn�g<������5�w8T�;����:9B��
)v��@&�U�6���x���$�c0<%�)��� �m>��'�+�5[��cp� �%`��q� �.���>(A��U�7{���tkC|� ��=��^L����*�YU����d������5l��Tx�����b���
u?�� �V�N6 �w����k�"
�� =�C�����z~�h6l���fi��<�c|���8���5W�Pb�-�w �Fit�����N��
�b���?���-+����U�� I��_u>�T�x5C���U�� ��5�~x���/�{�|������=�����=���8�5Q�Y� !G(��h\X3��DR-*?�J���3�[����a�6m-&�{dT��g�}��d� [H)�����wL���b����g��zI����������b�>�A��Tz�"��l� b�BiQ���� �� �
k�rB�p �5 I-�Q�#��]KA�)&dL�p(��� aA��;Por��9B�A��4�/���`�M� ����^������Gd ���.�hP@{QL���������v�p��k�#�2%���Q�G� g�f���s <�����N� ��DD� � �l10$D ._Q��,��0O5� � �qk�o��K:���� ��_���?R\�n��������� �'����C��s���fu1H���oJ��N���"����,�V����I�$Pd]X���N�7>#4�%�tN\����} ��� k�D��W���`Y[ De����.UK���R�M���9E�=n�����(��N�uaD���&T�]�{`1���-��_�s�� aZM���1�5]��� ����L����V�p����G�~����Cd��H����G��� #�!vp���3�iI���/ �������� R,r_����T 4��8*L��]�JA�X\���A]`� ��Q�ku�v�������P �w�7yP���O�F&!���Z�r�e�Y�t��~����w/�X5C��0'�ur���E��R�F���D(�i�)@BFl�)%���N�C�)���XB.J�{X�k+5� TBe�W�8Ao����{2��
r�����&Q���x@�����nw{q��u>��&T>��C�7�XW ���v(� l������o��~G��(g�� )T� ��;� ^1�{�>�[hG���F"[h�N�g@U�� �����Q��Tv� �� o��u��jI �-8H��"� ,��oj%��W�;�8�����:���B���4YO�����^�����PjJ0���=
9���/��� ���[����C��*W�1�^�T�z\�����|p�4�_�{���ui�*�$>b� v��R�������!Y�� ������4�x?�M t/�� [ �f+V��#��^f�"�L�� �:�j��)���+5m����� ���w� �T$ �� ��\}6�b���������� fu�W�
�]�'�$t�� ��� �S�;j9�� ��9�K��#q
��v3�?3!t]a���U������jM�t� �NF��?��*����������W��*������V
���'�a�qv����M_��
�>�+��Z������%��&��{C��HM2�SWI�dI�8E�-dC��*��K����ND������ ��?�������2�j��.����u]e��z�G�� �������x�����Q��l�*����BE�9����Tl!Xyw���0y8�&�C����/��5�7O�������$�x~D��a�3���;=~�� ��q�x��9+d ~�'��T��� v�w{��F���I@c5m�����28d�������G}���VL�J@wsD�[V����*1U`#
+ ���w�f�� �������\��7�� '����g�������Uw���h/��_"����4��@�E���Y��� \_<�C*Rz�W� w��ln���}�)J&S�GX�7�=�\t�&A���".���`����}���+�� �c@�������*O�#*�� �j�D�Q ��u��K]�Iw�78 ���(HQ��?^�x5>z�����\�ze�G�K�B8�3�*��37V�;����_��Z=�P�P����>�>�u*��d>A-m�� �>�\ y:�[(;������X�!|W�_C�S���+j�����������gJPnc�� �������t��zN -�������}M1(q���O#|
D�)8�>�����/��L (���RhsW}��`!�aFoQ�������"�8OjZZ��P��h$��:@k��\�/;1|����?/�]f�M���#�^T�_���������'j��O�f��� �B��/H�]��W�M� �������=c�N��I��n����0����}, �y��� �(���*. C�DS� I���_�^ �����)���tXnw����~,�6Y��E��
��*���ix^�N��'�.���l�mi;��������m`������C �`�Z�+��
�EAW��m��6~����!�)-���������]6 ;���s�aI�7�@���;p�>�h_��~������M����j��CWH��������Z�Z��};�}����P_����J�n�;��T'zb�k*{�M������-{
M�~U�5Z{��� ���k(o�;F i� LV�.�����{@�>�~xS��
�^kd���z����/�#��?�� ��Z�:7��^v�j)�_�:~!���N��F�\7@�$D�Fr2� ^�K�d�`Ith���2�D�U!1��� ��M�M��j�o
v�d� ��N��{c��}�����o�'�M�3�L��U� ��B�������2�O
�9����(2��t���x�"����Lu1�����%�;�f`�� +06��V��<�^>�H�HN�v�D�$
��.��t�2���T?�m�~���'������`���:���K�A ���
�b�_t�(�.hP���&�L��� ������aq]���b������6��:�L�=P6!_8�=�hN
V���������{z����v�*�Y���g��{�� �����>��`���o�IC)���Nqt5 X<r��]��v��X�L�1���(6I_����]��r�2t�j�f)�3-����7Y�m�i����v �5�qEP�7VA� ���%�t��Z����(h���p3�X��%���.dx��� �����0�����`��-\�2I )��5%~N�/ ��f �*���~[��/o�5�� �N ��T��)�QD�����,�<9Ia�c���|T 1p���`�Ci����z?zy� k�%����/�b z��=�������vD�����i:�G ��TT# �O�SO8�h;�� ��P�|�~�M�q1�{:��9 �G��^#�?�TY�&�����h3�b ����C�� r�Z�0��
�iler�,S��vm�~�����{�N��j�]�=�� z< #�S��}��D4 �q9=��Z3
Z��c:a �l����m����� V2N6f`�d�5��rK��a��y&�)��������B4�xYS����X�W�9|fN+�F��d4x�&�C�o<�H������o��5 ��B^� �G��$� {�.�B�`b�&�;lRAs)�@{ �@����^%g� ,pz�SI��NC����
�t���Lq���� �Rb
��~_@����K�R������<�2����$�� �{��Z�6=m��CW�:ad6�)����8s��A���SE^� &�zl~��V�
c�M;��
3f@H�
:&�������*L3����u��p�"��
x���^ ����jd���[1=�-p`�@��3m��\��v\��z�w������819��$��� ~���C��R���t���[�us������Oo3��W��B� �����^����&�\���LI%\�F$�MJ���,J���i�w�,H�������� 5�'��4vi�����4��s�8J7�_�GM5���K��}W6 8B�J|���O�T8��EI���Bd�f��� *={\H�-� �' �s���*v%�������!/��Zn���/9s<�K �I��], ����"�(W��#�� ��Z��
�sJ�u�W�r����*u����-�+0����/�s��ZS.��<eG�VCK/� FK"�
]����S��&����u��*=����f� ����/��l�]�=������� )����AC�N�z�� 6��HmHd��� �B�k�`�� �f���Cw`��n����
��*/��c�/��>/�&@+���g�N����yz����_�q��$8V�ks'��Z 5_}/�m�G�T~M0�X�Q5l�f���R �J�K��@����>��kk �FwE�%�@�2 x�(F'w��S�j"����L(�����@����h%��$��s���c�����J�#�A��/I���a���f2�*����s�!
"%���/����>�����T��~���k�w�p�FO��%)�e}yCm�S<��-e8N��}q
� ������N�J�(��F���;�h ]@&��nw0^��
?���u
�d8�����@�7uE~�7����X�L&|
����Q8�:6�����Ju��T�;�����/)���<�`�M�xY�F�V4���Lp��K7 �?:�!<�M� �V�W����H?et*X[����c��}�{[m���iv�lD?�� o
\r������?�a����9������1�lu1���eO�V'�X�2 \����T�M��,,��m���� �CS�j�)�e ��H �T}�|l�3{���3� �����#�L `n�'�����kO�������\�w�����({^��� ���XofS^�a#>%A�diJ��"zX��(������#��1=�n�:����fU��Vg8H��A? M� �d����z��s����wd���g�2^ �IR������[�|�p�;����2�} U� �g�@��� ^)��/H�
�P��������������f�9�F{Ps��v �NgP�d N�u2 �)�U_�u����1���W�i��R�����=\N��Mj���&�~'�����7��9�����l
3��z��"���U�+��;��f����l� ��r����HF�g
[ ���Q���&g�V��7�h�*��b�<��Uo�.��A
T�}�xME�2�a_3K����1d��������j��u�q�+�2z�y`�1c*:���!^*K<�rqTB!3�*��C2V^+�$�sta g�1:X� + �
C6/���6��gj���D�a������4;��$1 R��Ri>�$H�4P�w�g��@-r�1�����=�����S�p
Q>(e�E `.%e1���X�&���E';��o�Lof�A����0H� ��))�
T�l�f��'jY !���'P�� b�(��{C�0�hm�f�d�I���� �+7���9�Nz�Q��o�{i��}��Nw������^lT.��A��Ar6)�,����+��H{�8D`����t����+5CG��E����%�� �4b�YV��^R���� � -W��A��=!m��DP����+c������^�
��b���W� ����mHJ���5� W�-Z����r 8�M�#���N�U;d�O�!s�� =�����D -������4�*Z!�%r���-&�!���������V����MS5�y�8���GpG�',i��<�*5������uc�d|��r=0 �w�H/Z�������>�/�w��X�2 �������l�d�w��:���2��B�8������� =�����Hx���������%�X�r�P�Oc
��� �$6?Pl���#���2'
���4�S�s ����� ���'`� ����� Gt�~ ���E4�v�C��y��<�z����# ����8���{���IP
R+k@)a��P':�����~�!.�83�=����2 �C�Y�D;� ��QGi��#��MN�����TD p\i�����0�1Emdl�p2Km5br:���]]
�"i�Md��M�g�������,�t�v^���H� (�������}�Jo�� �� {��G�;t��ur ��T��wH��1�.X�@���"�����<4s0d#b5����MmR�6��}��6�#�0k�<.Z#��.;3��y^�_����BF��9�i=9�P���Y}I��,���!�] ��'�_�A�9��[g!�ch�t�
K� ��p����2����lQ���t���S�!�D�9�D-��BE�����B
|<�'XX���f\
oc�w�^�~���=�7�j�#Z��_��/���AeK0~<0� X���5��������+��l�R�i�V�nZ�����w1��a' Eb�p��G������%�xR�%��=}lr�@k79���i((G]Y�,����nw��0U�W�!��!B
����P��6��i�m���J�q���D���)�j�g��@�|����|b9���_*��z)_��G+�s�h(R��/z������K����.���6V�TO1C�a���q;JE�� �c�}C�Z)�:�]����*I������=��A=�V��T�B �uQ��nG�����,{���Q
���
��/O��(�"(�xI �S,�Q|[R��;�H�����!^��S��=�Vby�RjbS3���}K�r;H�����
OkiK?��B��G(��Fw ��P��O�����>�Z$��r�{�iF �P�� ��3�<-�$�/ �n ����U0$�T����(� �i�|� �H�>�f�p�� ��0sp��)�A �N�~��?�����z�C'a��\�I#Mkd�8����s|��XM���wc�@_����.�_�� ��d�LK����fY���y�����6�-Q�boo l"�R�~����������N��W��TQ��^aLM1�`�����~�� 2'��5'w!���kZ���cD>R�����s�� �C9]�����V��Z-�-LK�������U�,����z�!��&����n�"�N�9Ity3sG���,����s���z����"���
��XwE��*Rtzf�� )�� ��i����(Qr����������� ���8/��S������������Z�8��t��S�6�tv �f��m4����y�V R���3J4��H�{���
�*z���v�F�.��Z�G��1Fk� �D�M �f��8�-�l���H���0
���"� ,�����Y61�'�tQ� ��l��H���;������'�]A�j�Asw-� W?
�O�\�������h9������ � *�a��JD�l�� _lR��Kl�*w�O�����? ���I�fOu98B��]:��\�)��`�A��"� c�� �
�����G���/��= �-f�l�d8 2�� �~&f��{(�����!n c;wi|�&;[�>��L)��!,4�]?*D�C�".��R���o�.`���'������k�h�*B�-�J u[�rO �d� �)."ZV_�m
�s��I(l�z��F� ���[E�
�p����C��3*�z��M �������?������7��8X����V�z����'~J�i���'��6�qP�x� _ �T�� ����:}��
�������������Ch�pi \{��U�3�<7��1������f/� �m)Q|2���#�SD�9��y����I2���|�a��;V�|6����������N�| �H�� ��� 00MO�/������BC
�#>����������}�T���Hv�� ^���)�G�x8[ M�9�z���#S�o�.� DnB�1� ��%��|4<��A�fg|��=�:�;�N�x�N��^;����]��=�K���[5� z�i�m��VW�#��nS �y�b�� ������#l�C�LV �@/9�������+�Fh���&��� ��V�#� O�(�����~#�XH�B;�0x}�`xH�&r1m��<�VL��� 7�{��)N�OiqF��� ��s��3�����a�e!��Etgn@��� ��`�����#Z���� �m���8C�{�=��]>��hl���m_
+��4������;)S��|/����i��jR�'����h�n4��N�dp���.��2���D��I!{]c�?(A���� �p?�We<����_����l�� ��Z���&����� �?�� {�%l��C,�;PY��������ucp ot�5������?* J8d�W�Cz�v� �F�v)�C�~BX��bT�Z]p: �V:Mc2q�������(\���H������%{��`���� ��vS$��F���rjS��A�� A�e�������K''�dt�a��!!�`����:�[��p�A<j��������
�Y��
�u���*5��|���W�����?n����d�:9��T���y^����Lc�
� ;�j�Q�J @�����oJ��u�K���C9k%�h�\����������q��z�)���"'p����p9GgY%�� 8[�bx�==]����Yr�T�Sq8��T�F��Wp���<�m�����HU��M_"�������(��,@/ xU��^�w���tR�O���W�=���N/�d����P~�����x�����*���p����#*Y�q���
�;���Ve3[R! � ���Cd[NpI��/�]q���N5]���T�R-�D�\q�N�5^x ��:�[������`����� ���K��)�T� l��2�i�kwG�F��&��L�+���0�m*��|��q��f��Op%i���A������sa
�^��#���������G���5�$����v�R0��'9VK�� %f��2v8�'�U�� 69�}���h����<;�Q�Z ���ls��V_`T
���R�����e�'���RP7u H8e,�I
c� �R�����3�������~�?�6��t�j�<_�����&U _(��o��r�[8X�0��>�(�
�y����G]��b ��Q0�0�\D���u�ky ��O��D^��)[�7-]��sS?6a������-�@H6�]9��l9;��6`�B���H��n�;�� ^���@��?t �I~#�u�h�Z���fo|2=o �������D>��W����lu�juZ�^��k�{j�z���O�oW��e��*~^�����>��:�hp�OZ�f{4h�:�F��lw[�^��z�������i���E�tR������������o`����5����--��o�������O���I����]��L�x� �/�?Gs=r=;~��� �{V<��[�����k6������Yo��������������z����W��~x=�sk_7W��|6M�bJu��Vk��}{%'>���� o��n~P���t�nKG�.���2n�����a�t�����p��|SrC� ��\���G�'�����\�7��I�������=�w���v�����<*��������c6 �7���,7d��A���; ����7� ���-l 5�����s�B��
F/3 dY� ���,�5"(���X#F� �����N��� �!�l�p���.��@��� ����T�=�K�����O����E]H]A�� 6Sg8�{Mc6�m�����R�'!lZ{s�D�\Esu4303)"1 ���/&���1�O�z�F�ZB��$�B�e���_|C�^kt���q����!o�h����Slp�����h��{{��m���E��^[�����<����&_� �sE�h��ZK�3@p=m�1��B������E�tb�?�'|���� hu�:��$D|3�jt<[&� ��Fh[�=Ic�����m�}�X� �M���"��t����r��R�Ti��aor�:n4��^:N<���CW�.i�������Z�G��6P��8��� �:�~p,�|��lvD�
[�
��"yAS�q<�m!�1 ��� �7i���Yu�2&�O~��F����� m�.G ��}�E�V��(���I ��w�R�C�m��6� �w'��������������k Q � � ����A�����<q� �:w���1}���e����l�^����X4�(��]��^��%C�=���QW���wY�������\���"�Y���RM&4���� �k����4 <����:]vn��L,�'��"xc���U ���8z�u�^�`�#��h�����@"/L�Y���w��������������iw�j-B��?�Z�g�.Gr�^-�a���<
%� p��[��.W�nA/'u�z:O�J$$pgv���3���� � �RE�����O�a\��C�$�P�H w���44g5��h�E�=���c�g��arO,�M�'���� L �Y��Qg��~� dj� [Q���W��{{���������h�
����l���<F�B�6� �y�������/�5�%�f!T�.�`[ B�5�6��!��H�"�y��jX
���"Z�;w��BHG/~~�����|�w��C�}
�S!\"C�g���|#c��x�t�H�����7/�7�:������C�$����������" qQAv��a��v� '�i��r���}*K �����'�1�
�q`^��R4.3QyS�������QE�n�Q����"���� o�y��Qy�[���F�n�S\e������^�� �V��1BN��
GKJ=�az�����p��Z]���Q���l�����1��QS�� n��>��,�xW�x��� ��!�pH�_����!(��������i�h����/�Ce���W��#l����X �����}n |"�������� ��dc�47=�k��R
G��8T������� ������� ��^�/����*iQ���#,� �X*�����A����/|L����%�*�VZ��* }�R���������F�+\�])$��l ��(�sK����GZ}$���Ea�7��n�SBj��� @�����8�����v(p�����,f�M:� )}���PZ�}���um�{�����cSXw �.��� ��/�F� Q�,�]uG\��kH"^�L�a���$���O"&F�s��U6%<��C���g��%��3
{�/�;�|�����%��}�,"t^�sD�����c���8KuOF�|DX�����6�����2�� .�<��7#6o��T�#t�����8�������(�Q���E��,
�T2�_����G�B6�5��� � JoX�� }�����7/ +�e���W�_ ���A1r�%��+������[�l�9y�4l�H �FE��h���� ��)������"}�bj������M��NS���u|���i�7=�j�(� �F��N��JXM���I�"D6�u�!�0��
[�� ���|/�E�5�2LFm���t���i�;y>5P��d{���g �
������>c�ar�z42^v0M)��D�
P� �<�����9���
%���8%h: J �um���d��M���jT9��
� |Ip����:t���'Pp*L-s�ce�rE��7����wm ����P�z����hsu��X�1��-@� 9* b10RP���/�aw��#�� ���������P !g?����z�,�oAyw<����+���f���)��<;K������
A�����Zc�'��1+z�J4B�4��_('
Y�w
����Z� m^���|���*�� ��L������7L��,g_s.�W������ �jC�@�%�5���' ��}� B���a�W9��pMI�l[��o����!� ���:��u�r U���C���KT��Gq�;� #
�^!"��~
H���� ����x
(Y� 4�<�U��R�����}��`k ���\���E��nk� U���6���j��.�tcS*Y�Px��[hM�A� ��$���Z$[��
��D �PJ9�M��/
��d��6�T���Kd3.�'EG�4����-9��}�??}����/�E���F$�w �IIZM�|30� � ���R��'Kp�<�$ZE6�qxk����P���G�^�z����`8��C��w����*�O�%�lU?|�\jb D�����?�A�pU��<{��N2n� U�u
�p���,G=_��F �M3�xM38��{������ +��-����T_#�1c)��B?wES���,�C�v��n�����R�_��U��Wp� �n��G��� PSY;]���u%����HN�\��&,p,J
� �A�����0��lI<gmJ�M������`�1 ��-�����n��� 9Q
��anq4����R�
]�OZ�� Y�<��8��7fN� �� ���K ,���|3���Qv�k!���R����/�t� \=��%�w� �5�\��4����*��7��,����2j��Be�s��i��&�JY��u*}�����
:�? �����f�|�B�?�"�'���5���'�O
��� y��kg;���� Wt'!LG�b����v�"�B��w;����d#K���Q��m�hq= +E<=w�K�"S� 9#l�����[E��*
-'�e�&�������� ����(3������T����&>pN�{<�{"�������k�S�m������QK�4�P�{ �������8X2W��>��0n�I(/2�
\m�>��5?q��<��9J�Z���PjN�Dm:���8���J��9�(�%����8K���|\K���.$w3v'���!�&���h�S�\��l�3�["K)�@��e�[�o�&jo�(4�<�B*��D�]yw'f���������(��)�� �):�R}G��F����i�~w��N~;��I{�� t�H23 %��+�'KC0��
dI����W���}���������9}�5N��-��!�����L�f�k� �7i�{�*��S<-Y�`�O�����l�����
zn����zq<��m������8��9���J1��� �OD �0;���jD��l����X��(V0)��j�fow�C[T��#��b����n�ja ��>�k�������..�[� ��� q \�����P��hK�^>7U�a�CJ\
�w$��CN����0�[P .
��g�����\���f�D��W��_���x`�/�*We������ �/S��<���� �u.�1��b��U�������Qo*���]�`T�����d�#�Q�����@���|���}�����>����������(� ���c�����n���U�\h���� ��f�n
����=(�:�V[m�v��g�������D&t!�Ld (����q��4j �1j��&+�F���f��_P��X�g�������(i�-}����:������sKd��Q�M<72�0��^#��vO���s�?�9�b$���� �{���w�<Z���A�����5Wm*������c6��f� �?w�
O���)m����gO�)}�����???z���s!
$-,iMN�s3 �c�&�f��%����'o��� �.J�-73u ^ %�� �����bAQ%�������6AwO�{"�������#��M��������nD �����J�B��M�C&�<������Q��T\cX=��Hw�K�c0����>*7�����Z���$���u��x�����O�3)e��6�����:�}wt=�j�yx��,�4�\7�4�.(�"$@�P}��s5��|�����57>��vS �m ��B�� ��mUl1� F'��Fc8L��
s[��
t ���H!�8���j�ioa'n���q���@5��j�"W��.�J��@�>] Z���
P���)��-�� !���a�����5�7W5M ,��B���O��������.�|�������;��?����:� L���P�D��<��TuBv_����(���V��wX{�������1��\������]p��K4)�������W��kO��o�k�����&�}Q���
�� �Cql��B=I]���
�0Q)��qm[ �,!�@��_7��(�o�� ��� z�s�������������Q�GG��S
R�@i����up� �s����*d<,�����_d]�:4�w� cKv���D��}:�*�c�$q�<
��6�q����X����g��� '������������@jf�w1�)iI+��((w����k:�u���������
]w��#>�eU��-���r1c �Z����j�����]W�g��`��1x�
C�L1�=]O*]�:6U���]V�G�O�Jcq;��\T�>��� ��v'���r�y3����F���r ������~bI���,:c�)J��k�������l>�T
]������� �(��;���~�LR��5S�����[�g��fU?�\6��������1�����5��S^9��*`���'�5�Q�~?��= ���� -�fCc���e� ? xeF!U��8O����:2��U����R�5�4�AO�N�mh�! _����Ad�N.��;p_��>*Cdb4�)3^�]���<���x�1�:`� f���_f��h���8�O��Y���!�xY������\��; ��t�?�;���3S�F%�a0A ��0O74
��������[��V�=Oc��
��<�4��g�wuW�?���t��<�P O,��i�,�@����x���w� ���������~y3~���]�=Rho�7^>/�c`��&��@0��aw��B=l5��aN��i���^���H��s/���b#^%B`��rWI|L�)�^4 �� N ����P� ���-��n��m�������T? w� `
`���� ]����t���0�M�Gx'�l�:�`��R�,)Y�(z"xi����
��C�8���)Y ��$\~Jv�a��"r�5�0�)�i� ���O_�V������P �9����g��WA �#w�:��=��zX��$-8���!�@��2U������0�.��8���?gC=!��$��I-���D>@EC �A� �,��U8����!� �PT�lM]���P��6��< 9_���0�
�1�P��0hI��D�����Xm����5�����v*�C� ?��6�m(:�����t��/�J�W�c�P1��
d;J_� ���"��iplg�/5��x!�J��� X �'"�t����BM�{����HU���N9U��l(z,�P|�FI��3��=�l��j�VOW�� 2��e�[,K���ZO>�<�<uH�<n����S�b�X�f��eFf��5�%[pxj�MeM ��+~���5�gT8�[�� �9�1��XL �V����q�SR���|9�i��C�b�����w+z�o
�A&���0X
`���&J P� �c l� ���s(R�3�L XE���4��1��3�i�U�Q�d�I*n8 ����x)��zb�7���d
����� �X��E���(�4J�E�HZ�|�
���&w�`u7$7��B���D/t��.�Q� ���4�'6-3�-q����`�8���%��t5��*#��R�.�Va] ��<=�P�5���!��}�RB ���\X�.��0�x-�.������N���2�
�� �:(��$���lj�Yw�~��JL�|L��k�Gn[�7#A�,�&�.z�f����x���^ �[� Pr����+��� h�����yi�O$&��yC��s<����a��������G���^�hN��zC��9P� ��X^��O����h��#j��%!��� � ��P�HL�� ������^� L�6 ��*���;� qD���^3��j���REC��j~���w(�L)4+��*O�h"����@�4��V�Ss��c���oq��5�� %�� *��q0�>����* W��w�b���8��:1�1u�G��>�@�;l."��jc2@���Rn�Z�)^(]�km�~{��"jQIC��amv3���k��'K����<<�N�9 ������k|�{�^B�/���CQ�'����L
s���Se�Ai1Sr��A�������M ���1 0 T]���zv���@�|�{z��7/��x��� 8Ta����~�������� 3WOf�����%P�]0`s��!��rH:`�<�G��d�A�9� 6��fi�g�����0�?�vm��}��~�h��'��tj1N�U��8U���� ������^��7�v��mHE# <O�\�S�Pe���j��;j
����Z{�� mh$���h D�2������`��-?�h����Vy��v;W7U7�tm��R��Dc�T��i��?:C
Y~�T�,Z,x�l�RcN��_>� �6�.�#\�����p����P��������w�v.mD�C�X,��T�\�f�>T��� b��[��������A��^?x\�/�f���N; J
�~�@������ �`���o�\��������?Ji�+���C�9J�
}���Rez(����$hCW�b���U3$������u�S���������]l���*��8����%
���Y�2�c Ds}���#����b�����d 0�5�H��> �����?_,7��:]��[j|��b����/a�@}4�G-�� mtR��nUh�����u&xk��-�$`��#�;��z�v&pG g�4=I��
��|b*$q[G�?r��p��;[ ��@#��I��9����\9��NUc���;iaZ� s3�Z�U����LQc%eQ` D�e�L�������DNH�W-�����;�U�+K����]���;���<�] �����N����[�@b6�G��^����D�����qx�.�m0 ��B_�9f��� �[�����R����e %I�N$ ���|����2O���R4d9n(
��!38 �l�~- ��x��B1� �����=���
i������P������X}w
�{ qp�
�+����� �d���*�n��[vB���cv7�;�d���������6���>Y�����#(�� �B�zl�j��q�i��:qZ] �X�:�~p�c�I�]H$������Xc� �FC��~��U�)��L����
�`���G�h�S��*�����7@^����m{E��g� ���Y>�VX Fy��mP0�Y������w�^�1�����������~�.�P�N(�am
�]�'�3�>��y�q�����DL+h�� ""�2�/w��s�R�4��<�����34x��:��Q�&��dK
N��v
��2�� ��jv���-��q��s-��
�q�� R{�.�t����'��q�1��K !r�{9 �|9_�^5�j|^�� ��O����l[}�=]PO/���C������\m���jY���'y�%���$�iS���
�!! W� �2A1#��,r 1 �Ih��:,�p�h�6���P���8���=���/�J������B��S�a|:��a~�hf$KGJ7ju�,� r*I�g��b?�z.��={��S�o�� 7�����;�� � :W�����2���8b��� �����d�0��.�17e�w��!�t��+��y��)�T;�
�N�
pc��eG ��A�U� @U�7� ����e�L?t�"�+��P-����%������t��T�;�q w��d���>A0��v�(h�����!6���$k� ��7t��p6s�y���:a������dc*
���i�oV>\SN�ZJ��2��8��3^v.�f"���r��(,'^2�_�M6fn�3 J� �.��f��:�n�p|��~����� m7� m�Z2��c��(J�Fg���7K��qX�3R��k=� ���:�H�(�n-B��\j�Hg$q�=i��YD���;�R��5��2`l�{��+�1?��(�\���-��mM,���������l9����6H;�=q:�Bs�^GQ�o�/���}�>�)�[��P{/�9�;�=������?z7�;���~�����a�FN���y :� �/L�A ��[�J��L6�������_ ����A�B��� �'\FF���Ai{��/[�;� B��u+04�����$C�aI�v���{EL�TR1���<����� ��7����]�v���4Fk�.McD��/�2_:}�z�WJ_���~��)np��E�t���E=���/>���E P���k��KJS��b����I��;�Ee�+e4r�H������'`��*��)/3{`����q����������+�?����M�Md$�����) �
������W.--��,)j C�������h5K�*E@�}�.��{��!�Q=M�Iz1r�^��I��EW�E�����(nQ�
.�cc{��(n �<��p�-�"�[�E���(��E�R��EW�E$K��@T�\Dq�p���"\�x.\�fG��E��d�0��!#|������� �{Kz(�/�A�Ql����,�� �{Iz(���P\��P�"�A��I�������I���IZ�X;�A_�+� ��'=�Ms� ��%=W%=��Jz�M�
�5� ���Bo��'��b�����)�����%m����G��PT��� �t�m o�!��x7�� 2�z�TkY� �t�6������5�Hd
��P��;=�F�_XO�pR}�irvq��/���}|���Oo^�:;O��m��8�i�)]�H*�0�B���w���e. �4Z����,�Z�JQ��+���a������#��.�xxZ8<V���*ql���D��Bf{3� ��5S���N �o�w��UL�eE�:���B6 ���o!!�������q��
,R�b"���EJO���j�F��j x�O� h� _����3�N� (J��,�|���T�A�� �TO�� �����j�����!� ���:�����x�
Y�J�E�L�ZZ� �t8 "��c�!f^����U-�����NE�����p�6��
��?�d�v��mEc~�=��m�m�mY��9}'7?�-�� ��l:k���Dz�� ��;���N�U��\���J{��R� �<P��-�z�%�]�I���
��p@Qp����`�$4�t0�}+�i K�� )�84�� �<P_hX��G����rJQs}5`A�Ep�d�������5$�9��/K
����m�I� +�� ImD�����)9��k ul��2M Kz����C@���hQ4��>
��i4A�l����Er� �H1/���(�P����\ |���]F�����L�du/� Nr�Zd��*��6�bp;+~ hV�b�3�%8���)�G�1:��J�t����UnrM� �#����� �7���r���
�� N�������L"����� � t7P���V���F%$b�O)2q����^��l�:g��~&������9{����dz����T�8����<��e�-5�oJ�����<S�F?D]�<S �*�S �9��J� ��[�e�"P�}qFC�/������.���|����������^]qL�#��{WW$�{r��1�'K�e��^�c�gtj����}fdo�&_X��- ���+��bb�h��!h�yo2��bi�����@I�A��V�����4�$i
(rE ���\)�p`-�c#tsV�_U ������O����`��|�K�]i��a�� pS�M��� LHH�
V��[p����$k�}h��3C��,I<����O1K��&qG,��M�R�}:Fs%G�����> �B ��{^i\$�����o.0]U�����n����vZ���
�M�{#2�����l�S�r9�!$�n�iz� 5��6MK�r���� ��?�n�x^�M8��������j]�F`TH���5I�B8���|)����iXyW!;�u�����al�/��@����iW��St�? :*������ ���6�b8�s7 11� ��
�����a���0�z�(�+A�U�cw���=���S���A����V�k)��H����/ f��� ����$�!��v��B����������1��,����D�c,����<y/8�
Yx��[Z�C�����m9Y//� <r�'Ny ���y�o<S�� ��z�)�VJ��S(u������|�����@;�:���G� �h��q"Q���74�d���sj@$���r. �]����� ��T-���F�_�!9��� �c�q�j0�N]���@�R)+���0�sF�b���U)p���R�A���������K����\�v+��]����iy~��|t}g��~i�M����O*7��
�2����\��#2hT��R����X4�0v�K9v� .n�p"��� � g:M�JI� %���:��jT��p4� Dut��!V`��a0�>,ePE1w�K[A�I5 +hQ�U�Z�Y��J��T��
b�R��e�8� ���X��U��rC$r�2�c l����.C=����La��y������R�����0�gO��&� 4�����g����� ���� �j71�c�������Q�/^?�\�(�{�a)�'�P�B_�����]]/z����j�R�����F��ZL���� s��M��3�/��� n��
�7b� ����,�1�/pw�AN���1W���FFH�X�
sa��jZ9��sO����G��HQ<��p���$I:M-���9���;�~"��c��a����W� ^�OAS3�U�x���/�S�"�dq�j�qi� )�0�}��p5]P�2�7c�����%q��]������;�F��0��F��,5�u j���(V)`��3l �Vu���1G��~.qf��y$�>��z {����*���nH ��p�C��m!0�A^�CE2 �Rx����H%J���M�q��kE������V[>2�?����������<fv�n
����t�42�N��i��S���qr8�1�z���h������Hyn~W��\��O���A��{�T�S[+�6 ��
N�!�6�@�-+��*�������m�Lb�sL�1Z� p|�Ib��� �7�%�;U4]��� f��HC�=FF���%�|��E
of�����9���+�>�2�*)����� g�;�
��S(��
���m4����.�zy 06��v���|���(xsqvyzqqv������G����wN
�o����l�jZWs�S��[���]X�� o��V�-�7a��1U���@)�+�z��6� t|��
�����Z�E�� gK� F0���c���x���.q��-z|��� �d���*'���ap�b$~��vw� ���]�'��������hh% J����9��|'��(u�v��RpCz[��f)������67)��Z�,��������\��C�e+�e��+s����%�C:2�`0�4��3����tQbb�����H ��F�� ���L��U�V��wdRJ�5��F ��@-��5��<%��}.y`(�x ������,o��}l�.)����!����dD=0]"�U � ����V��%H,��%x �;~��e ��|�Y 38q�Q�|���.0�u�
� ����|�6��HY+��=2�� ��$~�A6|�Az�Ar�A���W���>{~����o4��R�*Y�Qh�|7���S�+��� Oy�3�
��o(aL����D��1��( #��t>�@fK�n��M��+qY�PU����5Xv=����"/eD �����^ ��n�q>��*�G&�W�x��0Y��z}��1�I�Fw�.m(�����������P�UE��� H7)��ll���K���[�!��!^�D� ������K�C����sz���CR���hY��K�!�r���`�6$����E� �p�1�O��ytu���� �c5E����mMJ� Lo@�f�|���u��6��\��@�v����nm�U��~�0��4�����
S�b�8FH�xXz����_��Z�wT�[1Y��+ 1�N~]��7
�?R�
'NT�IdP���j�Ko�HX)vS^J� ������'�&��=��c=����^�=�} f���:mwu�C�W �O;2F ���3�T��d2*�N*���8Z|�5t^!^�B��P�,��� ��k ��'�?V��`\���f����e��CW��Z%gf���7�x ���L���;��|��Z���W-���vgC��F��!����6gD��~�q"m��/Z����6fG+�����B6;*����i��|���6x �.MV����+��P!:e�\� �� ��g,�2��A�x:���1�Pl�� ������g3����ON�������������Tu��TD)�x���O�6���+b�vYNn��&����q�m�G%����T��
9h�i��Mo�6��@�����m�� �r�
�-��vL�� �N�6�xs���
���� _>>{~��Le �j����S�5,��8d;�On��9{ Q��s\+�7����'��������������7O���>;a��f���%DE�����2b<I��W�Pn{<�-�}���V�O8�3~�N��|B�]g]QUh !0�Z����G�E�4Ea���-���gq �2��as>4^X?��5����������s�%O���
�S���T�v��?���C?��N�����5D�����2�S\��h�ax�J��,c���5 #9Dr��
�,*�������8�����^�2 ���[K\cl�9����a��U���
�D���{cV��
��=�����Ez^H����N�e8���<��&�&�Y�l��{����)�%����W�4�<Q;�VU�2�3D�����,=))�cb|��2
[�4v��� ZOVbV<Y�bHr�F����C������G��e� D.� ���0
qAf���@�0��}�e�r*� &�t�z�p�h��T�9+-�<)�l� ����doz�e�&;mmN���u�������v;i|�����w'~�^�m~7DM$��\ ��YU6��+E�N�e�������L����
�����Mh�e���d� ��5;yB\����"� �e B�"�e�,�Bw E����������x=�������������{�f�TJ� �j>�]������M��L ��z��W����t/����������`M/�h��X�*��_��Ey� �Xy�v����J&m�9���N�z��r<t���G������V�$x�����q���� )K���������pG��a���S�S�Oq�� suhd*�Y*���I_ V��axP�Q"���� udy&U�R�-�h�|K�� g�u���| �u��9O��Huj Rf����q4�S=����I� ��g'^2�J��g�)I���
��u�Y������*���C���u�j�����5X��-���+�v ��p w~���L+����5��s�|�Q����y��w����6�j��kW$�����EG��2t����+P�$��
vW�l���cB� /��F� ]�]�P�
�R�$
�I�Z��i����)��9���3�������de� ��Zou�Z�t� ������K� {�{�H��i���S����lJU��B��o��|����D��Da�]g[U���,nC^VU �2�iG�r���X�`��Q�_��kb�v
����7���vl��
���g�* � O���=�x+!�� ������ ��� �.�:��uY�� ���
)��d�2��z�v�%��4Ia@��>&��%��Ho�8�P�FD^�(��C�N�F��� x��^%�,���,�������o�_�|������{" �s#���5Q18��4Z�^5�=�R���lz
v�-Z�:�u�=��
*uJu�Y������ �7�NLg TX��#B�v�jN�$��4�y8����N#�h�����
k"����ig�G0�y������T������$�{�p������bG���)!O��t�
��U�aV�\�
A-NA��Ut`� �� N�0OK��i��G�� ~ ���<�Q�'vI�V�_ /Wq$��
�N;l��M)��Rr�ywk�,v�����Q��.I�kC
&�������x��^���^ ��r�������fF�dUO�<yM�K�7����F�,i �h� ��Od,V����C����aRJ�{Ti�g���b�1�d�fd�l�vx��[���8 y�7�Sm��s�&������']�� ��wZ�>f^���+����&S�����C:M}���ao�;r\��#jI������}�H���u�G!�m[�E��my[��w�uYT ��#L \~���}��u����`��Q !����tms������;�RD�y�;J���d
�
H�< <�7�E=��Y�eE�`*���S������v�bj2�V��+���~� >p���4���|��X�h�X������X�T��t� +�D��5^�PS. :z
{"U��~Q��� h���-�Ik1� �6 r�B���/��sLm���QEx �`G(����@�
g��O=��.���J����]��O�
�I��nBQ��2�S[%�/��a�Vp;G�9D3�I;-�'��9NC��V�GQ����p�# #
��g�)
�d�Q!r+\%� 0�w=�������0,��d�u����d��6X�QEP�i� ;p
� ��Vh\!�
Zf__��4��>(�mL���x����������`P�2\����n$��C�� ��$!����(IzD�����t����Y�q��I<��������N�� �������d1�4�9n^g����#)������x�jp��� �P�Q�fF|V�j����6l�b�� |f�d�Z)u�X���Ax�X��P���������9����4� ���jVd� U� ��w\!L�kul�m����E��w��}�o��f^K�0� �� ��p�J@���� 4 �O��J� :��.�� W( �A�%��D���>�|����7��Pc??{r����*Z�������D_$�C�!3S+�t��|(������Q���%B�
$�V�e����_�m7G?����s��e�v;nm��j���>��uE#.������_ ��!� Hq>��uK��� *z�O�:�6��T>�c����)�������@<��K ��WZ�y�m�����QF?Jh����W���)��t�f�R��`>�����.�J�x�D����{q��$�Y�0Tm ��v �� �7���8'�#' �rP8E�PWK������Qu O�W ��@E��
�� �rp�F\L�Lvbp������J��So�Z (��d�lG�wIs;r�Z�@��q�� �2�n!C1/�������Z-�N�jOm����h �R ����*�v��Y�}�i�r�Hp��6XvNV/�<� �$ �G Z���D�.��5L�����I�_��-�����A�z�]����s���� ��}�P��8���r����4��)�
��
�l����������ZZ����;9�����*C�@m�P:�R�6�5 V��lk.:�]��Yrjv<��;��+#�0� ����4e�}�D���U#��>��5:���p����o[����e�� ����T��;���Z�6Bo� �Z�!���I��C-���W�=�p_)(�{JA9�2�
��G�=\�<��>��P����Nd���0��I*�w�����q��5������a7���S#���WYT���������Na����Z2���&�c���I���X�\�[�{-�P������gd;��S�4�S2��_7�vSk���e�K9����v(�� ,E���Z� ��A��S�_1������M �<�q��
����� �e�]��_#�Y����H����s��p�<=G �1gW_�k�� �xw��K���e�J� �3��s��@�@\��f� 8Q� ����I��l��,�U��������Sg��� �!����b7�EbH������ T��S��?{�K���:���#RF�a\���Z�@OT��%��?��A�V�0�u�!_7�Y�V*�F�^4���v���T��r���j�����M����=&O��UJ��U%�R��-���1 �g� �����{�U� �X�E ���R�DY�hH��9dOU��q��
�ez�+=)��.����$��r��y�����%b2+(K�� �"#�7�j�C�'w � �P��~>�j�?=��8���h
(�zB��
z����V�3�h�+{��� L� �?��R�K|���a�� VI��&'�)���%U��xk�d�#�2e�a��p�%*����N� L�Z0�|Aq$�l����SX������> �{� � ���lp:X��B�E�b #
^ N�!D��cl��>q�M�{T�"��������f����v\��Ex�1_�P�F�8�e �-���G��?���>L?��iG�p�G�0� f a}�����#����i � O�bu�2�..��WY=vr�[�N�V���M��"�� �a��D��_��B ���%�=��V����,]����&���bG+���|a/)��D���z{���Ry`�pM����
�����1-ZKx��s���,K��.U�M�\>�L��� ���I %3I��6���">� ���I���p�������G�y<��X���y����L�V-Uk-sIa�K,{)k����y8��?�� �$y\ {���$��+���h��8��x���5�_�$�zA��Kj�F/E� � � G\�� ���t���:�pI0 3�9�'N�"O���` bD�����q��HF�N��2 u��Q�1 ( �������z�G��Y0���� ���'C+vN��$$:�������������Ax1�h"-��s��S����U�@J�X�d/(����" r!�������q ����@3�n�T3�7��4��J 0��ZeG�����~o��������'8���xL�6x��^�<��;����J�<���v$��z��n�^[�0��\��=��D�Y�, ��[�R���i�"`��2F��d��_�yor����(��|��;���I�Z���wCb����D 2�_m�����RJ{C���DQ�-���� ���E-.�Ix���z�)^x5�`VR��6��eB2e`�` .8�L�D�zxo�f�-`��`p3���, ����
���w�a��������F�oQ5QR��P�V�>�&�L��V�8$O�$4 �&4&���1�s J� �:O� U��v-I��C�z�����,e $H� ����2+�}�+��Ii}��� o�7h��A�9���rV�eY�Yr� �O���C4���P������p���=.ZK�l�tkU�Y��dX��?���Sr����H��I�*���$|��]��4 ��Ag �������i���|�QO��f������Ie�� �dgkM>�1�^6;v=��.h ������g��j��3��Vqw$���������1?�W��Y3�t�\��������gTz*�������jtg�.~d�N :�{)<%�`$��]N� !�+�*8�U2�i�'�:�f��*�[�J�����A�~W��V[�?(9� 1.�T�x��j�^��p�~�+�������d����<���1n��0l(Y����AS��/\�$����� T�"��p3Ty&���C��p�� ~L��r� ��:���Z�Z��v���U���[:GVpg� �,�Z��A��v*�8��K�d�+ �T���$��/p�N����� ��4���V;�fcX.������s��\��6��}�s�-i���T��n�Z�U��1
��g9��za�*��@1S~�?���v� *V��$�QJ � �������8C�( ���I.pDn f��,�K=#\u %� R�������O��w���.v�2*�s}��������5�A��s:5� ��_={���.�yz�|;_LG�X�S��������q�z�w ���=���} ��q���y�p�.U��\��7��}3������d��9���{r~��*�c%�po���7D� 9�(����x��
�zN"�����vZ.���"k�Bb��{~:'_����R<TW� �����YKQx�� ,��1�����B/�~� =c4���a�p�NxeZ��k���W2��V��w,��sxa7� ����� �f���S��}&S�x���O������wv��o� �XB�N�����{��� �~���z�:�U��J������(����3�$D_ �{O�" ��Z%�
��pr# $�����t�����f[�T$���^F:� O�+�9��6E� F� ���<p���dQ��C���������k�_�L��o�7���B��e��6�K� 6����k��h�A^#cZ�%���s � cp���9�7��7�sd�����{D��zW���wK�VJm��cQ
7: �_�o>�A����w��W��OAk�q����+'�\�W
\ /���Q��=J�����
���j��h�+�j��M����Z���`����Ua( �]����
nY��j�R��k�z���tj�j�u�^ z���-�Z�Qk����=xt��������z �����'>�����U�wU���U�?��j�����Iu�{���O���gn_�g]�%�P�
(l���n��Ou���S���5�V;.��W�.�����@���d��,.���0�Q���m�����j�e�����]t���;B��(���%�1��dJ�?:&�c��!����g�^��q�)����ut���qZ��� �CR���
me�N�g� E��lgl3�L� ���':q�7�?��s�Y����r9 ��f���gv�4$�d�*b&��A���{6�nK���P�/��'���g���y�zT���w�A2�t%�=�U����N��?G Ld�Fp �������H��� oo|L�ta ����g�> @�H"Y�D����x����o=R�'��bL|��d�gx�lL��|���/��Pu�c�q)KC AlV�����7���L#��8��z w�� �����VJ��LL���8��|�������Y�T�Z�c*h���%����iOo1$�`����� 3Mw)z L���}M�A��Cn%
�1Y�k�n���7��e�o������Cd��
�RSl�l���3��+��c�RQ��g/N~>� ����X��"�/w� 2�������X�x��������t+�J�\�4*�N��r�������Z��Lre�e��.,����
�h ��*��@ & 4��AP��)��8�+�,Z��� -CM�>��P�3��t��0C�CZ�$�h$�c����7g�OU���1��5�T�MD��K�~�
Wh�S���gF~�;1��=��@��6[�~��{bTy�E��8-d��g�haC����|DkMh7J�������r|�)J
?CM�zb��J�E]�`�� PJ?6���|� ��k,����t)cO{%�7KMw��$B���+�O����F$�����S� Nv"���I��
�� I��l��O`|!�����t��3�^�<�Q.�k�fu��Z�b�n
�X����+VF;
�L�����'�^`�������g��� ���!t; �*�%���ps�5�6����g��xX�g����
� 2��i���� �n@�G��@���ic(p/�����}{w�}�����5�����G ��_�=?{�:��_:����� ~ �q�R
�G��EV�*+�k)��������`R
�R 7�pzS8�ym���[J����
o?�`N_�>���;����-�.��+�����X;�����)�S�<�����������\U?�D4:� � �~�LP��!���������t�6�� ��@�@,qh���c����4Ue��g���.��36)�\g�zc|DPH������� �dd��� ��{wUT���x3�� ��� J�(w�"T��C?������� ����������0�k?B�����+p����k� ����8�{���8U�����6���/��|����_��p����|:�%� ��^�8� kdzw��J�L��o��L'�����>:2/1�� _�R�'����D�B?)S�w�h:��+`AK=+$�B�B��G ��u��@��>;u1��xy3����]���'�=}���������hy�&W ��� 3��7p 8�����?[�X�����e ez���e>4Y���C��h�u(��#�q�&�����1��m���m����i�j��l���@�{�H� ��Di<�WX����Oy?��J@����(�C�Bdr� .T�'j��=z���/O�����_q�k��;��-Y��aG�%��"��)#�^��;[����9������9��-��_��y��h����sx4g���'��
O�"� ,���\dT $��� �ao���k3�u�j[��M)@��(})�������dC�1��� {t���� �/ ���� e �t����s��~�����~�:��n*O�g���X=��`s�j� -nZ�v�X#Iii�q��+s���c���� &���!\�,�� �e���EbR�vc o�%"�/�u����C��,��i,<}�f���l6���������������B��7�Q ���q�� <J��{D#�I��F6(M8�%|N����6^D�
���,�/�0���|���� �7����T��j�����^78�-���9?��s��`�tX�#���tc�:�+��m���J�5R[�fqNV��\v��i�����:�k���\��2�����������^��
f��c=A�m�������x���) N��a�Xv9��C�r����0 {����
G�vV�J������d�D�54�k0���`�6� ��q������r�cN�h�����\�k�o�� G�8 ��������� ����1��kg�=���������p���/�X��{�7� �&}���������
����� T�z@�e
��x���a*��#�' �����$7��� �z�����N7� �.3��o�Nn_����Zo����]}�h�8:�l��������{���w'�i!�����
��/RW(��� ���Um��o����?imW��u[���{���>�����m B�z�w Ev����:��d+������9e���Ae��Dqe9s���b��&���?H�� ��~��������/���tt#���N�XB6�Y������7��`@���j��SkW��o*�V�R����1>��������r�]����Z� �_��r���_;��:�~�S���g�O�k������U�J%q�����_ ����������2����<#�hP
z�ry����}������e6��� s �8��� �� ����Q@d�4p����n����e������2��1n��8�R�&��.$���6rE��A�F�����7�yH���B�KCT'� PA�|�>o��`�DUu���^��[�+��m�@��������� O�9`�\%�- �}L��������+�sE�8��M� e~@���`�%�uM5�p���Hy���k8@��b�!�li����I ����ZG�}�.�J�c�G��Tg��Er��PI�K��X�@UI#T��G�b�Bg���*!J�3=">pE�)�ew���m�C��M,�t9 HI��^��[@'E8��]�|:@Qzr�%"{�����*L�C����O���h��e�k�H���z\�p�������0�j�@A� ��� ������a ��m1���(�������`~� <r����Jr�p����> .�p�
�U
nq\�h���l�/�#W���M�f�\�2`Mg�8b�:�����1��1�>�;���l��RP�� �k-t3��yh0��6�#����J���o�A�*o*�ajQ& r� �' H����`&��L������53��MzS�9����������D�Tp��Y��E������EDL�3L���"I�� �KE-����_U� �R���5��������fx�����N|W��8U4#�%��t,a�o�i��+ ���� c��]�� �% w ��p�$q� � ��Ff��
��C���3��>���q2x��Tq6|OU��(#.�pOqpT\I�g� ����O���r���Cb��'�/���G��z��.������!�� ����>�w��\�� ���� <�8�h"��P�����tz �`�Ks���h5���&��>�N !�$ 0����]��a:#[�0Xs�
K�,��H����+F���.�Y8�����
��B� ��w����N��%�2v.,�h��gt3�n7�S�6���
���})�;��ie <i� ����C��O������������������������� h 
Re: WIP: Aggregation push-down
Hi Antonin,
To understand the feature you have proposed, I have tried understanding
your patch. Here are few comments so far on it:
1.
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct || aggref->aggfilter)I did not understand, why you are not pushing aggregate in above cases?
Can you please explain?
2. "make check" in contrib/postgres_fdw crashes.
SELECT COUNT(*) FROM ft1 t1;
! server closed the connection unexpectedly
! This probably means the server terminated abnormally
! before or while processing the request.
! connection to server was lost
From your given setup, if I wrote a query like:
EXPLAIN VERBOSE SELECT count(*) FROM orders_0;
it crashes.
Seems like missing few checks.
3. In case of pushing partial aggregation on the remote side, you use schema
named "partial", I did not get that change. If I have AVG() aggregate,
then I end up getting an error saying
"ERROR: schema "partial" does not exist".
4. I am not sure about the code changes related to pushing partial
aggregate on the remote side. Basically, you are pushing a whole aggregate
on the remote side and result of that is treated as partial on the
basis of aggtype = transtype. But I am not sure that is correct unless
I miss something here. The type may be same but the result may not.
5. There are lot many TODO comments in the patch-set, are you working
on those?
Thanks
On Thu, Aug 17, 2017 at 8:52 PM, Antonin Houska <ah@cybertec.at> wrote:
Antonin Houska <ah@cybertec.at> wrote:
Antonin Houska <ah@cybertec.at> wrote:
This is a new version of the patch I presented in [1].
Rebased.
cat .git/refs/heads/master
b9a3ef55b253d885081c2d0e9dc45802cab71c7bThis is another version of the patch.
Besides other changes, it enables the aggregation push-down for
postgres_fdw,
although only for aggregates whose transient aggregation state is equal to
the
output type. For other aggregates (like avg()) the remote nodes will have
to
return the transient state value in an appropriate form (maybe bytea type),
which does not depend on PG version.shard.tgz demonstrates the typical postgres_fdw use case. One query shows
base
scans of base relation's partitions being pushed to shard nodes, the other
pushes down a join and performs aggregation of the join result on the
remote
node. Of course, the query can only references one particular partition,
until
the "partition-wise join" [1] patch gets committed and merged with this my
patch.One thing I'm not sure about is whether the patch should remove
GetForeignUpperPaths function from FdwRoutine, which it essentially makes
obsolete. Or should it only be deprecated so far? I understand that
deprecation is important for "ordinary SQL users", but FdwRoutine is an
interface for extension developers, so the policy might be different.[1] https://commitfest.postgresql.org/14/994/
Any feedback is appreciated.
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de, http://www.cybertec.at--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
--
Jeevan Chalke
Principal Software Engineer, Product Development
EnterpriseDB Corporation
The Enterprise PostgreSQL Company
Re: WIP: Aggregation push-down
On Thu, Aug 17, 2017 at 8:52 PM, Antonin Houska <ah@cybertec.at> wrote:
Antonin Houska <ah@cybertec.at> wrote:
Antonin Houska <ah@cybertec.at> wrote:
This is a new version of the patch I presented in [1].
Rebased.
cat .git/refs/heads/master
b9a3ef55b253d885081c2d0e9dc45802cab71c7bThis is another version of the patch.
Besides other changes, it enables the aggregation push-down for postgres_fdw,
although only for aggregates whose transient aggregation state is equal to the
output type. For other aggregates (like avg()) the remote nodes will have to
return the transient state value in an appropriate form (maybe bytea type),
which does not depend on PG version.
Having transient aggregation state type same as output type doesn't
mean that we can feed the output of remote aggregation as is to the
finalization stage locally or finalization is not needed at all. I am
not sure why is that test being used to decide whether or not to push
an aggregate (I assume they are partial aggregates) to the foreign
server. postgres_fdw doesn't have a feature to fetch partial aggregate
results from the foreign server. Till we do that, I don't think this
patch can push any partial aggregates down to the foreign server.
shard.tgz demonstrates the typical postgres_fdw use case. One query shows base
scans of base relation's partitions being pushed to shard nodes, the other
pushes down a join and performs aggregation of the join result on the remote
node. Of course, the query can only references one particular partition, until
the "partition-wise join" [1] patch gets committed and merged with this my
patch.
Right. Until then joins will not have children.
One thing I'm not sure about is whether the patch should remove
GetForeignUpperPaths function from FdwRoutine, which it essentially makes
obsolete. Or should it only be deprecated so far? I understand that
deprecation is important for "ordinary SQL users", but FdwRoutine is an
interface for extension developers, so the policy might be different.
I doubt if that's correct. We can do that only when we know that all
kinds aggregates/grouping can be pushed down the join tree, which
looks impossible to me. Apart from that that FdwRoutine handles all
kinds of upper relations like windowing, distinct, ordered which are
not pushed down by this set of patches.
Here's review of the patchset
The patches compile cleanly when applied together.
Testcase "limit" fails in make check.
This is a pretty large patchset and the patches are divided by stages in the
planner rather than functionality. IOW, no single patch provides a full
functionality. Reviewing and probably committing such patches is not easy and
may take quite long. Instead, if the patches are divided by functionality, i.e.
each patch provides some functionality completely, it will be easy to review
and commit such patches with each commit giving something useful. I had
suggested breaking patches on that line at [1]/messages/by-id/CAFjFpRejPP4K=g+0aaq_US0YrMaZzyM+NUCi=JgwaxhMUj2Zcg@mail.gmail.com. The patches also refactor
existing code. It's better to move such refactoring in a patch/es by itself.
The patches need more comments, explaining various decisions e.g.
if (joinrel)
{
/* Create GatherPaths for any useful partial paths for rel */
- generate_gather_paths(root, joinrel);
+ generate_gather_paths(root, joinrel, false); /* Find and save the cheapest paths for this joinrel */
set_cheapest(joinrel);
For base relations, the patch calls generate_gather_paths() with
grouped as true and
false, but for join relations it doesn't do that. There is no comment
explaining why.
OR
in the following code
+ /*
+ * Do partial aggregation at base relation level if the relation is
+ * eligible for it.
+ */
+ if (rel->gpi != NULL)
+ create_grouped_path(root, rel, path, false, true, AGG_HASHED);
Why not to consider AGG_SORTED paths?The patch maintains an extra rel target, two pathlists, partial pathlist and
non-partial pathlist for grouping in RelOptInfo. These two extra
pathlists require "grouped" argument to be passed to a number of functions.
Instead probably it makes sense to create a RelOptInfo for aggregation/grouping
and set pathlist and partial_pathlist in that RelOptInfo. That will also make a
place for keeping grouped estimates.
For placeholders we have two function add_placeholders_to_base_rels() and
add_placeholders_to_joinrel(). We have add_grouping_info_to_base_rels(), but do
not have corresponding function for joinrels. How do we push aggregates
involving two or more relations to the corresponding joinrels?
Some minor assorted comments
Avoid useless hunk.
@@ -279,8 +301,8 @@ add_paths_to_joinrel(PlannerInfo *root,
* joins, because there may be no other alternative.
*/
if (enable_hashjoin || jointype == JOIN_FULL)
- hash_inner_and_outer(root, joinrel, outerrel, innerrel,
- jointype, &extra);
+ hash_inner_and_outer(root, joinrel, outerrel, innerrel, jointype,
+ &extra);In the last "regression" patch, there are some code changes (mostly because of
pg_indent run). May be you want to include those in appropriate code patches.
Some quick observations using two tables t1(a int, b int) and t2(a int, b int),
populated as
insert into t1 select i, i % 5 from generate_series(1, 100) i where i % 2 = 0;
insert into t2 select i, i % 5 from generate_series(1, 100) i where i % 3 = 0;
1. The patch pushes aggregates down join in the following query
explain verbose select avg(t2.a) from t1 inner join t2 on t1.b = t2.b group by
t2.b;
but does not push the aggregates down join in the following query
explain verbose select avg(t2.a) from t1 left join t2 on t1.b = t2.b group by
t2.b;
In fact, it doesn't use the optimization for any OUTER join. I think the reason
for this behaviour is that the patch uses equivalence classes to distribute the
aggregates and grouping to base relations and OUTER equi-joins do not form
equivalence classes. But I think it should be possible to push the aggregates
down the OUTER join by adding one row for NULL values if the grouping is pushed
to the inner side. I don't see much change for outer side. This also means that
we have to choose some means other than equivalence class for propagating the
aggregates.
2. Following query throws error with these patches, but not without the
patches.
explain verbose select sum(t1.a + t2.a) from t1, t2, t2 t3 where t1.a
= t2.a
group by t2.a, t1.a;
ERROR: ORDER/GROUP BY expression not found in targetlist
[1]: /messages/by-id/CAFjFpRejPP4K=g+0aaq_US0YrMaZzyM+NUCi=JgwaxhMUj2Zcg@mail.gmail.com
--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: WIP: Aggregation push-down
On Thu, Aug 17, 2017 at 10:22 AM, Antonin Houska <ah@cybertec.at> wrote:
Antonin Houska <ah@cybertec.at> wrote:
output type. For other aggregates (like avg()) the remote nodes will have to
return the transient state value in an appropriate form (maybe bytea type),
which does not depend on PG version.
Hm, that seems like an awful lot of work (new version agnostic
serialization format) for very little benefit (version independent
type serialization for remote aggregate pushdown). How about forcing
the version to be the same for the feature to be used?
merlin
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: WIP: Aggregation push-down
Jeevan Chalke <jeevan.chalke@enterprisedb.com> wrote:
1. + if (aggref->aggvariadic || + aggref->aggdirectargs || aggref->aggorder || + aggref->aggdistinct || aggref->aggfilter)I did not understand, why you are not pushing aggregate in above cases?
Can you please explain?
Currently I'm trying to implement infrastructure to propagate result of
partial aggregation from base relation or join to the root of the join tree
and to use these as input for the final aggregation. Some special cases are
disabled because I haven't thought about them enough so far. Some of these
restrictions may be easy to relax, others may be hard. I'll get back to them
at some point.
2. "make check" in contrib/postgres_fdw crashes.
SELECT COUNT(*) FROM ft1 t1;
! server closed the connection unexpectedly
! This probably means the server terminated abnormally
! before or while processing the request.
! connection to server was lostFrom your given setup, if I wrote a query like:
EXPLAIN VERBOSE SELECT count(*) FROM orders_0;
it crashes.Seems like missing few checks.
Please consider this a temporary limitation.
3. In case of pushing partial aggregation on the remote side, you use schema
named "partial", I did not get that change. If I have AVG() aggregate,
then I end up getting an error saying
"ERROR: schema "partial" does not exist".
Attached to his email is an extension that I hacked quickly at some point, for
experimental purposes only. It's pretty raw but may be useful if you just want
to play with it.
4. I am not sure about the code changes related to pushing partial
aggregate on the remote side. Basically, you are pushing a whole aggregate
on the remote side and result of that is treated as partial on the
basis of aggtype = transtype. But I am not sure that is correct unless
I miss something here. The type may be same but the result may not.
You're right. I mostly used sum() and count() in my examples but these are
special cases. It should also be checked whether aggfinalfn exists or
not. I'll fix it, thanks.
5. There are lot many TODO comments in the patch-set, are you working
on those?
Yes. I wasn't too eager to complete all the TODOs soon because reviews of the
partch may result in a major rework. And if large parts of the code will have
to be wasted, some / many TODOs will be gone as well.
Thanks
On Thu, Aug 17, 2017 at 8:52 PM, Antonin Houska <ah@cybertec.at> wrote:
Antonin Houska <ah@cybertec.at> wrote:
Antonin Houska <ah@cybertec.at> wrote:
This is a new version of the patch I presented in [1].
Rebased.
cat .git/refs/heads/master
b9a3ef55b253d885081c2d0e9dc45802cab71c7bThis is another version of the patch.
Besides other changes, it enables the aggregation push-down for postgres_fdw,
although only for aggregates whose transient aggregation state is equal to the
output type. For other aggregates (like avg()) the remote nodes will have to
return the transient state value in an appropriate form (maybe bytea type),
which does not depend on PG version.shard.tgz demonstrates the typical postgres_fdw use case. One query shows base
scans of base relation's partitions being pushed to shard nodes, the other
pushes down a join and performs aggregation of the join result on the remote
node. Of course, the query can only references one particular partition, until
the "partition-wise join" [1] patch gets committed and merged with this my
patch.One thing I'm not sure about is whether the patch should remove
GetForeignUpperPaths function from FdwRoutine, which it essentially makes
obsolete. Or should it only be deprecated so far? I understand that
deprecation is important for "ordinary SQL users", but FdwRoutine is an
interface for extension developers, so the policy might be different.[1] https://commitfest.postgresql.org/14/994/
Any feedback is appreciated.
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de, http://www.cybertec.at--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers--
Jeevan Chalke
Principal Software Engineer, Product Development
EnterpriseDB Corporation
The Enterprise PostgreSQL Company
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de, http://www.cybertec.at
Attachments:
Re: WIP: Aggregation push-down
Ashutosh Bapat <ashutosh.bapat@enterprisedb.com> wrote:
On Thu, Aug 17, 2017 at 8:52 PM, Antonin Houska <ah@cybertec.at> wrote:
Antonin Houska <ah@cybertec.at> wrote:
Antonin Houska <ah@cybertec.at> wrote:
This is a new version of the patch I presented in [1].
Rebased.
cat .git/refs/heads/master
b9a3ef55b253d885081c2d0e9dc45802cab71c7bThis is another version of the patch.
Besides other changes, it enables the aggregation push-down for postgres_fdw,
although only for aggregates whose transient aggregation state is equal to the
output type. For other aggregates (like avg()) the remote nodes will have to
return the transient state value in an appropriate form (maybe bytea type),
which does not depend on PG version.Having transient aggregation state type same as output type doesn't
mean that we can feed the output of remote aggregation as is to the
finalization stage locally or finalization is not needed at all. I am
not sure why is that test being used to decide whether or not to push
an aggregate (I assume they are partial aggregates) to the foreign
server.
I agree. This seems to be the same problem as reported in [2]/messages/by-id/CAM2+6=W2J-iaQBgj-sdMERELQLUm5dvOQEWQ2ho+Q7KZgnonkQ@mail.gmail.com.
postgres_fdw doesn't have a feature to fetch partial aggregate
results from the foreign server. Till we do that, I don't think this
patch can push any partial aggregates down to the foreign server.
Even if the query contains only aggregates like sum(int4), i.e. aggfinalfn
does not exist and the transient type can be transfered from the remote server
in textual form? (Of course there's a question is if such behaviour is
consistent from user's perspective.)
One thing I'm not sure about is whether the patch should remove
GetForeignUpperPaths function from FdwRoutine, which it essentially makes
obsolete. Or should it only be deprecated so far? I understand that
deprecation is important for "ordinary SQL users", but FdwRoutine is an
interface for extension developers, so the policy might be different.I doubt if that's correct. We can do that only when we know that all
kinds aggregates/grouping can be pushed down the join tree, which
looks impossible to me. Apart from that that FdwRoutine handles all
kinds of upper relations like windowing, distinct, ordered which are
not pushed down by this set of patches.
Good point.
Here's review of the patchset
The patches compile cleanly when applied together.Testcase "limit" fails in make check.
If I remember correctly, this test generates different plan because the
aggregate push-down gets involved. I tend to ignore this error until the patch
has cost estimates refined. Then let's see if the test will pass or if the
expected output should be adjusted.
This is a pretty large patchset and the patches are divided by stages in the
planner rather than functionality. IOW, no single patch provides a full
functionality. Reviewing and probably committing such patches is not easy and
may take quite long. Instead, if the patches are divided by functionality, i.e.
each patch provides some functionality completely, it will be easy to review
and commit such patches with each commit giving something useful. I had
suggested breaking patches on that line at [1]. The patches also refactor
existing code. It's better to move such refactoring in a patch/es by itself.
I definitely saw commits whose purpose is preparation for something else. But
you're right that some splitting of the actual funcionality wouldn't
harm. I'll at least separate partial aggregation of base relations and that of
joins. And maybe some more preparation steps.
The patches need more comments, explaining various decisions
o.k.
The patch maintains an extra rel target, two pathlists, partial pathlist and
non-partial pathlist for grouping in RelOptInfo. These two extra
pathlists require "grouped" argument to be passed to a number of functions.
Instead probably it makes sense to create a RelOptInfo for aggregation/grouping
and set pathlist and partial_pathlist in that RelOptInfo. That will also make a
place for keeping grouped estimates.
If grouped paths require a separate RelOptInfo, why the existing partial paths
do not?
For placeholders we have two function add_placeholders_to_base_rels() and
add_placeholders_to_joinrel(). We have add_grouping_info_to_base_rels(), but do
not have corresponding function for joinrels. How do we push aggregates
involving two or more relations to the corresponding joinrels?
add_grouping_info_to_base_rels() calls prepare_rel_for_grouping() which
actually adds the "grouping info". For join, prepare_rel_for_grouping() is
called from build_join_rel().
While PlaceHolderVars need to be finalized before planner starts to create
joins, the GroupedPathInfo has to fit particular pair of joined relations.
Perhaps create_grouping_info_... would be better, to indicate that the thing
we're adding does not exist yet. I'll think about it.
Some minor assorted comments ...
o.k., will fix.
Some quick observations using two tables t1(a int, b int) and t2(a int, b int),
populated as
insert into t1 select i, i % 5 from generate_series(1, 100) i where i % 2 = 0;
insert into t2 select i, i % 5 from generate_series(1, 100) i where i % 3 = 0;1. The patch pushes aggregates down join in the following query
explain verbose select avg(t2.a) from t1 inner join t2 on t1.b = t2.b group by
t2.b;
but does not push the aggregates down join in the following query
explain verbose select avg(t2.a) from t1 left join t2 on t1.b = t2.b group by
t2.b;
In fact, it doesn't use the optimization for any OUTER join. I think the reason
for this behaviour is that the patch uses equivalence classes to distribute the
aggregates and grouping to base relations and OUTER equi-joins do not form
equivalence classes. But I think it should be possible to push the aggregates
down the OUTER join by adding one row for NULL values if the grouping is pushed
to the inner side. I don't see much change for outer side. This also means that
we have to choose some means other than equivalence class for propagating the
aggregates.
The problem is that aggregate pushed down to the nullable side of an outer
join receives different input values than the original aggregate at the top
level of the query. NULL values generated by the OJ make the difference. I
have no idea how to handle this problem. If the aggregation is performed on
the nullable side of the OJ, it can't predict which of the input values don't
match any value of the other side. Suggestions are appreciated.
2. Following query throws error with these patches, but not without the
patches.
explain verbose select sum(t1.a + t2.a) from t1, t2, t2 t3 where t1.a
= t2.a
group by t2.a, t1.a;
ERROR: ORDER/GROUP BY expression not found in targetlist
I'll check this. Thanks.
[1] /messages/by-id/CAFjFpRejPP4K=g+0aaq_US0YrMaZzyM+NUCi=JgwaxhMUj2Zcg@mail.gmail.com
[2]: /messages/by-id/CAM2+6=W2J-iaQBgj-sdMERELQLUm5dvOQEWQ2ho+Q7KZgnonkQ@mail.gmail.com
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de, http://www.cybertec.at
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: WIP: Aggregation push-down
On Fri, Sep 8, 2017 at 7:04 PM, Antonin Houska <ah@cybertec.at> wrote:
Ashutosh Bapat <ashutosh.bapat@enterprisedb.com> wrote:
On Thu, Aug 17, 2017 at 8:52 PM, Antonin Houska <ah@cybertec.at> wrote:
Antonin Houska <ah@cybertec.at> wrote:
Antonin Houska <ah@cybertec.at> wrote:
This is a new version of the patch I presented in [1].
Rebased.
cat .git/refs/heads/master
b9a3ef55b253d885081c2d0e9dc45802cab71c7bThis is another version of the patch.
Besides other changes, it enables the aggregation push-down for postgres_fdw,
although only for aggregates whose transient aggregation state is equal to the
output type. For other aggregates (like avg()) the remote nodes will have to
return the transient state value in an appropriate form (maybe bytea type),
which does not depend on PG version.Having transient aggregation state type same as output type doesn't
mean that we can feed the output of remote aggregation as is to the
finalization stage locally or finalization is not needed at all. I am
not sure why is that test being used to decide whether or not to push
an aggregate (I assume they are partial aggregates) to the foreign
server.I agree. This seems to be the same problem as reported in [2].
postgres_fdw doesn't have a feature to fetch partial aggregate
results from the foreign server. Till we do that, I don't think this
patch can push any partial aggregates down to the foreign server.Even if the query contains only aggregates like sum(int4), i.e. aggfinalfn
does not exist and the transient type can be transfered from the remote server
in textual form? (Of course there's a question is if such behaviour is
consistent from user's perspective.)
Yes, those functions will work.
One thing I'm not sure about is whether the patch should remove
GetForeignUpperPaths function from FdwRoutine, which it essentially makes
obsolete. Or should it only be deprecated so far? I understand that
deprecation is important for "ordinary SQL users", but FdwRoutine is an
interface for extension developers, so the policy might be different.I doubt if that's correct. We can do that only when we know that all
kinds aggregates/grouping can be pushed down the join tree, which
looks impossible to me. Apart from that that FdwRoutine handles all
kinds of upper relations like windowing, distinct, ordered which are
not pushed down by this set of patches.Good point.
I think this is where Jeevan Chalke's partition-wise aggregation
helps. It deals with the cases where your patch can not push the
aggregates down to children of join.
The patch maintains an extra rel target, two pathlists, partial pathlist and
non-partial pathlist for grouping in RelOptInfo. These two extra
pathlists require "grouped" argument to be passed to a number of functions.
Instead probably it makes sense to create a RelOptInfo for aggregation/grouping
and set pathlist and partial_pathlist in that RelOptInfo. That will also make a
place for keeping grouped estimates.If grouped paths require a separate RelOptInfo, why the existing partial paths
do not?
partial paths produce the same targetlist and the same relation that
non-partial paths do. A RelOptInfo in planner represents a set of rows
and all paths added to that RelOptInfo produce same whole result
(parameterized paths produces the same result collectively). Grouping
paths however are producing a different result and different
targetlist, so may be it's better to have a separate RelOptInfo.
For placeholders we have two function add_placeholders_to_base_rels() and
add_placeholders_to_joinrel(). We have add_grouping_info_to_base_rels(), but do
not have corresponding function for joinrels. How do we push aggregates
involving two or more relations to the corresponding joinrels?add_grouping_info_to_base_rels() calls prepare_rel_for_grouping() which
actually adds the "grouping info". For join, prepare_rel_for_grouping() is
called from build_join_rel().
Ok.
While PlaceHolderVars need to be finalized before planner starts to create
joins, the GroupedPathInfo has to fit particular pair of joined relations.Perhaps create_grouping_info_... would be better, to indicate that the thing
we're adding does not exist yet. I'll think about it.Some minor assorted comments ...
o.k., will fix.
Some quick observations using two tables t1(a int, b int) and t2(a int, b int),
populated as
insert into t1 select i, i % 5 from generate_series(1, 100) i where i % 2 = 0;
insert into t2 select i, i % 5 from generate_series(1, 100) i where i % 3 = 0;1. The patch pushes aggregates down join in the following query
explain verbose select avg(t2.a) from t1 inner join t2 on t1.b = t2.b group by
t2.b;
but does not push the aggregates down join in the following query
explain verbose select avg(t2.a) from t1 left join t2 on t1.b = t2.b group by
t2.b;In fact, it doesn't use the optimization for any OUTER join. I think the reason
for this behaviour is that the patch uses equivalence classes to distribute the
aggregates and grouping to base relations and OUTER equi-joins do not form
equivalence classes. But I think it should be possible to push the aggregates
down the OUTER join by adding one row for NULL values if the grouping is pushed
to the inner side. I don't see much change for outer side. This also means that
we have to choose some means other than equivalence class for propagating the
aggregates.The problem is that aggregate pushed down to the nullable side of an outer
join receives different input values than the original aggregate at the top
level of the query. NULL values generated by the OJ make the difference. I
have no idea how to handle this problem. If the aggregation is performed on
the nullable side of the OJ, it can't predict which of the input values don't
match any value of the other side. Suggestions are appreciated.
I haven't thought through this fully as well, but I think this can be
fixed by using some kind of all NULL row on the nullable side. If we
are going to use equivalence classes, we won't be able to extend that
mechanism to OUTER joins, so may be you want to rethink about using
equivalence classes as a method of propagating grouping information.
--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: WIP: Aggregation push-down
Antonin Houska <ah@cybertec.at> wrote:
This is another version of the patch.
shard.tgz demonstrates the typical postgres_fdw use case. One query shows base
scans of base relation's partitions being pushed to shard nodes, the other
pushes down a join and performs aggregation of the join result on the remote
node. Of course, the query can only references one particular partition, until
the "partition-wise join" [1] patch gets committed and merged with this my
patch.
Since [1] is already there, the new version of shard.tgz shows what I consider
the typical use case. (I'm aware of the postgres_fdw regression test failures,
I'll try to fix them all in the next version.)
Besides that:
* A concept of "path unique keys" has been introduced. It helps to find out if
the final relation appears to generate a distinct set of grouping keys. If
that happens, the final aggregation is replaced by mere call of aggfinalfn()
function on each transient state value.
* FDW can sort rows by aggregate.
* enable_agg_pushdown GUC was added. The default value is false.
* I fixed errors reported during the previous CF.
* Added a few more regression tests.
I'm not about to add any other features now. Implementation of the missing
parts (see the TODO comments in the code) is the next step. But what I'd
appreciate most is a feedback on the design. Thanks.
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de, http://www.cybertec.at
Attachments:
agg_pushdown_v4.tgzapplication/x-gzipDownload
�
��Y �<ks�F��J��9Um-%S U���e[�z�D�n?�@pH" %kS����3 O��%�� +��y����{������s{�����C��^���.�,�g�4���?|�������W��w��h�k�hh|
��� ���K������ ��c���@c���� tS�� � i}]���5O^f���n�-�4-S�_�T�y� ��??}J�~�[��:>2M���L���� �-����@��I�������l���|���xM���y���<�/������o�����e�G�e�i��O��Vh
L 4�@_���a_��� -��
s�����9�S���u�|������ �e��-����N�?���bvz�vf���s�����_�����gN��v�� �h�r�V�����N<��K��9�= 8v��\� L������^����]���as����_u���& ��*
����>0G=}` ��2i� Hqz�������?������������.h�x��<>� ���.i 0���@�`���u� ��+_�s{��2 aGkR��K7�H��i������g;�}�A�� I:����Mz��sB��M=%�g� �;;��2 �]�B�D0�x�����������h��:Hp�5�i�v�0��� ����o�g7�&�g�W��~���H�L��������XQ��U@�����H<��Ov��$���Y� �XF���;������W����$Z�(���8���+_ru� ������L g�QF\2����
3 ���*:[�C����� ��9������xv��f�x�:��o��u�{�9����!�Z���!���J�����g0PW���R��K�L��f��k����
��4429�8v��l^�N�S��Rg�h�s1pq�#1%������\���$ ��G���������a����Z�D%�pM�������-��$ [���+�R�'{A�^�d�����D/������/����ed�0f��M��5gH���bc STf��z��`Ba<���q�`j �]���(/��CCL+��U������&_���O .�;�$��^M��I<��/���� �J�� -������fz�x9��m]��U���eM�Cht([L<.�>
ts�������F���:&�����He~J@_i�j�R�� ����-����i��[+c��}>�&�A�����O S���3�� ``�N((�%
�����F�a��X h�����Y�6�� Z��o�F&����b��x����uU�;=� �����V��: ����`��XA��|e��D��{��{��p��cG; R�`����KF1"��=����� ;K��uf��<%����p�w�� �
��yV`�\F��������Ap��7�(A�biS�C�Nj,S�<�d��TH�h���C� �S0�
q��D� ��B��j&fU��e�s-�/ ��1Hn��D�&���'�M�=���� ��Z��p�
�~���~H4��~�8���������}���$.��G Y=C ���'��5��x�6 !��v�r�+grD�P�?%{�Bl�3d*D9H[�5x�}�_z|��la���4 !x��x��`��85�a�����h�0a2� 2U� �=O%U��@�����W�* �O j5�=���
d���� �]��c����A��tn>����tX���S���a��)�n�y���,�Mo$���h-R���}I:������F������6�%m�d���K��,�g��`4h�$pM��2�,�s��e��$r�5�Jq������u}���K)��My��� �f����7���rw�^3�T �]�P#�
�u#������R8� ���jG= �=a��1�����:�Q���r��|s���������>6� �����N����
>����V �K[�/; ��
'�����V���<�(�O f3]���w�}�b�v��0
p@��L�����`��j�u����#A( �������� ��yb/<��y ��_����q�.������-� ��:�� � ��� Iw �
c����t ����pti�[��c)�pa���P�9Dx<��G �F����Z�]���]4��.A?A��!�� n�1 ���� f��pkkJ$�H�{�{3���C�_J�O@�a~ Y,�aY
y��������(�d�<�r��_v��=�}��|s?����}{w���*4F��[�� �n�P2Z����{�e��x�.�b�b��(�� a\�] �!)�d��B��������,%T��S }m�06��M�3i�FL}t�(4�P�����-�Ws�:�[��*JX=3� �J���,�)�7^��������jz�����������9��� �y�i����
�������7p�N�����1�l����e�[82m�����?A��B���}rsu{3����wM����t�Z�X!X�,K���P*uCm�M�\�e��[��O��oh9\B��j�Y�8s`�3`e���{��S1S���X��6�U�b]��ScI�`�yZ�)�C�Vh��3AM8}�������cC]3zC]�jD��0�L�/��In��8 B������� �=9: Z�%�J�}5ga����_9��d1" �Y�1MW�T�v�"���=�7�@9i�Sk�Q��QU -�����r�f~}w1�~��,E�
.��W� 4���G�������eW���-)��EQ��Q�Xw#
g��B���g�3liZ����t�Fc��[��Tg�J u� �2 5��������
B%��B%�K09)����@���-��98� ���g�Y�lW���~�G7�������=68>n��2+hU����=Pfy:QC�z4.�n g �lA��a���az#�7�� L�-���Y��>}mm�����qm�k5��m�~���mi��~5������e��%"Q��*e vA�c�.���vt}����I�`�B�SN���y��gp�s��n��������*��*Y:�u:�\ �@�*c� �w)E�@��/b;P����6�/��50 `��!|��d�� ���e���E-��y����������� �=s����!�.���m��� �. �eH� �z,t�U�O ��������[���R,�UD�\���2���\��v�s?\�����*���n�1wv�����'� ���R�C�BE�#�6y��m�$��b4��}n����BU�U�j�W�j�.��� ���Ul !6���D��Cwl�� ��IXMwQ�Jj��~���u6K[6�Q��M2�������%�b�H��ch[����R�hKk�K���V�� Ve�nTY�����;�V���]jy�����#G��JH���kJ�����<��K�M�l ����
Ol���]���w^B}���: `V��_� �X�x� g����>��� L�;�g� ��6:0���PTg`���S*��plg�� w![�~������T�q��� B ��o�w �
�- ���y��g���������� O5���\�l:J�F�f�HW ����7O#R��`���e1��{ ����7�!�m�� ��� ������U���"��5@S*�y^���������@��������e���*��;;�F�1p���<�� �![�0Jd���i�M�V��7z.����w���W��������9�� �u������k�����x�C�� N���Z�����\���/8,��Z-����p��/���Yh������������Y�y� �t�lf�iC �q�z�rY� ��<���&�i�����b{�H�C��������W�t�5�� ����b�����XvF�R.�VS�
U��.��F�
s��p('J�����������Ye9eh���OG�&�_%|�n+��]��N��>�
#w�&�JkQ����i��j2na^��c��|�9��&���
4�p@
F��fu�s�;wWoX�bNz �[��A�R�-������d�6�x��%��� E��� }+�U���#�a�2@e�wNP�S�|7������ KD0G�`!K)�P����
6=�?��U�(n
�s��n�}�\]� 6!��S��0�����@YR@!\0VUn��u� �d��<��� 4|�"/� !�f^������ �����Qq_��;�-"��%��#���Qd�0r����������&^��z �� ������FT2/ ���o�I�K��(�8ry��#:�N�=
@b`!'��R{9�P{���q����O�y�. F@���� ����������>N�c� 7��
�)�n�L�5X-�U�o�c�'�K ��� �2d��)�tN#pN�M#j� mA�5%b)�p@����
��!�_.9��k9��[�~.K���"%8��\q������=.
��r`�,m�P�X�9>@�+$���X�o���I^P/[�RB|.�F� ��3�9 �u�X��Qc{ J�D�
�
<�f9L���>�|����4�^.Qw�'��,M� F �!�32�G!��� +����q@��.��t�`���aY�9{AD
�2�� k0�0 ���H��v�]�H
���Ipp�;��9,� ��q�:@A�
Q�!;�� !%�r b� 6��x��^v��LC3����q��/v���J!?�� ���z:�|�^C�����R������YXS�$�@z ��x�
��%�NW���-�d�zX�S�A���qLGn��:ta��M+0� �lj��c�@.wg���� ���������y�P�\W%P.nb���yS"3rz0�VB�`�� X{x�_$fZ��g�����C�r�UP���$fD~�����_ �RPSP%h�@tI����J(���"��Vt� ��D�ZKtiY;r��kr�SE�]�� �R���~�P6����L���1� hK�y�Y�n(E?�a�d����t2O��u�b=�R����4Yy��[��M�W�Bz�b?����0|G[U+������j�'�%�R�������X� � ���.��T��
-���Tt�\��)�#���Jld���@�� �)��B�d�E
d� |��:n��#�_�HJ0 n�+;��
����� �)�y���7$����?�������=��I9������x���q9xl��^Iu~�� �*�� ��1<����"h�l/����gw��*@����!W!!Qy��<C��?# ��%����0�, ��J����6Ct�(���t� c�n�la<�D�F��%Ix�[�'��a�bB��.U����A&�����=b��}��%r��fb,O%D���A��G�G����� �$1�m,���o ����jy^�LX�G*H�~!n@h������S�x3+��
M>�}B� 2.�1�*(�rm�(���,�6��$���]����w��0�y��Be6�
�, �V`�'���� 0E�;{��
�CIj ��Yr����l@�Y �#\P���>�N�
��qvy���e�<CX�f�9bGrLE@��������q�5$�{�=���N^�c'���n�dyU �SVa(2A�
a�IK���M^ �R�������B��
A;���t� �>���qW��M�����L����E��2=;��#������"��!�rS�)BLp� �!������������O�ZL�jh1�@��)����'��e��q���������2�P��Gw AI$����7��a�`F!Z�m��U]�C�61G��E= �0�8c:� d
�!����I��N4�X2C��v,�<a� ��$�,����$� ��s��R�� �#2��B�:��+���Nw���S(UZ3��l�����A�q�M� C�R6�|K��@�4\���]r�v�J#F�v< W�(���!���/�J+
��\��T��g��-<� ��<\<f�d�1��0_���q�<q���Y\Y����G6S��F' ���O���@�BD+�3���J�G����c�#�����q��"�qO�6qT���bojX\F+�����:������� �
�x��py}w?��8}���
��gl�1v(n�QI6
(�N�� ���;n#I���� 5�J��j_Z#���lk����������V�dY��PE�����e,��H��E��c�n��DFF� �x�n�����Q�� �p���I��[^12��o�_^��M�/D�L��
�@9��H{���$X����G/��T�d��d "���(���*�TB�[�L +�p���@sCPq�$��:�G�H�v�f��5��3� ��no<T�� ���n&`��!:{gG�����zGDu}#u8+���c`�w1�����/2�B��8_�1������n���� ��j�ij �S�3��G9��'A��� ��R?@��a�v9���C� ��<D��VI{�i~I � �g�+M��v��+�S�����G~�|HA �+IST���4���B_��� i��se��S��u��� ��h;����E�R��� /_9���-��q���"u��8t��_���g�5b6 ���� ����Q%�#�Pv��8��X���:!.� �aE��W�p���B�����f��B�r"�P/�� �{���u�o�#`z�&' �(d�dZ-��q���Q�����O:�����;mT�EV�
��� ���V4&��)�.!s���2 ��
�t�����K�V��F�(�k_B
�����C��$��2�o���������� ����������@�^/���_���n��l>P_����/?�������������)���hv��{�_!� �����Jt��!g���e�b ��eN I��i7����<itO��G��K�
r�}�]d�� �� ���!j�d��@;qO���?m�~AG~�R ��#��&V m������� ���!';���_�OIq0O���X�o'��a�j��4%�C�1=w\��u:�M����G���]I�������4�Dq9^m?����e F�mj����a8�y�>Qv1��N���@�`�7�C��A��I��������h)��P�r�%R!�2Q "b'S�x�x=��`����z�qr��5
J4#���:
��� ;m���S��&��S�;z�2hZ
���/(ac�X�&��b)���P%��E��I{�� @u:2�����E�-�����l�#+<����}M ���b�.���h��M��1 ����'/@� N��A3��" �������_���
~�;��� p� ?]������6���� ����i�A/�'�E2���2A}x�6�����4�X����F���$#{9�����&U2Hv�
���*�p
MN�|���1���c�2�S� �]� R:~a��������d(p6���*�F9n��Z �z����4Y��Qo���x�~Jx�P��� �F���/���=(f ���t2B{4�� �&��c ]�qo{���0[lV|� �G����Mj �]�"�C��:(C�%g�s ���=Lm���-�h�>z���(E&}111W� ��|o��q&]J��>�!O'+5(P�=4!M����������V�w|�{n/� Z� 4�H�j��d�VTj��&�g�
zK��}]ay6�@���:"2����c�sh�v2�����<���u������ �
(�B��ijr������+�g���Q�F���i������>�n��Iq
�f��N�G��g��+b��&���GRO���Y~{|��o>�=�eCu��R���H��8U������^��D����"N{�����C��A�%����]������/��,E�Zl*@��T� ��O�'H�$���Ygj��e��C�\���C�zv����!b1/���^FO��w��&�hF5��;�����UL��)]����>sX���`y�|�Fq���T�W���b���u��*j'gsPP���/ �e���I6�X��UB�:�MPB���V� ��d�I����L�C�� [��CJ��X��y:]��������$�<"��l ��Q<0�G�8�P�/w:%:�4K�}j& ��d�4�� �oyD� /n�� g��q.���c-��m&V2��d�����jH�� �V�; ��e�b�AC ]��4��~3����=��x�SJ>������B����d�F��M1
[� ����}6$���t����r 7�('^��� ����$DF� v�������8;�c���5��{ �����A�Kf������"��}���q�Wv�� ����{Ty����?��[^��f�V$�;IP]���zqU!�C <�� ���r��
j�E���h~��+$�ew��� dG qou�R'��KA^�H�>L�l�[+dF��EN!]8��]d�s���. �b e�!�AIfm5�z�
5���ej�s1t ��nl�i}�f�P� V>BW���6��}��;���0��3�
.���#+��{�g
�9�����Sq&�E, ��1��_-.��4Y�O�!�����9���U �����Ofk��hOdrCQO�IV�1�Ge9!���OZ|������u� s����)�Q\C����E�Z� ��#T�_ �'�^��u�+Z����^�N�o�|��#�l3]O�� �+ �����35S��?����� �/��(��Q������&g�!}�ta��c3C�� ��r�y�b��{f `������W� �];{$CN*FC��}����JM6ufF�A?8�f������NML0tg�\Y���v����:TtA��q8������,���������2� �������d�w��8D�[$$`//c
��F��Rc(�11���</�W�rG��]Q��^�_4.h��[I��o�r ��
�FT]�H:��
B��YF�F<,�kB�}&����� Y)"F��4C����w�R�v�]K����^W�+������ :�q+�Z�YV���?�a ���oo�x��m<0X�W���R����3F2�%x��@�9�=�&��o
�c�: �$_ �X%���?wf<�IL,�pW��f��7K�S2�eC]�J�E��w�y�0x�r/�J��Wp>3��=�HV�#R �?����D���5�p������N��������x��C
~�X)Rl�y������J�����]�3���J�Qp �_����G�����!����k;B��ia F�F]�(�R����W\�@[��'pj��X�'f���������
�rpA���k�A �@B}���f���
k���B������#���cku��:���] ��t�O��� 4��� �4�����(y�������[4�/O����M�R�X�L��� i����xH�G��V~�U�W��@$;�����m8����o���w�6�j�U����o�����e�+�^;=��k�n���e:�
oM�*{V~
�j�M�'� ���8�A�P�����jzHn%�x�f���*��j`���u2]�=]�
���59�������'� ���~����k "W�@��� ���N �AlkH���)o G���+v���0��9)�D �%��L�������d�^��}����S2�, ��-(�-!��_6 U�jw���60�;]� �s������lb\��2�
%�[!}gd'&� 2��:r��Pn'?�����Y �(Arj�������h�F2 �����p]�U�2����_����6�%�����g�)x
�$�!v���d:�n��� ��?���i�h<��X�&r�&�6q%6nB8��hlbb"�d�Wt{�K�a�81��nB�2e�E�>��]"��l��<� y�����aw��X!�*��^���A?�1���Z� d"=��>#����O�1�R
�������)JD��9�_���g:#�U����b�� ?�X7�o�#��&�I��k5[�aGNT�{ y��f����9I�t����L#P����������o�H..� ����(���
���n��I��^<�+0T�8��s+�����UZ[I�\E�/�-�"r>�m�d#2v_cCj�
�� �U�!��A��<$!^a^����d���d0����M�4YqL*`�d�y�b���$��]�A�6�s� Bh4 �!FR�q^� �?riT� ��a6����0 G�E`��p��c��&sZ�D'��=�9��W1��v�[��:�bP:����K� O�W�gd�n/�x?���qB_��� 6�y�A�$A[\��(��k�WQ��,o22���]$��"� ���I��3L [h2���f����^�z�9��Y;�H���uB-S��s�=Y���]����H�?���S�,7��WQxvZCT�5>���A��7LI�1��P�������`��f� &��1KV�S[� p � "g�f�>��X��C.��)�� �("������34�9����__ _�~����� }����� �y�W� M����������%Lt
�g
e��(��1f��H����7���4a#���B�@�\��`O}�Gb�? �I�
r�.W�CG6�g, Z���__���S�k*�x4�%3M�.�Aw��f\cJ��p�[8H���c�#�4�V��|X���\]~)6���v���4�yA���xG���@z&;)�edC �� C��R�33�<�����v����E�k���&= �
�a(^� ���T�7���?��q4��P�W(���{���t��f�>W4v�� ��
���4!��&��A��G����m�8�t�8 ���T[�.�P�V��HW�:���Ypl
�"��xm���e�U�Y5���8�U�j����Wx f��vg_�~��������k�z��4r��:��Rj5�V!���(���:�*������2���$k MQ��t �
�W�9�y��n�R�: �����U�W�g"1���s��P��@xJ6g�l���h
8����c)H3.`X�����1w^� ��5�I%�?����/ �� h?3=�q�� } X y#��@� �9�{ti�����A??k�����+*��$�H���w�2��0{� [�����������E����2������d�A�#v�t��>�5M^�~�����C%P���;`�NGi�KA�����+ >�������o
�NX:����j�sNbh ���������_r`�$ l����5��;�������R�P>��O�9�� �en2�i��@�KJOg�>�Lk��H��Ti�� ��O������/J]��..��l�
;��c��M�n�X�F���A�EJ��]JJ6PQX:)d����;��:�
�����dL�6+' �����������
�:$�;�X�6d5�Q(������������ �h��"� ��������U�H
�q ���#���80��2����,�U����e�=��G
n��q� �Tr��s��{����g�0�8�R��K����Cs��:���P��sr:�t&I�K��e�p+bz�?u�+�)�vS�<���J
�VXeBE��m����5�>�4��+�D.OV������� ���XM�0�,Ey�R9�|���J���t��\�������`m6N�g�T|����N�o��p?
���?���G�~A��"�q�37�� A3�LL��]m<4$T8+�4.n Bv��L�"L2���3�R��� ��E�> S4��Z���2BU�<�P�F~t��2��r�ECQf~8�v���m���5 �d�}���*H��cP�w��:$ w{ �aKz��� �l$��b��iM���l/{���t�|H��l7���Z�:���~ '�|�����:K�y�����b��s�,�v��/��>��1x�Y�}�������\p�{��2*��\��p�k��o�T�������!��|~&��p�d���$���
i�q V����!EG Js�j���lh��k�F���0!Ig�dB:��S{ ��Y�2�Q��M&+y�S��� [
=�����$��?�U��KV��G,7�]��FP;D�R�~�:J,*� ���Q[)�kY�����L� #ie��X�vv��B�����/Bg�hy!�IO
y>&�� <���f�2�N1q"c0���@������D�c��~_]"�1��7����/&+�v.x ����b�����,I�\ |��$r��U(���m ��@|��XjW �q��8�x5U��=��)!o�3�8���R^sF�_��R�)���/�������j�Jl6�o�6��o��#��p{�N���5���oe�������lE��f��I��I {�����CP��3���cE@f��.yb���
*X �J fz9�3t�P��h��6)���I����\`<0)�Yk �� k6��m!2��A�m���1�y�P����p��3*���#[,DN�QZ��?���a���,&������:!�-|����� �������qN~n�f�����!'�H���k�<05�d��!Js�%q�T�f�VPh�c!������ ���8����� jPR��W"��f��9�IJ1�� F%a�WP�����!����9Z
f7c���$=�C�3�?D�f� ��V��������GZ��� S�K� ~��O�;�� t�K8r4\�$z�n�"E�������E�UY92�����}h����j
K�����%�B��:��P�>9�!`���~�(DV�J42U �����Bl�:�$dm�����������:�����A �4��7�R��)v�dNy23������$��RP��s��F��!5,��]���xL�AT�w�����~��}��[Sxb>K��!l�:�wZ;�������7���5j:^���$'�����M�W*"��ly/��m�� �Z�E�XV������5���Kkv*D�@���� D�5�m H� �/N�� �g ����@�&�� ����S��k���On���u�J���t��� ������EQ.�
\!}�w����/�����_ ��q
���z��#)��e� ���.���,�Q�����^� X_�� +�#1�uP���u�:�rd�� ��$_����X���<Dl,��|s�_7�y�(sA��N�
U�P�m�����F����G�O��@Z�S���-�nH{�g� ���:�R���3mm �bZ�7���?5 *�\_m`@�W������e Z�VB����QTR%Y1W�W�"5 W2��d&�������\�� ]��{m _�q��.|Z��m*�k�6 ��N��3� ��
PWDM �
T���i2�M7H��|%� �=�:�}��8Kc��|�L?��U����`T�� ���j7��X�
�"`�� �����;�aU ��� e*YYWE�y��;�;������0��d+�~�a�*7� �*����(r�� ��_s����:�6�6 ��)��
� �c&��O$�[ oY� 7���C���g]wb TBou6d� �H���b\�{ j aoVh��`Z��F����2L�����&]�(= �����X;F�>��`�[�i�E�8Ce0� ����E�!V����V=j��p������ ��� �w�V ���|l������&P7 �lI����;��:C�A�<0Le���Q:`m���OD���V���s����q�L 8�G&�
��� �x ��7����&
��5�t#K�E ��� 5:I��Y�E�3(2����W�Q`�X,��>�9�g�dE���L�*9u�M2c�!I �7��z��<r_�h�Dl�aMtp�Gqc�P
��.�����jm���sA ��%gF�!���,� �l�����V���RO���1����!�&����yE�O���,���^�Q�*~���?
��a~@NNJ.(�����=�) UC1��` S��M h����j�Z_��$�e��n��F�#o
������dM'��(M�gk���;7�������e��#��&�VJ����I��F�+N|����E:t- )�\�k�.�[�j}5*�4�Xd�&f��<7� � ��E�*������� �)/O.|q�?T��j��b|�% ���r�4�n����� it�N�a����*D����� r&����t��G������(S��H����_�C 0���/�_�Di�9��>.�f���u��#'��&��p����
*[�����1Q�|��8^��(Y� �A�N3I ����Kb��l^5 l��%��I��������J��cShpsB���fY�����P��R6����9�v�2t�e�:�^(J���aB���s*���)��*�-.DM{0] _]��m�E�T
����6+;p�A���;�Z��K{Iyr �����5�I��@��g�4������[�,2�D1���b��d����� �R��izg�[ �c�� D�=�%b� �P�M�N=n�;MCFv�����;��L���kHV�{��&2q��Lc �\�b-�h���j�N� ���k���BuMx ����0~~��, �|v1���6 �d�����ji"5�K
?����su.���;>�a��lIa�y�[��t'�����Wa��<i3�
@�A��������{n���� ���P�
�LD,j��'��S�H�k+( �.�B$#��1y�e��8rW����T��6���E��,_����,�[�X ��<G��Z��T��0�T��{���#Q�Bc� ��t��+}�A!��5qa�5�Uh�W��]pm�
��(�"���@��"U�Z���I@��oo�,���@A��|�dB�m*��Y�@l*-I;\���g�����J�����46�< ��}"D)������p�KQ�1��W����?��H�(��"������s��a|]���^�]YB ��-�H����(�����k�P�=��71����:D���x��u��
����C�
��_��Wb�������8q��;�f�<4}d��u�NO�Rl���.k[s�d�J6a�����<�L�q� �>��M. U�p)���W?������� AG(s���R�� �9��,` R
1K�����{ ����7��TJ�F0]������<�~� � ��6��� 0<�aa��Af[� &����-`,���]��d#.����q� �E''��%��GkW,�q��\�e����OlF0�$
8�
���,�S�A����d�TS�D�����Vk��RD���U2 ^�����f�0Z8��I�D���:� �v�H&S�C���W��j�� �M����7 a��L�$����+���IC�=roR!�r��A��<j��.x!H�-0,2����� ���Y��&�rN��as�� ��Xj��&��f� R�4c�
� Y�N@��� ��s��heAf ������W�h5��������Tt������E� �*cr����VP������nO�i^�:� ����7
��������_?��|��=�%�BI �\��C�x�������|��`>��n����4&o�����^���WK���Ot m�'i:A���X
�������2��J������ �<r �xA�F�� ���� ja��Q��fQ�W�#5�B��"��8Q����rs�hHL<rQh.}��o���$��h p�C��� <W�3FC���� �mc7���B���=��/, @��y~�.b�>KNN��6�)$y4=���F�gLj���yYn���![��k��4��H� �����f2��7� �dcMAkpS/[�=�@���P��]��)u�� �i�[�� LK��3Lm�H�2�T$ J�5F:����B���#�
��^$�)���3�nV���Q�\NRJL1���n��l�������(]��x����o�w��kbg��E:��Nh�
�B?c;N���������
�����:�8Ma�C�����"�t�.�g��(6i�� K�����/!OLT��W(p��m�
�����C'p}+�Chs���<2j"K R� � ��HrtD*j����[ ��j%:��"��`u�����v%o�� i�qD-N��,�~.F>1����a'�PkPP�Y? SD�8��t 6:���X��|oF;�"`��O�h;_%��F#,
0�wp�9�y�j�c
������O���'� �@O�Y[} �-�� %��K�
��G�R1
�o�Tv]��`��3&�nl��G�*=��zp��L������� �=��!�� 0^����h�[C��e��{l��b��W:����x�F�W3���s������{���t� T��cL Y \� �pi�T?��CVC�a�G�4b�[)]&o��1���(����o�f}9�{8��T 7k�I��WGIBL�|{�&������`����k��}�-q���v��-?^�6 ���I����X������\ �����B+(�3 ��rb
��U�rA���\���^B~N�����������Z� �/��
�� ���dW; ���gfj����ejh6��}�4 ���v������� ����+�yie�D�?���m�aO��:�D�=yd��w�:����� �KR��z^�� �Su�_s�D�\ ���@V<�*� ���pob;���}! ��?^�u� E�K��8*ak�� �}&wc�&���LF�E& an#�^����t2���i�����=a����`���H
r���^�Ex��I�v��I g2|0p��N`&�� <����_�B��P-Px�������y� � �c���k;�2�>K)k,�^�F&����lkz��n=�~����z�<:8�T�� �u
�+��63%�|�����e���p���L"�u�����_�'�9��&M�R&i��B` ����Tw��W�,z% �� 'M���D��t>���;��M \O� YG��9��t����
a�v`��D�n�%~���y�����VhG1�\���` ����o>{tD���� C��(��O$�uSVc�N�����4<\.�T������a�Op��|��D��qm����T��b=Z9 3Z���sa���a �e ��2�K����Da �s���F�u��e� �� �`q�Y9�m�;2�Z�^����#jY�O�|��-hnL'���R����
���������&9 n���F�E[�N�������3 l����n���
}wF4A
8��I:e, ���7���o��~�:3��~��s�� � ���w�jiN�5'j���e��:X�m���� yJA���%'q�0�#)���\�~��<<�^�2o;���N%, ����r��K� r�=t����9[�gT6����4R��I e�I���aH(f�{79��o���j����rN�kzG�V"�� ��'��/\
����q������c�Ey �,�+�\T4�g�����������w%�"Ai,�d��3"�E�Y��#pP�����8G���H����I�6�m-YM'���;
g��u���!�&Vy����Jd�d)In {V I������������*;��I�|������4��F_�@���� l��j��=��|J~�N��&L���:���,�ut������!{�6�P^v�"q���w� &���� ��������|���(c'�V��l*v�W ����`'�z��DY+���:��Ar��<o
�����I����#�e����<O&s���C� H[�E�� ]1�h�m�w�Th� c����rC 0? �|��x�`�YMjP���}L��)J��K`�]3P�6����$2�1[`RQ��`�O1��&��:H��n�8_���4G��*�,�U�� Y��.���'1�c�)�i��[��FWK�)e�Fs�|<�������b��$���:��j����I�� �-}��2�����Z��$�������� ^����Ew�,���/K�"�UDQ�]�TA} fZ���I��6��Nk�qk|r�i������m��G �l�1�����@H_'������4�k�#:<�+���� P��)��y]U����B?#�Q�&r���a��{i�Ym�py���?nI�j�8{�!3~���T��������E�������K��9�X��
��C. ���������Mu���~�_zte�@�)�T�n]�����;��v#����� �d���q�� =��w� ��� ���^������E��+x�J�r$�/h�����p{ ���c���rM�V9c�(ute�W��������F�]F���{��t������U �<C%�P��C�'&����� 8�v�r��(�Z�&�Fb����I�+������ J0������G��.)��0A��j�t�O�h��@�'(>b��N�&C�>�~��\�@v=RQ@n ��l �BYK��?�� ? ���� �_���o��~H�q��j�v�]Zoa:0��\�7�^�[��~��������{�F���v�Tot�������|���
�]����X��O� �@�����z�F���u;�N�� t 4Q"�v��Z��t��������O��?~*�����{�G�H
�74=h(������k4������?9��N����>q�B~ I=�''OA^��/�/h�-y[j�m��z:���h�z]�r�z#����������w�U:9�?T B�NGI4����B6��� cP���)�� �h>C��b�!�����%�5,��Z��V#���)���"�l��:����d{t���&P�l1K���
������`��u��O�Zv���"48��)~ �H�� vI�Ru�I��6�Z����'��zu�������6��� ��x�h�(����[�5v��m�Fn�&?\�&`�GC$!��bt*�8���`���-(��F������loM�MT��T�f���k�x���I��|��
��� n���<q���*�������r���+U ������=��JH��15��o�<~���
����g�.�}"�^����?����p��� v������_�u,)<�+�P���*�����*/��`��!8�Nq�q����p�h��$��js8�!���n��p5F��|�`�KG��O,�S��:��S4r�u(�YC� �H � �O � ��/��N�)a�U-.��0�?���A��?����Bn�h)��O
�Y�G���` S���c����� V ?M�S��U��mW�&&��������34[-f��[��k�#a��+)'����#�Pw��{u�*� u�JA�8���Ml-w`z �,��Gt]������~�Y5�&����CC�jy�!Rx )W��HF��t�0'0����q
��������_�N�B��F��
4�w���N��d�d���������F"�!j���b5�*P&#R
�� G[�.�% v:��M�C��!�&����H+b����I f��`Z�,�|~t���h��F��-�����<c��[�kq���)�����R�-f�Eg��9�E�5lt�.��(o� D���3��R� �m��IN����X<X/�\ Z���� 4c���+A�� u���}��0 6<�m���XXw�w` Mvi��Cj1CG��$��c��c�_Q�Ka���w�� ��A Vl7����P5vW � �t��L=�<?�����K3DC�V��3>JE0�$����\��+������}i��NH�X�9�G����
����bs�^�-1���R�'� E����UW8N9�=Xa2�)@�o�@o6���o5������tu���C<V�� ��n��/��l�s-����gws���@P�����
��������e�`v ������?8P�KVX��j�����!�� R��koC��->G���7������^�t��uu�������41�y8ML�3��N���pV��p KM���� ���4+�Y:Ipz���m��W��o�����t�$�6�oo �n��O��B��.�B]\�D_B��f����?�0CDR2�h��z{���U(�^�D�3
�u���7�N�
�)���Z��]���\K���*l����?2j�e��r%@*r���5{K��i��A�OmZ��Q\�v�a�kh'(�P�>D�h'���7�J�B�H�F���~ �K���l�T[���\}�P���Ms�l�!l�wq^��[qq��f�zjf��] ����:�G�W]x,c�0 �f�������w�c�����^�E��V��y�x �7O��X����� wjSh��c���*��y�bJv���)�L�,(�R����� ��W{vH���z���7 ��n�.!�'P����]9�m��������������j����g7���S �u9K@ R�.��h�� �o�5���������ub[��k|rT}~�*��J��+��|�R�Il��xQ������2�[;D2����Ys������ $8�V����Q({� Tn�]M g�%y.m�r�FJ�"q�^���H�'�����G�}H�l��j
$?�y���������L��d��1��V�S��?�"#��=��c ��3���� R��!G�Z��a����-4mN�/7gW���p�L������+�����I?�����t�=����o��m=,�q�x ���O���+� %����6�B����VI�,��[ ����� H���_C=G��m���R�#��&��a�);�A�vL+��ktAy������� &7`�|_��C���]�* O���1����f+�������(m1<����� �!�J(pz������� �a�/����q�X1)�K�,#
0��|�?W�j.� �6������.�8�P��
:���5��w�����n �X�f�Rj���
��������� ��[b.���i��W3�5$��g�3 �~h��]�$��"y i@���s
��:�W>��N�K�*��_�#��O]d�����Tq;f�����o�> f�OrCah����xa�uK�*vY�c�Pa�:8��X��A�}�� �8�F
�}jI�~����z�@��d�j��� .���J-.]��E�R�!��\�"b����4ZuQ���~����6W��^r\@�?*$BXSe��5������*~�Q�^�+&� �,��(M6� �C).m�������Rsu_�@.���p���o-���R �M���U�� ArR��!��I��w�9�$G�NnC�������
�m��6 n ������ ����-��
��9`������G~)i�#"�f��� ��R�2�Ev���G���O,��G� M�w� ��/� �4!*��u�����-��jwq���(*9za~��$@��?h?k6`�|� �����&1'��9e~�7�����;���� 5m����rp�n�"�}�*�e�{|�ny.[=�O5�i�NgZ*�ZQ*�)P�\E�i
�/Neuu����j/9�w��e �\fB�Ow�m�������������c]�W�5�
�>��A7�?���%l�z�9����rD���7t�N�n�FU��<LUU�"Z��/�T��]i��V����j��8M�]��w{�[U��������YZ�~��Z�~��t�*cAO�XdF��DW)
��8���'���D,���J�X\*�����1k���Y~3��H ���p��A����|�@��a��[ar��hV�!n��b� �*3�@V���k,�[��b�� ����� �u���������/�`�2'��c,r+z��
1�� ����[��Wq���D[����p���<�*��'���<������)� ��G��{k��i�&�����O.TD 2p� ������a���}�q��^r��,HPkN \�%��Z�� k� 3t�g����*z��q�=t"N�t���s�9W
�� �O>�0n[����� ��F��-�3��V ���vR� ��f1����,����k��F�-��h����nA����V;=���'��l��w+-����^]
�i��� �X���n6�=�D`�o��C(�
�M6sYF���2�Z���h�S >5�e�VcH.L�X��V��"'|�s� BZ�t��/R��>�"5�~��� :��6rq�[����/0� �c�"" ���q>��Z�����������S �XS�2X#�;�V�i�@, �O�)������G�bSyE�'U�!�( C� vp}���y���al.[���u5m����.�
Q$Og~�����R0�]FW@�"��!/���\���;�4w q �B ��P��Me�����f�������i�H�(�e��0�S��V��\/��-��j��V?pE�s��b�#�� �� o�N���_�S�VKMl���gb7����N 4���KC���;��]�j2�[(�irv��&��q����s0��'�C��j����-�i�~���Ut���@�����O��k�M�����W���F )d(�l)���~�����5�j2�����'S��;O�����i�=��^�Lkp���@{��:-�����m�5��Z -��{�U���Y�[�8���\ 0-]���J�A��� �^��u� � �m��"�����I��"g{�" �S�H�_��P����p�;p�OtCn��f+�7�^��'
�p�~�7;q��?� {*�Kt�{�5��mW��m����Q�Vk����i ���������G�Y��`�8�oj*IG�#s���/�8| m��?����o�� Z��Xc����V:�8(� ��s:v
����%{��>��P�v�Uq=W�H�f�)���i�����q�����c��������:3�dlB��#�������|����Y�,�r i��}�s���X`������|\�Ysp�����oe��5,��n��l�k5p�? $���w\�?~� J��V��_{��(��nu�v[���0- �gt�E����x����~�]��mC�:���#�{.F������o���7V�
7��r��) �z.��DpJ�k/� ��'��I0�R�!�"0;�n��7,0;��z�q\ 'h����0/�g�T��2J���e����#������4��� ��l�V�$�?
w�������$��qZ�u�F+9�~
l�[��m�o������(���l2�%������oB�?��4jY��*Zj$�T�N������X���z+.�
U�VYz��y�%�>��r/H��g:���,�AL�R�������a�����V5-�G5��~+�&����3Q��d��9ICJ�i������* �/J �L�r��7R 1O�����w��U�~�K�� �qr%��3�b�W�a ��sC w�G�u��B�*G��������7k�n��<�J���J����e2?OW�u2 ������NR�7�@I���l�gCZ-.R���4'�G���$�0��r��� �j�#��� �TNs�%J��9��������e�����bAN~�F 2}JC���T )�%;_\����� �����sn}����6e��
W5���+��n-���rf����2�����YFz��&�I������d�����//��$=�w��j�-�~��{G��&��*n�{�� n�{���^9v��z����H�?}DL����s2�j4~����6�i�����y�`�5WA)!��������|�O�pO��:�~<`�}z}�~���F����/����s����r��E��~g����O����n uE� ����!�pJvK/ /)��A� S���=E����-w��PK��9�*+,�������jq������g���?x��m��^������ \k5���^W�!X�F�� �BDJy ��-���t X����l��f���f'n�[y/���ZY��+ @��.���5��:j�����X�?�2��n��[bC������.: ��� 8���S��<�����*{�E){����-Z� ��d�_L2�/3Y32@j��c�t���Gw��[�B �4h��+�|�m;8WsH � l��:,:�I�I�_w��[b�����MS
.�j"�u��-���|��0�`��O6����Nd�v��5���/��4�-Vp��mu�4�:�:�B�������2�'��r���c&�4?N�\���K��d�:�� mH��D� :Q�V������j1��Y����* q��)���W����>�\\��S�X;-���vH�EG�?����_~�����)^M�^���*��C*��6��,(�)�v�t�+�������B��l|��m��������v��G�H� &��9����� \���eE�c o��BzWr+�B��Z/��t�k�a���d>Zo���mH
��X|��f ��6����������������|L��t~�� ~�L�B�r��|3����[\3�@���?�u\�����=�s��#��
!�E��Y}���L:����iD��w�f���i�R��<��+�s�[� }?��Dq��t ���j�k�&�)Yj!E,Zi��,�����ci���
Zy��VP��^H��U��`w���zcw�M��A��P!mz5\���T�v��h��P"��"y���K~*\*<nHM� ss����[e��Z`��{�'�{ ��g
�*�g�t��(��(u�����5F�v� �j�������UgP���7�1 ����K�XB=7(UP���T�)2������f���bI������8Z��������WPQ xr�����^jj�j~�����3� �:�1]C`Z�����\d��D������ ���"�:�|�-�/�����}v=�uj�d� ��r+ <���� H]��^��c}��I��J��V�U��~�e���������qw�FV��������j ��I���N8��PH���69��������v����0� �jwDO@��X���v����f�Q��J��z�M� �a3�*��� /�(��=��K~1 ������di���)5�@���\�I@Uj ZL:.����
<C�"r5���? ��IS����8����(\ �����z`=����REN�^.9� XJd��� ��b<b����)-!7s�|� c8��E_c��?;/�Z)�
��B���4�X��"�d�/8��l}��D��uU��n������ �e*���X,�W�����B] ����Y�vzmP%�E��?�i��j����G��f��<�(F1Q��;<����`��3u�������Q�GM A3R�]������_��8���-���"����6�?�T+��]�����ok��&�gA�`����������N3�x��"h���|����R
2{����i�E��p�E� �Zt��#�?�)��t����WB{��4N�)T�W �H�r��#�#�[�����uB���Nqa4�w5�� ��@ c����W}���jW�M��f�N@ �U�CkZ�g}g��FZYI��2�Fj��|� �Z����*6��� �{)v�a�ra��\��z��/��k��d�f�\���N�`h���B�;���M ����N;��d�z�.l>�����4��L��+
��DE �h��wT�����ly�f�Kc�2eq<�q� ��������C��z��Q� L��g$y�
yz�@�N�U���F�C�6 ���e�<�4����*��$'���h��54�<_N�������F���` >�yB���<�O����v�-��h4[2���qWq����G�
���u����J��Va��[OX���^^ > ����:���c1E������2�s<F�
���!M�oQ�
� D ��d�t0���s�/�1�� �`X>�w ��Q5x����9��������_�{�����{B���� 2�����V�KD�\DJ����� ��jYT����qG����E������o:��t� J�F u�H�R������M=�7V v�( ��������-�_O������@��i�����c�b�~��b�7{q� � �c�n�v mV�m���<:]-f��{ >�
���iT�tx��o���_��;�d��~������"E]�� +=u���wq��QP
dw���fn���!Z[!Z�X�4�h�P
��b�B/ -��Y��Z+��h�d�aI���1 �g|�_\N����E_���2��P�q�x�^vHo�*"�0�a�������~�,8J���@������e�^�~��C����������;���H�WD�JUz>3W���������j��Q����^C�W�mU�^U��7���n����+����.�p�u\����B ���f�-���,�^.N��E�w�a ��L�k�>��VY`�+Qp���i{_K-������/� �H�
��6��{�VD�%���
����b����%��__������*���d@����~�O' R�R�� ��er�� ���
���z�S��Fd��Uo�oKH��# d~� 9*;_��t+BuU�I=����(�g���hM���������ZNr�( �i�� Q���(�G�iZ�x �e��=|jrc�o+w��km�����{�����x��*����6E��I)d]Q�@v)��,QP^�'�%�pq���~�u� �7���F<���
�8�P����Vu8�H�2;�&�)z����uZ��O��tP��_t�������k:I2*�E�'c��q����Y��k�q�.@G�>)� ofu� X���{:�Mh0P���c+_�������b#�(�����3]���qJg\}��@J�=U�z�@_p�2F��m�������L+��F�����A�c8���e+�Y�]L��F1/�=�E M��8x 8��d �an��;z��~C�fq�����7����7 QT}HL
wA�QR)|H�% Q������k�qIRw@���l����mF����.;�V�w:�j���=���"�]�L����!��q��H���N�������7��u2#�5El��"�V��$���Df��x<����1���,�u�0��~�b����w�/���_~~�����~������_��
����K�X,� ��''�4S7�<� ����< ���}J�A�d5En�&��t3N����|��+kRZ��3 �@�@'������_�����f������Y���|]��7�f��f'nD���U�� ���;ih��� Q4^l��D���y`>*�s��+������3p�(X������]C ���$_�fv����p\B��j+c��MY!��H��
���-��9�U�����UO�� 177�;�c3�
��(�-����I���%�l���oK��4��F�Y�5��Fw|Rz5Qoe��(�����X'�9p��pq:��������� b ��f��� ��"S���"A�f��D&���t��Ri������,��Q�X� �!�#��B �#2�����V��u�(��3�����&�L�����w� �/��g���Y7��xq�v���m��i����h������:����.V:�{
��c�����_�����h5��h5z�F���v�To�[�����>��������}������L���y�S ����tk�����u��m"��]����N��h������O��?~������{�G��F����� ��qo������)��3��r�����%�}����|��0\ �V;���fsKT��U�a�
�d>NV�����V� t�/w�N�U�E ��e�Q7��NR�p�If�`�R���jqi�S��(�� ����{{��<VK��{��c��]�jE�2hn�
�#������� ��EOD ,���'ra=�K��r�L�zy����r�����sO����d���~V�mdY�3��v/V7�e[!d����
o�D��� �����x/������Y�XD�|V>��Iy(�&������~�����%`�5�
�a"X�I� u!8B��8/F� ��P ��P��#&R�� ���%���z����J�P
����N��F��� ��
�+��d��a�<rr�f�{� ND�#����W���e��$�0�9vh�$9�QaQ�����a4_���ef�\��vq��y���
f ��4;^X�a��6�b���� L�Y��nf��<0� ��Zt4���4���������YH�#[:�t&���t��� �G�
���wF
&��9`��R����c�;QL|:�E�2Z{^Yjr����S��QU��ny�����0��.q�^]�(p#�+%?����T��U�E����Y"��}��j����y�����~|��kt+�P : �_����N���I-N������0 ��bj��&�y��.�,;<�:T u�^%��BS�E2��Y���[>��~v �>���� @^p /���Z|:�������.K��'�d� �e<��`��M!== u��7=$��\Z���c��NFW�ZB ��D�W=?�������u����t�Q��;w#��*��� �����B���1��s5M8��4?���T5�����������!���3q���^"�!��G�O2'0���A� _��o6�)� ��Si� �8hY�5eH�^B�~ ������HwZa��%V�2����
� 7��F ��!�H'����+��@��W��!?�]��n Xr�+\�xK_��� ��)���c�LH!����H�W�k�Ew�h�'�� ��/����K�u�O �c�
s� ��#�N����`7�'�yO���b�V� +�����gM��I����1 ���w�N��p]E��{�������H� � P`^ �� #��6#y�����k��)uH����R/�M������*#2X/��m ��E[�S�s;����BS�5:l��bSD�$E\��[�/K�-�^�������-��=�Z�f��������k�K��������{������/�_�{u���������$<�� ��}���W���!
�D��n��k�h��j��^��X��,]]� ��$�3�Pb���l!���B;�����L����5��_�m>(h�x��f���P��h�z���5��?��y�d��<9�e���xc��1z�(H�5�_u�[k�Ta/�!�����6'^����r��l����-i�����R!���E����8��E��8dz%z`�"#�)6.�J�� �����9�n��y$ `&[����Ez��k��/��� X ���s�}�!��� �W�l�z���O��������o(RT_�t�&s��^ 0_���Y��� �PPUl�h=���wO����
s�jdz���]��C�R�N���7��p6O��%���d~��<f�|������e��! �����o3��r<� �P����4��B>����|�Ub,�����{\ ��U��
l�5���F)���7��R�c���o�[f=*1 �b?�x�;���c�J��=���$��+�By��@���gyug��[��[o�k%8�- g�0������^������������|�����FY�:+�d�� �h���q+���\n
� ������B�F���9��PO�P?��P����[PGQ�����^�q~8:��}�R�S�L��������G9��F>�������U�k�������B��n_d�n7��Q�xHZ3 oN8�0����7����_��z�w> E�G��{��*(�t�T�����hK���Os]�l�����_��9�~L�:�j��GA��
iW��Bhx<E �P��L�� ���,D:������D�2�����Nf�d�Xj��')�1QK��ksRd?Q��|= D�LG�
���`k�E�>�o!|�R��z������U��]��"��/V��d���7�A�fu �%\� +�m�_z��[8���/���=7�HR ��5$U�]!�P�����a���h� �L\�9AT�Tx~�W7�;,|:
*��@�����4�@%�����szn��Z��b-7�VE��R����M���q[� �������vkE6>
��9R���6*l
��u�9������:��6}�+����M/�uu���E]Z�WW���t"����)�R����~-5���������5*�\A�{���y�r�t���(����,R��:�D���zuE�x� ��U^�|���7/���z�k}��^�j/��K�� {�_{J.�R6]j�C��wb�u� �����4���������IgD�U�eG����A} n�Z��.��T����6�w���]�e��_=��a*�Y������������m��`����(���u�n!����a*��"rei�t.��7r-�O7sL�������[��$���,���YA" �IY�X��m��q�� ���p H,�eQB �����P�F;n4"T 4M�������(Q8�8{�<"!V4g4L[�� �[_-S�u�[:�����hpj ���8� �N����y����d�d<����U�t�|H ����vh��d�����z�\r�
�q���2��7�G-�:M� J�M���&@����d�V���v���%A$wDrp� 7��]� �e��]� T"�l�1��]E�>4=�c
D�����$z��f�#0�*�#fj7� �MmK��A���P���G<���D� �M���]F� \'z���*D� X���]�CP���.JsI��k��������j�ji����t��]�]>{M�I��9��.- �����F�^S�`�F�� b���U�)�����������t1Y_EOf�����N�[pj�1#��?s�8oAoG2���LJ���S�� q?U�j��
~���e�B��d���dO����L!��g{���8#�n�����n��G���� ���7{��������A���P0����S����*?��?�?�����?��:�m����?����O�J��M��m�O�hQ���~k�U,a�$���BY�C=��9�V>c�R�z5�����t���Y�%����8�^����w t ���
E���*��y?����MV~�+����P�&Yau9��r��j��� ��
sC�(�E�UG��8�d x���PK��bU�G���W��C?�}m�d��H�M� A�,���D�e���%�|�&��$����Ny�������b@���m'b��|���A��eE&���L��B�CI�-$k�^? V�s��b������������mE������� \��V~��Dmf�v�m���]wF�BO�����.A�\H�5�M!+y�"t��Lc��Q���{9A�xAL��� N��n�����q��� i��)x�/V��N 6���)���t �WyG�����5o4�n�}Z�ZH�L�2��<i4��N�������V�� �>���{��t37:�����2�~HW"�*����v����V�M�z�������li6J����i�������� U6�3H����~��@%�S�X��9p�?X�9\��T���X
G��K��{(���m1�/2�fW������!�XJ2��
�p�5:mv��Z-��G�� �k% �e��5�q��RF��lY���^�2������
�����J������bE1�����e������/� &l�d����K%���%X�L��_�H�������g ;�y.M0�<�{}Jvv����wTK1h�����&?.��$�T=?U<�P���mW��vF�f�U�5z�Q��F�[���k� *��Z���)�h�j�{q{��G#?t�BH�GU�� �2�p5S�E���2G���Q+� {��y����o�� ~st�R�JYe#���[B� M7g�uM�u��)�b��r`�Rl�[u@`��\Wl�4��G ��Y[C�'j�+�wk���Nx���~?����;h������d^ We���]2?K��L_���J�#��"e��k\����������������m�Y.+Eo6�9l<������!9B��'��� ���*+%�
�q�Q��)1 r=�LH�Un�)��t� Po��jb����0A&��u�*� ���nP�w�6
�Vu V��B %}HOh�'�D�:g> �po�\��x{�5V�i� Qg�C�����E�.����v��j4�.PCP�����4w��P��� m>��k�H���~��]��=����������|(��� R���6���.~���
�l3!?
�DH2���2�\���L ����Q�����q1�m�
;
A����L�����=�R��I��_��q.�V�uA����
�uV-I��\�/�!>�%�R�Y�E���dd���Q|�)N���}eS��;I��i:��}��3�����<��r_l �T�3Ka��Ut�U��l2DQ��z��<��8W ���~ ����5�q�).�^� �:�����_����
0X; �j.mya�-��a�t���c��q�giG�T�4���+!�4�%��L�q�5[r4 �A0�w�~cI���9W�(�T����|�D6u���E&f��N+���V��x �v��Y1hBWQu �]�������3g� � k
E�;�����Q����K<Y�nx!qo�Z
�F �B���rn(� � ��'������L��L�Z5��@����{������k~=���t�����3*��d �� @���5 ��s5���x��}��z���
�`��J�j��.���t5���3N:V����0��@S��H\G d�����R����&0O�(������To�`���Q�k;��4�� �<A�m��Y ��[D��p[�AD��T�&����m�Pp`��9
��9����B�t�1b'�E �K�5P�����3 $o ��m�X1<��/�a4S8:x��F�cRE�$�#?E�?�������-U X�K�p\���&E�b\��,1��Y�������{i�L �����N�Ck9$� �����6t����,N���){�������!rOD�w�������/ �
��3����i���M�s�+z���( ��L �R$��P��pWj�R���"Oj�tgX�Ms�0g��@�=*�,��b~��56�� �L�� �������b���V���,) ��j��3��v���,�����c�u@'�Z�{�3�p"&� ���+����;`� ���#�H�����w�z��}/_�~3>��`:2=���,�`�����T�GYj��n�I<��f��w���H�Fa��&�f��O��Ud�y�������@��^��I���j~��Y�;�����#�y.hB� ��(�e���x��^<raYD�pC ����jBq0wG����:� �� �6u���9q�@5����a��j���t H,Ei�v�%b�9n�a�hI��StK��!�:�!�\��6����5�|9���Y��Kw=�
q$�: � �Y_��8��f���s��qh ~���;�k�Z�zq.g ����ki��b� C�z��'��^�v �t �
]��9�zh� Q��a���;�pH�O6 T���}�}DA�����F[�k��q���#m2�E1WCc:&��Jg���a��=
�`>�F
H��\�v��+1@��t��fu�)9>����x.~�^cN����� Eyk U`�0D��g����I���&�������#/ D���@ �6_N�;�B^��s�\��A � "�btLl�e�@�H�yH�s��Y'=�+$���>|St�����j a�����F&�&�$�H�3����{~q��#��>_ �KP;[���hy��[&/����i��0(�����)�������&�+��
�aj��EI Y�[eIO�0P�
��by���� �|�K �����
�s�F�� �
o���F�rC�����9��~���3*7�TZ� �@D��A0]�A p�$��ki�@�� �����71���F ��!�i���y���{���H�e2)�
GM ��$au�d[�W, � �3)���|� �E{�]����(�������^�<?{�j��.� ��� ��V6��wm�-�IB��" +�������^*��"�����4��"�u��(JfP5����g�1����D�W���h�6��3u1�p ���hO��.9�"�K���tW�!���?��=0����d�Q\Cu������4f�������� �#��^G��vm&��N
Gi�P#���e�7���� 1�����0�!��d��(�����H�TS�}�-�%Vlk,��:� �Q� �G@0�3�lRlVrt�@ ��!X�U_98������(^���H�j9��L�9H�&�[��0z^_Jp'�:� �`�@8S�����^
��5�����G��F9�$��A�o� ���2 +����t
����j�M�=�� ��#�-�WK����)���h*&����N�Z[MvWr�WG��5����X�c����a�� �Nlz��R<yK�q4:�n��t[�M����x����d�0
��V�'������>�^_��Wm������oP2/S>VB��3 �B���Q�����y����?=�.�[���dT*�%�Q���;� A�+���2�Sm�A%��"t?`�������zs������F)M��a�:r{vQE�� ��P�EJ��U����w69;G���"����>/xH�$�^0uK�P������XYm��y8s�6����_Sf�Zt�9#��1�]��Z�X$��f��b�'�=~��7�����v����p���h�7��a�b1�P���+�:eh��H&�DW��w.`Ly��L�Q�� 4�@L(�h��%:�=���&"v 8�\p�[���QF�x���&5(� |G@���T� �i�h8����� ��*�=���e��W�q5dA�h49����xD�9Z��
S��F����?}� �S.��|���c:�����ct��HSo\h�h��A��u>�A��}�����R��3�s��H3!�x!��L��>| �����jDD& sOZOB�� 4^v
/
���
x��w ��_P *@Z/��)�����^����;RF��� dO��]2���� j�9�o����
�c-F����h�$9����a�#����������}!-. �Z_�(�|���w_M �F����~��+�������C5�{e!�]�z�7�� SR)�z��F��Q��^*�X���x&�����Ao$�p.�BW����f7���L��/���5Go�mq0
���j4���^��R�ms����.,!� ��^/00���.a� �kx.�������$C�69T^� O[R
�(��|�rQ�P��� ^q��e��=F���v>ga.��Kh����f�zN�go; qW{q��)�v�ouzi���
���������]�����2*Q�E�����|Gw�I���?�U���|��Rg�)�A�O!�N�([��
V�
*��qN�R��s�C�j1_sgV^��A����5��|>}���� �%�c���� ��%�G>��pQ�m��� {�x'� ����� ����P-��i=����2Ol�N������(�M]�B� ���`z��w��\^������q"s�f����2�� E8�c6�����*U]�����w���� �a7��� ��@�� ������������APD�o��Uo���� (�P�2���
~r�����6��m_,Y)��$�n� ��>�j�0�V+���U_'�q�S��,MV�P�aE��Q����w��|d�$ ���#U�H\3��J�q���l�^_���~O��>[8x~�f��������?�eg�����������j�����YT���=;�h� -2��s��`v m�EUTs7Zm�;
�N��<�� ����8PQ� rJ���V&����P�b���k
�eU�V��o������J3����� �j�d�Hz���V��uoE[ooq�V3n �6���W}��1���kn8-��o�E]MFk0���@l+JtX��E�%R}��:+x{�N�P5�#G9�=n�< x���k�{�gk�D��'� �� �b~���a =Q��������D��n�N }���Vh^` ����� ���� ���<9QG ��rJ� 5�7�U]o^�+��v�R��v^��X�����}�x�����0{ <�>L ?�|�u�Y�X��le W��h��pt�~��Z�7)�h��'�=������vS[3K���5tMD����46�) ��i����{�(_�X
� � �3��a* A�(����lu�&>�tu�����+XA�_�Z\/h>JW�t������O���7�[���������8�:��f]:�������s�m���f_�u�N6?��;0�RdO����A� ���9#�8� �~��9���� F W�q�J��b�p��>H�� ��Y��<Z��JE���-� r}S���wJ��N8����3YlP@��]�g �a�baWAQ�g�|(F.�,T}�� ���f�5J���������n�Z�u��^TM� �+��WT��
}�o���-A�I�+"�V�� i ���]:K�^ ����$���� ��:�MG���0v�t���!Qh��#������ �T���{:�����_����zk�g{�K��=&�j��2� ���,��We�E��L�5Rt�q�[�Y����.����z�����G������k���&�i� ]�ayu��?��|�[ �y��i�:Z���7��'�A�|�u����8E�B�n���v��+1���Bo����z �A�R������#���5��z�������&y;���= y� J�m2��|����0$��y��)`���|���|7�3m� �&�<S�lJ�?�I�"nDA��<��G�Z}�mO.]�0����{�ii
#�p����9�& ��P}�(�|!x+t���q����.s.)��$�Z
�i) ^�������,�'g�C6G�������* ��
��I*��G y� ��RzI����������)��{�6��n�; T�d��1��f�'��~���AFb��6H$����&��w�O9>��V��Y���=y!J�x�C���"�2G��TC��~�./�O�+��wWY�������}����W�<hz����$r|�3���.����%���[p��i���6eeCp�7��&9v��0�)� ��K��0���u��� iQ8?7U/���u��D�`���~K� �] �A� ��~R�-��p�-$y��\��s�%v�9C��� ?��������O��t���A_��?��
�z��d@s��V�:-y�`F,x�w�
�E%S���x��ygU`�FM����g �j���q�1Uy�A~bs�(���+���IH�s'��z�K9E��3��j���&�����i^�;�| �r��}��v^,��������2�m�^2.�9c��f��eC9�T �Y�IcE5I03�\���y�v��{�n_���vC���4}z�-��C�0G�������+P�s���d�Im�>�F�V��I�os�����:z��b� �����[;��u���^f�`Z���� h1�Y/OIXmF����oy�{=8)"HL���&ts�������vO���J�^������� ���@�N�f�g)����l����e�$�zPu�V��� ���GI�L6e��R���T�.����������?�p�� C��o�%��Fw�P�R�c*��hZ��L��+�u4��A`I����_E|�L/�A�5�����r�����IA}��\�T t��� �R\J >�>.�� �w�~���T�Td��Kd4�^��DL��ui�9�mWg�:�bUbaJ����!M�nO�nI�nK��������Zo�g]��������R�����?���%� ��F��7a�)]Bc+�`��k1;7�v$� O��4�%���8���X�����
���p?7c�:��k��U�B�K"H��S �[2H�����H%<R&i;���M
�*z��R��w����^� �0=BJT��J�� ��f'���"=�~k�%���`���U�HM ���4 ���n����k�a���)� g�[Y����,���o7NI���$I#Ip����<��o��cy�p��8�fg7�"�
#
��K�8P�I�^2�K(�X��pH�:w*��X5���N-�����{�:j7KE�m�O��nx��I�y L���H� ������R����������e2C_$+�'��3����1l�,�K@���Fq�uI������'U�?1��8M��4U�fH�!�0:1*)���]�z� �m'�4+��Nc�h%'�Z���
F���o�������i F�F���|��64x�p���MH�! ��t�V����{�Y�#[�31��� �*��C� v���t����<����l��N�TX���B���������d��D�ve:j Z���(��i{�?����o%��>�r/B��@��<O�V<j ��:6,�`E���w�����S'���������b��������@����u#��H�z}�IvED*J���V����n ��"�d)�H�g ����u����*'�� ��^?m6Ok�~;9I�����n+�j� ���<]M��|� �}��9 j/�c�b��� M� ?%���<Oz|<L�lr6 f�tD�*�!B�����gU� w�\�=L��T�-1�Q 1���AK�q���MQh�](���� S�QJ�b�|�pr�� ��%%�@���1�
}n�7��x�F�V�(�`=b�>�������
����p�[�^?nu��-��9�Z,��5��d��� � �I\�'�3s4���`��tvF� �C���
3S<����FC������1}�>S�;V�mg2Q�/0�#$���k'���� �Ae��oij�V-�N��������H��<��7�s/i���v���6b�����G=��$|�"�� �� ����b��1d�M��,�����h��k�pg��5B����S:;V��#���V� sw���m��-����7T���umpZ��/ ����Oa;� X�����9�]&W1{���k�b�&�T�:�i p_ *� R<�h�H�c����Z$��r ;��"J��A8����-@5��L��n�u, D��f[3��S H����tO�J rZ�cQ����"�D`r?�� ���
�}�%��~�"���n����p !J��\���_ ��i}�8�� {��75}�������V� �qH�OI��T������9<��]�MK�����F�.b�V\��bJ� ����� �����������:�qx������&\<� &J^�f��U����00��^�Vs������7+r����s�����:�7� �3��q�s����^��S}
Z���W��N��W?�������]�����wf��L[0�z0��^g*i����� :qk H� _��aT�D�� �����,��9�����6�������U�3�5�M�?� ���y�����R)
nG�U����DrPM��M�Kh~nP ���X��` ����6��� ��N�{dk C�n�!%�� *��:�4��]���������5�Z *@�K�N2��c��7�?D�AS!D���!�qe� �#dZ�/1������LI���� ������{��z����*��L?u?c�C�0�X����Mg��FL�1R7���V�d�H~Q����OL7�������"V�-���+�,T������� l���
�iO���Jd��E�� �B�����d���~�D_���,Kf):��������,�7��d>vi����W67�W ���}m��hBr�O��� 8O�?>]:����Q�9��yDG�l���\����"y���Tp��kQ�4����h��~D�x��� �d�X� 7*�,�Y(n;�]���5��z��������C�$�i2�������0U�4��d1��f0O�_��-�r"}��5�� �n7�����d�xr �3�`'������T���@r1Qk �R:+>yL>�,�X���eJ���H^� q
���+4d����{ ����o���,L�s�������CF_ ?<~����o�?� o<z���� ��7�����������]�"J3�IXL�-�q�!��B�{.�N]�.zMla[�Tkk��� ��j��A�-G <6 �Kw Y�1C��Bu�rE�L��������F��W-�I :��:���|��k��V[��vH QrB�d��,������z�����������'�InOI��~; w�
X��^���zo�?]��{M���&��}qQsI��7�B����)�Q��(l��%����ok5���'d�0
��1��F9������G���q?����KZ��C)S���m�X���;q��r�� X&����Y�^�R��R%��o
���+z�w��3M�����Y��zj"K� ��8�Cb���|-.���lzu�P[j�D��fj����v�>0��!���S�&���8m%NYn ���R�4�
�(q}�pK2��k�7�\�:G]u�: E9��~���p��Dg^����a E�S������b1��2!Z���&t ��
4��K$e���f�v�+l����y��� ���_[F����L�Uh���F���m\ _��f]!g�#�u���!|2$)?�F��9�Xb
Xr�=���c�_�s Gr�;�
��I����s��*`@��N��z�7��B[SH�E� �6r������?�U�5�P�Y� b>��|��n�3%V������@4wD*��b�!�<���e������������ �#��3�:nybZM��D.�v��R'�� Q���{;��<��A|uj�s�pO�+�
�2�l(�r����y�����F��.`|�*���E��va�Rxj^���#)*;���8��&(7�7�A:�f[�l�����UG��M!�l�w�* ��wDZ���;� �>m���(�b8����>;T�b-:�,�,��;�����6��Z!a���WY��x��� �(
mN��Ax,�G�?��6�n3�,'��m�DS��6<r�H�M�PZt���v
�R)&�����������C���D�E��Bf����=�G�xeU���m��&���p���<�.��Z�Z��8����dJy���(�0���,��k���G3 ���d���
�� ���d�TH3��'�n�2����>:
)� ��0K���"ZJ�y|������ ��:!�-����������/�:����;h�� �I)o �0n�-��%���[u��-'��g��|OY��d D�s�:-.,Q���n�Q�[���x�8�~Cq����p�)���P�F[-��F@�lu������CY�^�����!��4p($��z��$'�v���Qm���C����^�E����6��Ve`|X�c����7j+�g,�Y�� Y63� ����/(�N�hf��b{��H]��I� i��������?W_C�M�I���?,�jx��?��9 ���j����8�:��>�z[�_������b�R�U���m������.�zb���G�2�L�=6����U � D��T;M�6�d��ku�F�-��6� I�7���v���%�}�������6���C:��<��x7����� b��2g+��� O����0,���hf�A�U��;:i.P#PZ� 0M����� ~e
�����oD{�v~v2��4rY?�&���J����f)�.���z��D�AJ� =� h;�����n���|)9~�9��I�f�^��:��gD^M��[��%<�.�Sr�RO�xrj;�� g�&�����`}����i�Cw g�f`$��J����CMw�����Kz%%����B��(���5�/ >���~
�l�Z����vQ:�) ��MN���|�rxt������/_�����sO�]�-s{s���du]�Q4�7uLN��("��@�wf��N�&����lrv�F������s���3��xX�Z����c�GEJ~`��kx�.�y)x��B�wv�%SL<@S������ $ �z�e(�vp�����I��'�h�y����eE,�B/�y>..�La5v6���������l ��9l��1� ������K����P_���@������!���s�3�P��4 ���)��2,���� B,r?��c��Td�6>�q�[0y�LA X�>��A^`����A�)������1� ����'e�y�F�i22�y�1'�Q6��Va��r���h�]X~�Y>d Q��k'�a�x0���� �� �XR]& #�E�Z��T�k� ��P��wB�K1�����K�{ ��X�-8��Fe1C�l�N}�����7����c���>��z?n�Z"_9e6��C��?�� ���tq�����7���������XI��2=(�d�x��6�f����� ���a������� }t��D� ���'��P n���j1S�>q<8I�3� ������P
@��
e�/\qXd%����4Y������ >�{{ ���`�9�J 9�����_�����q�V��ix J�[k��`0'��~a G�{���ma�*|$>e
#N������k�;!�����t�dy~�:!��>$!�Am)���h7���n��&����TV'�TR�:}+2.�h����� DC~gU���� ��l���A_��� /1 lDB,�1�� du���"��}�����$ `��7�k{���m:�� ��������p-�f=�>�#|X�F����[i}�j�j��I���s+���)���q�S��[
qy�?
n��d>�n�t ��ILv����b��������5d�V�N���O�YN4!�$Rk����@-OR�)�n��>;<*)
'����&g��h-�}N�G�q�C]%� �; ���yWt�^�`z��%�� ���j�N�t � Z�#wb���+���4[����t�?@o����������� ����R�����d?��]���!� s��cT ��E�9��Jgot��QX����=��F~�aH|�I��$��D��F<�Y�%�O�� ��H{ ��M���.�@:
TD�z����n��5V�%s��p*�H����h������y�:�������FUaN-I���w�F[�&�I���� �0#����)*��7��i�k�*"��s(Z���b��(�0Rs���q)��t��|���9/x C�ln{� ����&jV�?9b S���P�)�3�_�X�S�����7 '2�Z����y�zx���W�wQn7:u�%7 ��|{�����3�>5������$/
9�.�?O��o�s����>E��("����j�HaHc�XM���/���'Y�M�� m;�t�L��1At�Z��=�b�A�z4N� J���R�
�yD^�clzk,�t�<��l�����K�ri�W��w��W�<�z���4{����Bx���6���k�\�4k�t��g�Fj��������y�\��#����bX���w���w� ���{/�b
X���E�j�(L:
�Q���Uv��Q�zK�i�����g�&���������5��3W��o�F� ��&Q�|�:�H�r]���A �#���B� ��%�i�������aF�$���|�S4�i_j����a�@�F��Z�"8 ��&�Mc���E��d�i�q� �}}�a��=�O:��
��wR��^.`mWb=����f6�Z.�F���
$��� ��` �`^�QE@�-$s�
� �9+�,������[����$('���B��Z�[�W=���Y���g �����%�9��I�q�8�
= ����Sml���`
a����qD��i�~�z���s���z��h�d�����uH7��;�6���A�2#�H�6��4�j��wR�*�|��g����q����Z�����Z�=i�h�D�^�7{=�B��qs�5P��!�;<�8j�;%���� ���G �����I��+� T�?� :�!��?`� �yt$Jt���tZ:&��&0��zB1�����"�����H1�#r.b6��O�E�'������>L�Z� }��O��Q�7+:�'��a���idr)n��H�'�ptS� �4� ,���&%&���K��L�?�j
.�6c��\�X����RX�B�(�)��d�(dO2Nt�b��^��j�<%����RG`��T-���"i����k���
��/I��������_�bh���a9���>@��'� � ��k��c�7�H�� CM9���
�Z���>%������U��>�9� +�i "�%��l�A�eE��`��
k:������������8C�$"�� '|?k���X�4ShP{E�xSE_n}��@�gt�k�T8p8>���3����. �t�]e�� �p��p�Q��qW���(���'����s��lj��bp��r�t��^!����P�0��$�����k�kw�M��l�o�y���m��g[Go���V�����f�= LN.����}�W&��$��~�=����U&^ �T\�2d�.��=P��������I��]
87��\E ���e�w���i����E�(��]�����Jl{_�
������g�h)�v���U.�K�0��^Bd�f&���)= {M�xt�e�/����L�S�K��m������5���f�a�;S1�R8"#M ��_�u��D��
��#���xzFd��:������ ���vU�CG��
�((��l�� ��S�� 5��b���
�X�e8#
O� fn�,� Q���C���|-+�YpZ�[ �c��Y;! �����$�� � ��l$ �q���0��H�<M!�-����3.+�j� ]�L/K��G���k�)�4����Qz2G�(��.�c����tO/_�|�G�R�w�5oW�|�W�&7����I�1�%vI��m�m� 5u�Y��M��]J5��fR4Bj�QH(�����2Gg�pk.�9�.��f^ � ���P�k%�a�]:�����p?}���@��,�f���=r���u�(�L���'r�?�{H��=E�����a����d.��i���-1�Fm�Q���6���!UX���j���������C���j���&��Dq�1�"�7�L�����# �|,1 ���W�z�����o��V�]�er�v6�Re
�>��������������cm������A @���� �=(�����1��`8Y��t=�N�;�h]�G��dW��SG\Q��K��I0q;g�� P���drQw��dt-�����a���<$�s<#��7s�,���="5'�wA0���\�A�W�pb5g`?��H�Z�@�z����/��=���fD5�A) � �����c�s�1�,��� ���E��;\�� Po�\|-��U(���)�{I���<�('���L���Q�8W
(J!�`\+|�Zl�`�0��a��^�A���p�6{���+��S�P�)Q/?4��[�T�W<��� �u��]���a^q��)� 7q�� �@�I�� �H���z�����C� OE� R�� +|�'�_5�A��U"L����F����~��l z���D�����lS�Na{ �/[\�j��3\+XaKp��M� � u��
`���s�n���+�Be3t*?�
V��hv����h(�4,hv\WRy7s�^��m�����X���wO��Q �V��5��O�i�����x�N?��^�S���>U,�0Y����%��o���<N�����Y�R ]sV\E+����-'�����d<���Z�������Cw��t��>���hHrZH?9B%cA4�yU�q����V?��[r�V$��h�x?0�<E������F"fu���.Q�7 rB��s/)yT�-����P�� �Q��\X-�9t�����C��b�X�/�1�
A�1���3�i0GMB��:t�*�2tq-1r[�����Qx�������S��b�)�"z��1�_V~�(�B :��/�� �����D'5v1>_����E]I��Z�4��[>�)����w ����],& �B5PW���e�k�#���0� �3 |hUE����]b�V3W��s�0����\{ZB�� ��R��9����W��f��������e#�� �Gz�����IW���r�� ^�y���|_��}GiC���"4#�kT�d��4"���C"2 ^�;9eQ�+4�'W�r5� ���h�_o�oX������YB,O�P���]����b-YNY�7
������Z����Sw1N�B��\x�ZRC��PV`58�;r������I�#j��L���a6 ��R_h��|��VN_qT�����Sv.0� ��&/I��NR���W������{�Q3;� �E
���h{j7��V*L�C F�W��E�8e���� 1o�r���tO���A%d���_�����G�0��R�� 4��`7� %l��q��Y� ���T]M����$V�v��� a�(�'���� ���m;��2M����x���c�*Lab<: W�P�Q� J�6��$oH�8p�,�3���2�5.f���WW-�#��1DZ&r �����y��&�5�����e�cJ�vD5�
�D���.�}� b��f�b�EP���2���B��#�����^�o��>��������;J�E��L���<��%����A-Ns�� '
�����|2� K� �-y.��0 dx���d�A�} A�QE����X}�B����Iz��G9I� '�#�I�R��,���h�9|��k�&�<��@� @� _����+>F�x��(���cu0
����\1=��Jt��o��)L,��L�m��R�~en�i�s��;/���d����B�v! ��i��LV+p8 �/��� ��Cq�9�_Kx� ��t�4� ��8�����^�����S�����e%;I��������� ��B�����6�i$4v'�h ��%DWv� F�����u���4"` ���\D�1�\���R�%����f��_9�L�M�ff4���Q�V;����� �� ��W�����2� ���(i��z�"�����zq��
N~��!�I �������)� ���{cj5��1� N��v��������@ 9�1p����B!���2w��������;������Z�}�)�|ko��Kj^��fP�o����2���o�/1��3�������G����`�|���5�d�F*2
,��i�uk�Jm�&G����_�1�t1:# �V<h4mP�"��-B�S�0��"���E�Nu ����3��K-hn����h$q��w��(����h� �������R��7�}�/�4%�V]��F��(�����e�F�A�P�����5=��4c.bx�_����a���� Y�)�����^n��f�l��l3*������)�uO��Z-m7�V��$�+��D�(���� ��O'gC��}��oV�k%dt�� ���z|� ) ���b
Ff�a:�(SD�Mv���q������_�;~�>�����w�5|���W�� ����o���/��?/�{_bW���d �u�'�rU��Q`
���:��[�AD��`�Z��_�����_����L �\��@b�e����|j�dz��s}>SW:[�Q� /l o�u�0��lw|
6{�5x�sF����6�F?M-���^����1
���Z��GV��9
?�������"6?j7ER/q>2�(������b�:@TGU�D�w���M���3�n�O��^n�Z��|��|H�H� ��R[�D�d4Zl��ZGv&:� [h_"� Z;��2 ��$���J����.���~;���)�Y� &<Q�Q������3M�'���}
��l�o����"�w�`������[8���� ��Y��s�w�`��Qs� B��-8�_;�;0g���] �\g�woQ&�[�Y��]���Y�_�&��w���:��v�=n%����.sM���(R��J�*����B:(U�c����'����)� �Z'yz~M��p���_����D�R��A�����3���P!f�~UL���A���8�c4MfKL/��� ?�c�G�@Q ����d�m������t�Kj�q�� ��J�[tY���Y��k�=)���Dy7� ����~��O��J�>}V5]�L�P8� ��&BW �� �� ����Zxg���R6�A���� n �q��� &���X_�������+������;���� ��� �
��� �V��J��6 �Vq��az_���i�j�q�������0P�x�d[.���fI�~�{���������[o������)�z�:i���� �$�8�N+� �ENI��q�SY r�,���F��4$�0�=�` ��w���W\g���&����.��� � 6 J��= 4
���j|�fA v�}�wr��w,O&�Y��<i����>�E�^�oHM����VxihQ�C� ����t�a�'k��'�� �aUgXh:���P|);���d������.��Ht�=�5��p>�
�oF:^I����R�{hd��E8E7�[�W}�����p������b�*�&���N�N��^Fs��#��G/�ONl�k���Pq|J`�b��������e�N���/U����~�$��I�[z����.S���2��x�C%Y�z��������t
>3�� �3����`>cn�}c�o�}g������W�Dmi4]��&�r�M(�/����2RM���^�U`���rB����C���� M|�i"��a#�8���k��DZ�K���&��;thH2(�9%��k���� :x�)�. ?����Iz��@�|R�a�]c�8���j������{� T�� �
��������
��'��W��?������K ���.��DV�ij�qG�'N�k�=�.������
k;>&���y ;)������S5<,�90���[����)��m�2��P�J��y}[Q�Gw��M
�^�~' 4���Ce���l���:�~�&��8��n��hE��%"�G���X���!$��� a ��P>MNNV�E!�A�u�u����
2���Z�8��c�9
�NVdx�Hk,vs�.�"L1V�|���3�4��4����~$�^���M~�!M��*�5��v�i���]
���Q�����U?���V�O���^���v��7�������t�?%���T�c����U�fpn�7:�A���4��Ac�������� J��n�Q�t�����@}:*��S/���J?�����������4�����n���������)��3��r�����%�}����T����g�^$��i�a���> �1J�������SI�e����>w�<�P���e���^���JDnJ�� ��pOM�����~�Q� ����Ju������z\/�~B:N��I������?�|r��%W�?�Yuv 19 h�^�,A_�����x����i2��U���v��(z�Q�����F�3��J���� ���9 ������� �Nh��uS�M~j�q�;��{������R,�"=�@-���d�Ny;]k� +��U�;y��x�$l��u w�M��-d���6C,c����)����Q�INq���:/�;��)�BPE�o4���j�� /j@F��2�X�D��(�e
@�T�T�+C��0��b`<V �����z�d�Q<���+"� �����4kg~������B�f<q�k��g�� ���V�� �I �dxAD'I�� l�3PzR2�i��2������C���3)��n���z��~��YY.=k~q��#���� H~�;;�t6Z^���6Q�I����� 0�� *<A�E��+����@���^�t��&M��*��N5��[ $��� � �|�����\".r������o8�����<�8��>S:
|2� �� �:x&�h�_��{F�z1?���`��iF'��N��w��t���iq��]�c������ H�����!�i���>`wv0�!_.���Dt�@*@���j �y"B����'$GJ0����h�8}��B�����^�<?{�j��.���� '*y��*mE0�]\D�A��3`X���&6*�����\ ��_L! ���Y����D&���q��,#�295�_y���O H�'��sYW��v(A��zB��j����3'c-��������2Z�,��c�=�
Z�������+�P�6S�W K'r�1j�}eL6��d��o0���.�$��lmq�B������P� bZ�=Pkq��������f�0����wo9AG���-*�u��~����q_��a����V��6��2l
�D��7 -c �2 P,uB� ��Vd5�7 ����1��������IwA����� 9����v��_�,�s[���C�7���L
: �nh����GU��+ �����q<�D��0��� �t�� 2"1/O��?q��[��Z����N�N��pM1_�W�!]$�)l3j/8��x��� �>��B�j!G�f��1�������������������������6o�����7��~�) i7��6-���0t�qt�������5����-���/Z�bj2�u�?�y�������(P������v�h_Gy��8��N�8�tj�Oph����q����D��}��KF�� �uj�@��w{ �p`�a/���1J�
8���a�X��� 2�~�n����{B�i���Af��!f�~h��=v���J<AA��,��r�����)��x� .^��_�d�h�}@�������D����Q�B��l����m�W�az���h�L�?��z��M��+E��� ����F@�p���h�����%
�JH<g����|� O R����/t����",�����V�=��w�,��=�]��)����'De���� �������U���@���w�`kw�s
�n�+o��.U
�`��h�r��mWf�>=���'�Z�t0j��4��E�[
���w��8�f�.������� %^T,������0K&��r
��gP�~X��6T��n�&si��5��b���R�� {�����D�cF�s��rR�3�}��Y�+mk���H� ������������ �o�� �4�����^��G�
;��s/�dx����D;���F������&�t��Q�� ��������C��x�����S���e���@OjD������eU�-���=��zueb�L�������[����D,���������� o��N�5uE6 �!7�'�����P�� �� � ��e�����BJ�'��Vz����{�� �@��m�9 f�K���|�#8���_}��d=: n� H��N��d{-/��Sm~��3�b�����3['
i g�)� \D��YE��]�y �F�]xM������8���t���#VF�!��d�4����J��B�4"���IKTp;�--h��� ������R��Uz�`l�u~q!���c����f�S����!�� 2��Wc}r<*��H��*��B�t�:��B���Z��\��� �~/ ��!��R�*&��yYx��8 �G �8�$�""S��~�n�p�9}�y�o
0)
]��� �����ng��z3n��CM������@�C�g��+� .��L!�\!�� �S����@��8���x����;��< ^4��y���
d�����S�8����������0O'%]�[��Xp~�� �Z�v�jY�/`��b����L��5�U(�#Zv�{uIT"�x����h�
�|UrO�0[��� :��iCz��� ���o[�>dK��r��{������cp����!1.$��v�7�� ���j�Z�2is��ilV<��5(W� ���sRk�{�������|�ga��,�a��V��o�y�Pnn�������T��k^��� �TT��2(���d!�J��� �`�f(��
P�4���jC7���������pLGH��F���B��X�4|�q��z8$q�J�E����Q$�6�rR�w2�E�n���T�m���:11Pgo�X b�1v�������� o���f=nw�;��_�#� J���9`q���"���E�=gGL� ub1
a)
��j�h���s2��S��!Y^ �ve?62:j�e�Q���P������?�m� ��X,g �=D���q|��J���T�Y:=�E�w��L7����Ok#_P=.�l��+t�k7 �D��hp� ��8�"�=����
��+4X0�B�25���6����cl ���>i�4���'l�+��_q2����4���i�|����c?w���w�I}e�a�=b�8� Z�� ����r_�2�w�����@4�L7���<#?�.��Le r�mw�q�a5 ���)4��y6 ���� 0
)<_yL�k���"��:��o��d4q����a�>� +���,�`�?�j�'���H�kZ�Ls��t���{R��3 ;p� u�dM/�f���� ��4r>�
E;p(ZA;�dnE������!b�^o!a���:� �2�`��"@����K e
����7 ��"O �����"��p<�v�+���I::�"����+� �����U�B�T`��C<�.�eWs���``�����ww����w�u���PT����f<�����%�� �/T�<1��'!�V[���[�m{���o�&�"��X�-{+�����'T!� w�F�B(��e���f/��}u�}B@�)�����`���^Wk����P��}��:Bo%Q�>��;B�[��N�w�u��}���A�l�� 9y"&p�������lo]�8��^;��:B�zk�_���]:c��j5�8����KT�-�$���-���N���6^�����b��P���F����b��T. 3?���D��j2?C%�4]?���uv�4�<���0�� f|�A��Hr5@�*������z @H�bv ���;f���s����}?M� �v���\/3U���{M�`�9����`��9G
B����
���
H��**����RQ��j��$�vG��Y��
�T���[�:�H���?��y�h�!��QQ W��+R������bM\�*��.�H�T�����xt������=�}��Y;)`�y�A�z�GB��� `L�=�U���Q��4 ~���D���I�ag^%���D�����B
\F�+���{��!�nS�����)�|���B���|�R(���y�S��&�%��&:��r�����h�n� S�Yg�}�����uy�7��1����N��{����T��8A�S@x�$%H��.��Z��Sm���s�rhac��
f��K��R���rmqo�� �����M�[�����[h� k�&n�+���Uq���I���e�~q��I����m��f }�A_�"��� { �Q���[^y!�f{t
24�V��{�<e� aW�]�����=��^.�2��,, ��c@
q� �t(8��� �B;?L��0fIm������S�� X�,E98�E�W �l�Q��� ��*��� C���Ti�p�A 8I1y�R:h����oy�F�t4�/����M��M�PZE�7��G��.���j�,�|�=3���!���hV������$[� �,�-�~y��A�b�$C�9djTn��yT�Z��:q��I��c�sR�^j7�`9�VN\�U�������n��'<����J�����V������g Y���M�7"!U
���� )�9��m(>�5(�c�����w���mBm��,����y9��F������0+E </��
r��"G��Q���H.�!��-b�J9� ���k�����V���������J^_e��mG��p ��Vp��Y�1�t������V~�� �ri�����m��K��x)������T�m�����n�sI�q���|#g3]�P�_G����)�t�S�����q���+���E�+] onw������~���}97�~"k�V��A)�OJ��0wR����&UX�B��d3� s#��H)�����f��wZp��'�� ����) ?�l��������v5� �[ �J=�J]�J|�d�� v1����L�]1�����=0�Uqg��=p7��� a���f.a.��� ����F�w��*��������� �(d� � )�r�Z�����b���N&s���X�xj4J�)��SdU�?�?�F����A������� �S�-�k �*%]U(W�[s�c]�g� �u�w�+�h��l�Y�x�iE1�X�1�p�a�:�?k��)�V��ws��M,eEf��1tU5
y�} �}��|�r�
�
��q�U5�������^0��S�|�D`�B�+�
P�W��+V��>�������K8%����C:�=�S�j��������
�K�V����h�����U��0p`�>��Q�_�x��,.I��3�������n��w�q���^/ �]�Q���2����_$a��|e�����<�LDRF��o�}�NS��ps�0,��t���* � ��������_�y���� ���~<�5+O�j�����n&����C�A���"%]��(]�n�O� ���� �W�����L��{����$*��qop�\��F��O�����6��\? �]��H��E��I=���Ktl��D\[Oa�����V�����(vo�����3sS�"��E�} p�IGBNhEb��������X�lH��`��R�"���O�G���` �*��Q\m!��S {2�Q t�F����X���O��h�?O v�Hu�����{�}�-$��l�m=��k� ����<��v���H���� 7���[�n��E���C= ��V'�n��ye������3� ���_�!�4�L��fAO��{S������x;g=nt]&�s#UB��7�^�m q ��Q�o����"��S��d�^��"c�
|p�
����)�iM�{ �����*P+im>J�?Pf �����YM�r �0j�!�g���l�dX����d�n��O�ud�03�m0�uz%6'qu�EN�N����U��k����>����F���vF��To��������?����_���,�h
����Z���&3�G��a ���@g.�R�9 �6g\x���z���G��O��V*��E���Z�r���nh7��_��]���V��/�W� �w�� � 1��'�D�j����I+����~�l����n� =qq.�b�LZ[���E:��58y������P����]��|�cb�?+h&�)���Uu�BQ��I�^�����}��K���������F�����g��S e�[X��p�$�E�h��2pE�*n
JeX�I�����f>"��E����
f3�`�$��/4�)|M"����S�l�X �Y oe����o
0�� S�]��2MB S�%3L�K�a:�6��N�\�C=/����h�3�K�����*r_��
��.#�������M����5�!��1{��6�����V��'������.���L}�z�����?��~3vy
z�r+j�2�Q����~4��|XY�[ b���G�X����ZK�Bh�[�B��dY����N
f��{�#����{�����������-;����~~��
���|�?txJa�z��/���*��{��� �Y���.���m����M���{������+n���������j ��~� h��P���E�h��� ��S��k�k�7��I���5(�&�~���(���� ��|��&E� JR���9�5Y)� }�X�������J�
~~89���N�S���U����j17GN��7��55��$\:Zi�H���p#�D�}�/�{��0:]���6j��k���)��,�q��T51�$rr�C�����[jw����T��a J��D!9�h!�6���E3hv�A�.�������d�\�2�|����������� �s��Hk!@3� �9��m��93e���� ����N���X2,�;@�B��G�����9��
��+���2�+��
]����!��_��j��X�9� ��m�@��i: f�)y��!����G����#�<�n"��b�Y� ���I1�����q��gNnt�
����(?��YSxZ��o���u���a��k�@1G��}������{QW ��q]�0�R9r#&������|��t��C�
*nOZ��k}���x��x���Y��F�Q �p��:��O��w3gf�0B�5%�Z�j 7s��,[6aG����o�+P�(�L�������Q��d�6}P\��O7��~��n��\��q��u��7�q��% ���Z, �(��@�_j�cVt3V�#:� :��b�� �+c�Y)p��X��Q]����p�@o
Q��D����0]��q^�6�aa� %� R��C>��s�^d��f;��6�Zh Q^R�D�<���| VWfS�SS���&ZT���(����r��������,�Q��M����[[��E,���!xf� ���q�� >�SW�[�b�������93�� �j]%a� 5`.�B�Z�
rQ����~����7������Uw4������f�)I���
���k�c�7�G` �� O�Z�?A�"D� /@�d�����#�SA!�:����!J��_ ���������Q��j�*4�R
�E��t�3��d�������!
���L���2g��C�/%�����l��y%���_~�6��q�&�7SS�| �j�g0���|��hMnD�[���"� �����rgMF5�Z�K6�,����|-����v"��,0�+'�����d�f�'���"P#D{�'������4��pg��K�Mdly���
p�����22�!�i��� ��DMZM2��-�g\IG4��B\TQu/#�H���y�dBy�p���z� ��e|,���v��h� ��,�R��,��� U�|�=��>�g��4_���b�/� � ��6v?�6j^�a< ����}_��EI{�i� ��'��}�� k�`OJ�d~Ep�r�`4�{�8�����
�v��T�9��{� u�1�s�q�������Z�Z{ `ar�BN]r�� �Dd��B�d'*�z���=b�-��`7�
�
#�� �q45�|I��E��WQ]�}x��V���Y�*� ������jt�����t�O�l�9�#3�1t�
���*[�� ��=v�C�l����:^j��3b� 8`���I:���Sb76��9nj>���E�d n$����Vs� ���8����t�y @�N�@|E]��:�j���4�9�K��6���r���y�������1�~Cq����R�� D��Q�!7E,m�
�V+kV��7�H��[N������ �oG[�-%�6;2���� ��@�8��r���k����������� �� � /�PP���zz�y�G������D��� ��I��g��#����E
e����A`�� �{��[��v�h����c�D�J�)WiS����w��T���� .�����k��y���\�k7��1�#PB+oI {]E MG�!0������9�)b�1��&�r�EY�����p|A0kl�6���29)O!*"I��}4h��]�j
�JY)�+������� % ?
�d�(�/�!0A-�
6��'$�V�r)n�aJ���)�9� he�F-�b��h��0�
>��E]�� ����?S�L������&�A �������hj��~���z������Y,��SP8���bn�/���i���@w��h�5��\���"N �W 9!���.pw��!�O�����Oq��E2�&� �/H�q����x_�� �}~�=U��3Ejq����n��
�l�����s�S
�9��lE=��1�s)��)�H���K:`�|y8�����Q2�}���S:�mS(���#�1��g(|��G�@��!b��!� �<�q�VO���U�� �)Xl�d2�_������2�7BM5{j�����j:g��vI l Q/���1R�� �5��G�x<��p� �"�n�,hG�
R������b����
��N��mq�p�Tl�CS(��6<~����c A���_���
�l�b���3��@�����������:�g�"�fN��������N^2_O8v�2��p@t������A��x+;Q��w��{�����Y!�= �|�����,��$��$6�4)3{c�������� \ �����H�J%� ���Sp��0f�Pm����B���9]>KW��"G��5G�o �A� 7���`�O?��:c ���Z�R*?� p"�>��Q�F�_�}S�����(�m+>�B}�#���+��ft� KU`�+��B����9 M��������� �@����x�RaS�%�a�A�M���#��"�{2U�S|�HB����v�!�]�Xpl�D�gCl�� �4����j����EF �� X�xNRV��O�&*F���h��>�:$�&?6��d������q~C�� �{;���h��Hs���z��FdS�p��k����A'�9�LX$K����-� �5�(l]� W�~���vE�����]�k^���L |J�����|�$a�u:L �o)��i�V ��:#X<���1RH-m��l�JXR��wf�GQwX�l��Q5;�/KG?�1��8A�
^`�<��V]�7]����C���� MiT
�~� N�o'�B�?m���U�� ��Jx���)b�� �iy�=�x�&�Zw�/P^M�|i��5�����lq���
�����^��R?=��-��+���pl �\9��0� �E��.�W��pOI<q���X�`}������5�J����w�@���f��2�����^����jXY� N���@x��|S�\w��z���TGg2��6 0 �����>2e����a�n���|`^;R�v��N�<�����n�}����Ze9�GL� �m4��
9�#��w�{7������c)�~�����2>������q_����^���l��7�m��u�=��� F �?��V=+
+�l�jQ�a���H\3Jk'd���8�;�]�*4Q�6���6u
� q�4��J%��`R�����������n?�a3x�GB�CB�?BI�BiZ���u-`�KS A�T
p�f`���vWf �{��0w ��)�����Bom�� �6���b;�r�Md��l�
��y9]��������R���A� �Twq�Cr
�Y@2 ��&��e^#�2H1o@3�T�^��?�n~."\�x�����w��wu� 9���n��E��� ���n�x����V���y[YKl�e�#� 3y4 "���2���e 7��^_����4}Y��F�z8���q+��4�
5�NJ ����)�|(2 �'�d>���tm`a.2sp a�E���d���� ���I�1���t)��������������0vYQ��=n��-1�Cd��g: C���e�[tL�&M�L���)���9��[~'4A��"��R�)���$������Vm�q�l���r�`dQ(��� �w�M>�C���IN'�&�#>P��e���������4���[��e�Y5���4�� ��>r_"���I���0���_�K���� n���������Or��6��Q!��m���nHs������VOs�W�T ng�L�%���g��s��'���f2 {$������[���I������p��T�.��kO�nI� ��fW����2����m�
��}:(t ?�+���9� �����9�� *�w ?�K��[;� ������&��)Ts �6���7p �S��@~P������ y���� ��5S��~�4�S��h�JC{ey�s��'i��44i\T������ ������hzk �~p�����������9�uS>�����|tX��������|� ���$W�Jc~�?P���/r�]+�����0=��YQ� ��w�A�4��_s�3/LBFE������ �aV�������<�!N�,�4M��ge�f ��� �F�*v�[��N��fF+�0���y�%|��E��I�������&�rhn�JmH+����s��o ��V� 7[��������Z�,����!��k �et�0�X������ J�G?S|���@\ VR�j�W<�Hqu��Y,������`R1o� ��u�� ��%�����w � %|������9R ��L�j�i_uE � d��L'��D�f��|La g���m�&�AL� �ex?1�O���i(����,������4 ���*e�=����Yy9����|�R�5�����Ar9w�������V�k[�|0�K-�,e�k�������H |=t��:����Ue��V���J����
����{kz
��x����]H�M�j��^���x~���7u�q�mk5}L|
�����#��(�p�d���e�g�zMn��R9G��/} �n�����;��8�]?����
,�.8���s��7�h+�������<��K �'�7&6t�w!F���9����~S����2� Tl���:��jlza~����g������B�?��W��CE��;^@mp9m�����g�`Bw���%�t M������As���N��������H����+�!��i��X�_(���� � �!���+-�`�,�%����PwE���b\<�"|�V��e��B���Ng4�pf����c��Q7���&E ���8�j��� �:g��H��J��J�.�C��8��6����?KN�R*CCy008N����%c�s����s�����}��N�hW1RF���,��V$����Jr��E
�� i�n���A����o� TP�5N�@����#L0;�IAbK���t��q��q�8_\���-Tfx�\�J�K0(V7�~�tC AI'n��.h U��:^{��]���S ��N��T���vJ ��M(y�C+�rl2<�ZIVo��$�#�
0����[Z������&s � �R���m���V��qs�.�R�s�{hW������\Y������+�~Fg�vT��WTZ� �'c�@)���?I��z�u��r3j�,E��6N|���X�s���D�P�$��B_�&�*�7jI���et,���"��]/� �YQs�wW&��I����^`u�Bxk�I���!����2;y�����������e?!M�V��g����,JQ0u��^ 2��� c-��s&�N�7���� �^��_��6I�L'^���e3���`�i�\%���t#S�]77��D��^r0�^Q������w+]3 ��o��QpT�9-�}�vw�Jf 6~������c���� ��kgW�~qW��� �A����308l����>�#��(�x�YN�%!�A��h����rh�5�P^4�aJz��������L7j Dv�BP�@)<��1����~���f�<2�g��M#�V@�/���� @�R>�1��������v( l�ps���s��!/X�[ v��l*v���q�%��YA�P~1F �1�l�@����X`��W0m�A^�T+DrkmA�>95�8�KZ��s�����[ �?n
'� _�|[v]�����jd �s��W�`����.07�%�y���kRs�+�_��P�W:M����ZR�|8LR����3�f22������N��;Z���CI���Sm�EE�
��G����v��lQ�h�<N�� 7-��I(�������tk"�H�9������sZ(���)�����K46��=�;h��X��{I�Wx��(���y��{r;��
�������U��aW��ZA�����<���S��Wkt�>����7����z��%S��t6���F���~��1�jb'��G�j��
�`.�"z 7(�A�r�>�Ni9�5�Q�&���6����f�^�8����E`
/� ���D�)�d��DU_sL b���]�a��AE���"L�h�� 6��FUV����e�-[����WaXLQ ���_���A\��� �.*���Vv|��$@��/1f#K��h-���� �~���?L�n�a����f��_cMw�5����� +5����4c�_(4�A8�P�Q���� ��T����8iH�L�u����Yv E+�iA�v�v��+� ���3������R������*\���n���E}5r�:�g�iP ���d��H����"����
�6Hg�� ']
i�^��l����P;���5�<�CPED��w�f�~�{������)���_rew��=��\� H ^pR��.x���� ��������J._z��2���;����yQ����\������dW�9���o?�����j��$}H�����z2S���)�� ��6�N��{0O/�����L
5��n�� � F�~sT?��j��ir������(sT���M1� ����$#^*G�H���C1��p��� X�d .[�d�T3P�
��z�y.��|���>YB���P���FyM��;��PW��
��T���*M��Cf
F�:����52� ����{4��=��g�am���*h�����-A���Q��h�P����� �y�������_����R�d���Re�3c�{u��a�����H7����T��q���L��g�~(���z2��U&B�$M�!H���;��E�����������Za�ko
�P ��B!
<%�$���F�-��D��=����Z��$M�I9���\J'r�}tW�q�<��&#��Y
�.�[������Evz ,=$��e!*�S�u ������'�>X17��t��O���`a�sG���MX�D'�� � ��Qq� ��*�'V<m�%~� N(Hh�a>��u<�- m�p�()��\��|���t��������2�(�X�p���� \X����.k��9��u���Km^�M��Y2��gd���j�K�>yC�������;q�3h���`b�e�� �a� �A:�P]aQ�E�R c��B�h����j�Z���}P��$G� ��Z���1�N�� ���?��r�c���j*���{2L�:�"�Aa�� }� �:9'�-�W���M�L� B�,���������r����S
�U�k �4�=ApW�I�6��V�-�����`���\���.0�B~E�2���$I#I�m�N��v�+;�J|e��#���q�Z'p���x����)u���[�Z��]���;�b��{��������,|vRh1���������Cx�X<�1��t
����fH�e.���>ZL73�9J8M�"�1�XH%m�;��������#��S�d���u@f����R�~X��8�d��T������o2���&) n�>�=�ms��A��N�2��??
���H] �.tGp(/E����h�d ��������f���J�|���W�� �����S�`�)>�^���K.��� | "������+��pc��������X`�>��������LW��n��)�+ �� Y�]�� �$�Q[��V��� �KO��o�����W��W��AF�&�r[�^�h5L� ��k<c?��qu��D��( =[B o(����O��K8]��!�+�?�d#f%t:�!�������� ��m����WM�{O)c��Kq�j �n&+�E /]���t? 5������� � ��D��!�����1�~��tj� Y�)h�DS���G�X �������Xpz�.�OO��v�e�ya8�dL��? ��~(��,�e�������$���)�.����D��G�4��{^�
����/�5�%(��k����No���\�}��Z����\9$L���\1�&7��9�� ��F�r�EV�F��Z23��ti��c�g��('��,��Z��� }���H��� ��hL8fx�c� :���ome ]����&iD�`~��t��q�h�b�>$"E�jE,�D���CIOq�7�W���3�l���#�!k/5���
���`����'`& �k� ����`y
� "��=������fe�� � YV)
�3]���u=��r����p��� �t��^�[&�����*���(5��h!�^� ��>5������� ����z
��j�/ ��u`��n��p(c2��[�4P���p L�1��`��C�/���� =~�������4 ���2�������A�� ���J�����selw��+�W�[�s��=o�x[B-����t�I5{/�}����B����G����M�s�.�D� /�m� ��e�?�m�;{�g�r �O����rn��<�n��������/Z���a�����wP
���Zw��m�`�s�G�H�� I��'>��"�}�t�����B��W��/i ?7�ZUi�W���f�F��������I��?�7�,T}���6���(����N�w�M{U�'R�� ��o�h���}3{���T�`Ji���x��g?���\����J�_�+��'�q�To��v�Ld���}#�x��j�d?[� ���<z��%�Tu��%�;�|S��qg �i��v��L2 �e{��/�c��
� ��A��<�</�W��N�)�A!� �g���� �4�����H#�D�/:�Y�8��"T�� :�3�������+�w ir�a�
~���=/tnS���"'�B5��P{�! � ou?�� >C%�1�2�S����%���+0�hzSK7�Mw���g��h`C&>���h9��M����T����#]�6��p��)X����-2�cLp
�O���a0.Cu�Z9��b
���~��<9RS�HB
.��h�.Ke���T4|�N�]�� �fV �&bI� �+p��P��0�6���>�f��R5��0F�B)3�g{`��6�}�#�����g��2�lO������9���B�oZ-qA������2���c3��0�� ��D�e�X|P|�
� ��dB���������P\��B�?`��:#B��10/p�{��E.9�G]��Iv�]���Z�1�'�ft���s�A}qZ@���\��5�qO[8�����'*�0���
s�����w[A���Z��n=�u�np����m��t������6����-��4g�Gz����)������5���4N��}����-��@���v�����d�B��f/��vv�h<v�&��K��� �_]� ���W�7��r�:�����%�:0�58-V�r��z�Qcg���;G��\p���H�����sc:�(�G�n�EQ���O $tT���'j:�g�
i� ��|j&|n�����Q�_�!�x�]D�����0a�#�<�@q�<#������\��M�.��j���by�y �([��S�)l/&/E�sz�h��,�����m �jq��Y��W����+����A����(��7NFj��cQ�O�a����k�m(8Z�CGi�)�d&=�I*� ����g����7�h ����&�p-9���AH+�4�|�^�"�J��� �j�Nk��E�����C��G�M�]�?�@�� ����f@���m�+�hX@>%��=�8>QhF
��+S� ��V ����e��C����[������u�p"&@��/�����q���kK����
����-<V
��C��L'�s�9k6O�$#&s*+���� ���`#T��� ��I�h������Zm�ov�����vi5@��|�!�
"�#�K��#�AqZ�w�?�����96(����# *���/����3��'&y�c�9�^>�{s�`>U�F~ J������e�D��������_�["�hC�����]Y3XU[�0i�F�l���m���� C��B�i��n���]��5�4��e���C���G��J���u� @P�"�]P�Gi5ZqK��� q�����B�,��� �8���P��Mp����e:R�#X�c���gc@ ���E(n,�k� �G���[J��y�9*��LL�E�M
39%��J �����o_ �9�l�L�*� ���07R<�S�����|���� �������b=�Q����Y�[��V��Cu Qx7��&<��y�=h��^ ��?������R+=�P��m^�&3�����~���6
���[D�}2g]: A�D~��"/6(i����.'����/� �y �� �Re�M����7�W��_�{����M��� {�/|��}Ye�{��'�id�)�*���q������loK� �_�H � 1���$ DJ��������~����7��d�{ �s2G��~�����3 �}��1LU6��4X)D���:� x6%��d�N�(���%,��9�&��f�����/z�
�5�Wx�B�3 �#S�z��B)� S���h=�B- [�b5��h�b{�,����O�m�������&�2DQ�)i� �����������h��������S�-������lo��]��^�����%&L ! l�q4j� U�Xejma"�����?`9�~����w��i��� ��~`��M��X��H��V���?��?�����W
�
��3Z.^���
�&D���V��q�kt��1��Q���+�����?���] .q������O�:~���� ����L 0n� ����'����
�]�v �[����Ez^�������{��[�������,������g_�Y������Xg&xv�������?�O�l�1�-��a2.B��������W���l�'��<]B������j�����LJ�4���om��� ^��������!��3\p���ES}� �v���/����~��l�>� S���-�ov������i�����Y� m�@�����o��@�^���dCP ���hz�2���{A9���K��8�e���~\�n�1��)u��������F ds�|�4$8#�#g�L�l��x_�S�[�_x|oO�`>���} �(�[��
{�V�i��?t���0���m$�=b ~n�OD ������ha� UzZ�z /V2 O+�����~�� ������6�q$����R����k�b������P��{dI��g����@�@C�0���w��VUY�� �h��aI���fee�����e��-�Y�V�/68>���lw�*����[5r��ae�R�0��P : �y����/d�����l�o�Px�>���;��`� �/\�u�)tu2�)Cb_p�toQ���5
����<�j{�},�^T N���Fa$BV#�(��s
V�rN�����""���&��������#�4tH��!�}KM��cj9�4Ex,]O ��M��2����QI=[�:�M���gz``�o{hDw>7�ut� �e�������`�y�'��b�N�e�Y� ��9��g��x2�������{?�n���<��EB9�!�@f''G��cd�� >���Zi�)o- ��la�����r��ad�wpaA22���
^�o1 �|v�=������k�x��a�\�����* ~���#�'���� ��l��*��.��6{�.W������&�]e ;�gms3��K: T��T7�&S ���{��}���l�P������[O��l��rjna��zr�#6������w �����C�h��Q��nk���
"m<�<�����`&�e�F\����?I9%�� p]|E�/����� HY"�V�ct��at�� #���Xp. �=���b4��18�A>GE�{3Ut�xh�
u���~1�������!�F \>�9�nc��������Y�tS[+�\u��<�m��sU�����m���� ��d�s���?E=]X��h�Z����|q����g��Xh#��q�.���*G����ub��+};���{qh�=�v������Y�������^�����������Aep� Wh ������~�����r�2��G�jQ`c������l�0f�-V�m��>�
��dQ�
YS�C�W+*U(��Bjp �4�^�����#��c�����l6M��i��WbB>(��e{D�\C�Ca�,�f0�WKN'�0@���|u����)bt��@�lXp ���_����|��V��(���z��H�;��`��b� ����^��Ik����*�� T��%�p����p�s���"2 ��� �au~��q�� w�r�m�3 5/��F�
��X�Lye7FW��Lt�������?�u���^��H�G�+�x]� #�xc>p��#/�����>���G���G_"O���u��G��������� ��_7q (����npw������x���5~tGO�G����6b�&����h�R��9�����D�������w�/��=w�����6��?����������?~N����\m4���v��@�tJ������P���6�J�T��'��?+�z��j��nW+�V��U�Vm6Z_ElU�=+�g ���h���+��r|�|_mV�m��Z�\i���v��xo���V���6��V������o?����������{���������� T�Z� ��V��U�A������M��'�y 2���W����1?��&��t�f��f���c1�_�Z�r����6����{�eBl_�^cT��p��?�'���j�d;q���{&Yz����^����������'�/����0�M&�!g.��=C�a����`vu1�H����G%L-
m��o���� ���bEE�����������^����So���h��
j���W�-�*�Y{x'K�\��oh;].�H�� � q�R8�G K�<=9?��:yVDa�����V?�����~����W?[f��_�(�n���u0����v�<l'�Z��� t}E��dH _
�)��U���J�-5?pIN���[��}v���U���^�[���g�lXa?#�S��S�5���rk������ +>��;�A����]���!P�I�� P��>����+��]��$F�����<}����y�.�[�:����)�%k�����y��q �{sN@��TQ��K���9��$;�V�1�]-�����|}=~���f����HIY�el����Q�&6_%%N�F_ ��_����pM����)-A���;�j~��(����na��"��-� ���z7�Z@ �@����� � `.�Kc�������G
����\�� {f�b� 9�m/z�����.�d�@?�\PM�d��t|z��4� ��&�������1���LU��:�*��� ���,����
��<��:B�a�%���oy)lY��"��
%����Q���5��0 �]q�,��h��
W?�1��
����R�~���*k�����{�<`����dx4�M�u#������3������j�VU���5�Q=gZab�$i-� x&��p����>��}�������)k�,��������f�tR����D�d��%�F��5Kk����K
���l.]�u�e�?����r�s����-�KU�F�R����R��9t�R<�p��R�n'\��NE �\A�F�dl�N��h� �u2��&������e?�>���s�r�^I �C�c����*g�+2��W&l
��!���r.�����$,|/�Ro"R���v�(�/x�4 �")z�r��=�I~<y!� �� �c�D�������'n�pi�I�/��s�g����������[��E��gH�S�m&��6��J��+�n<�7 �k����Kt��Y�� +��4�\��~ M����x����t���t ��� YN��s���������`���8]&l-11)�(���K� /�C�l�1zYq�h�T�A6r���3��"V|z����6� �V;��{�����{��k@1] �i�Xf��w���n��m]
?��J���A�~?���^��K��O�7�� �O �j������9���R�����L8�z���(`) i���k���m�q��Fw��*�����������5������(������5�Q=�Y 6+��YU�Y~� (��a-��I����5V���%� =������:�, �� 0�'���"��{*�D�, q�!�TT �Q����7ZB���s�6��-���V3��f���c�HG��Z�N� w�/�v�E�����mLx��K�FS�W1]y�*����p��v�k��a`������[wt��z�9(���UG�-T3�����*�&�e����M��B�S�����?M���]�:�\$��bJG�w;�k���\���BT�4��yy�\Q��l1L��@t��9 ��=V0L�l���i���i���d%1�)���pD���&K�R$I #�]���-�O)2�?����h���(zm���Hp��h�$�X�JL�%��8��G3��D�k� H���QS���������N��H��
h7br��d2���d20�q K�.K��~p-����������,��|��(�q+^\�=`�IIz�&���l� h�-��p!�����=����~�%%���b3������U` p�K�u�������D�I�� ��-�8G���'n��(,�@���F(�������������/z����OO_ ���x.����2�s]�7tk1��[�U9���q����v�#
K z
��$x���� yj���KX-r������d���(o�>���`���]'v|��l��$�� ����������a������3����<���7=l2Kh�h��%��{�x\����2,B���p�s���&_��?���I�l� � �&�>�&��M���M�p6-��xc�������t�����
�[ MW�E�8�����d6����&��
����r�] ��a T����=;&������i���f��M�o:�
���>�Uru�,J�8�'l�� x��&6&�k��L��>�y��s�Kx�L���2C� ��25� �D��\N&�Z�����?����sD ����XI^���fle;� )��O�l�&}<}��Q���K����f��S�/4�P��b[R �=��
z*��i�6c���� MW�w�oP�y7C��1������z ����f��T�P/��dF3U�S�s������t5�P �����=�S� ��5RAL��g�� �tE��}��dVn�_�TsE���ly~ �M����A����m�l���|��rv
.��[�%��4��#�i��G���A�*�p��Q�����m�nDG� �JE��z���������x5�
��wd�%�^�8��E�0A(���q�\���2C ���"^!��S '�G1?Q�M:q�(�/�I�Gy��:T0����eE��� �����A���_�`��h}�7��w��Y���2A(5��) �A�t��c $t����A�<
��R���9$rG3S�x�; �r������[��,����p���j��T��v��$X�4{e���Z_�����N��l|FB�h;�(��O���]wK�� W..`��r�5v�<��u5�� gJ��a @�$SU# | �d�\.��jWc �R
Rz���NYc��oq�� /���Bf � ��v���*�%�����)r����nS�)�S����#��o��{.U��f��8S��*��p��/��n��8��q����� =�O�zp��{���������_�zy\� c������a�$�� �����K<��K\S���?�
� }�K������8�9�N}W ���� �Q\j`�y�/� :����q�U�l��6n�f���k6 �
����������g��l~�hg���XK�\��U9t��N���"�� 3kf ?O�C�Fq�&.�^9�n�LO��`Z\�������'/m9Y�l��s�*��$%�����d6~���F@^�(����o�_[��9+ 61\[l ��4��$n{Qm�����U�����!�E����t ��R��e� �A�u���j��=�q�R������y�����t9'��t1��&��?��_�eq��T�W�R�$������� �j �x���p����� /��:5�<R������*�CO!7o ����'������=Y��~���W����m������\�'[9�*45��
#� e��}���u��9}�F�92���H-��N����UTK�X\.y��������c ��L�N����9,�a��kT�����"�_��Q�~�Z7f}�_������r��X��F��c�
8*��O�8��A�����7wL �"��8�����)��pf���t�.|��)�����B�>��^T�@�)�+�� g���{�t�c��^���A3����!_b �AJ7!%<�*�%���������
��O�$A�do�s:-�<"7��������5�'�d�?��A��s����������(�\�W���fk�35��2�0��W]���V �TUO&s��G>0��Y� ]&��h5�����0��%F}8mi?��u ��9���0M0{Q�N�EC�VyM"��p.J�H��l{�#iU����N�V�S�[��u(�� ����z1�
� }��^����f8��\_���U +�� )�d"c9�� ���������&bH:f6^m3��)HC�p g���G�[�����G�6=��d��`.�m�����DbX�+�b4H�=�M*Y� ���
�[�(����&.���T��u�:�A��G?G������}�������C|��b�d Q X[�c������������mt{]�f��5�)����$X3�2�9"�9Hp[������z R����}�
���z���D��[�_M���}p�]p��/����r���nkrNu����URG��%��t��@������?}�XKIE��'-� �1-7������F|~���Q��u5�S����W��I��>�3K9��`5��%x
��%z���%�#G�S��}1j����_���+�@��p<�k3Os>@��>�x��������f'G'��#� �3��_=?
+DUZ=
��J�TOp��$W������$C���A�� 7�H�,���GE�46�v�c�5��W��?�SG8|A��r&���
��5\�A�������X�g� �Y������H/������w ����q:$)�� �:�V� ��j�������sQk�+�r� 6I�����jW4-�N�B��v c��^�,�$y�L���r���pi��DM��k\�a��p8���2
�0��{�X�������C��d��Xt��9yyr~r���^=V1���< ����'t��|��=��-5F��;�`��N�oE:6�8��g��S��P���#P�S���R�7��$����&��
�P:����4B��'c ��3���c-�$���p_���g��d�&;)%�7e��ed^[ ����{��X��g�����#Z;!�}R��� ��44x�[�U��7H*d 30^f(��
���q���g&+c�� �s=||�|����%���i��1 � �B��P��Mq|����w�hq��{W��T+��!��'0f�#�gB�/>��rkS��/Z�A�\�'��Q _q����l�5�n�M d��_%@��j�]���C�N����� ����EU�`����k�z��X�C��iS�a�~�i���-��1(�
��[
}���n���������{���h����Q9z}x�s��� 0���� -�� �*g��>�V*�1��
Dglx���V ��Q \���G ��(m8��!@�������x��������5n�pOM���L
���X9W�|��Z���k�v�M"�����$�w��Z5+'�J�GipIjr����"%D0�4�V� !�}����f�w
��{�X
�~�����j���G��G&��H�&��p�� ��v���X�OW���|rcU�*%�6Y��f��b�������z��G����~�S��&K�.`��/��� y��qO ������!!*6R��f�.1��p���T �DC*���*d�D�%��`���1��|��N�Z����D��R�c#���O1i ]��TRpU���+��i4�4��d2���G���`%B�>|�F�.D�p��5��8�
�p��uV*1"nb��,^=�#���aL���E����*� �!g���&��g3��2D�����05��N�������;�T���i�����*;���M�cq��3�E�*'�h�ruE��
"�QsFj���p%�)�|x�s�� ?�|�p�f����/�K���*�O)�(ZU���}: ������������D������4v�G�Y ��� ���<����6� �+8��7����v@��u�b�{f�m6�Em�f���TL�k�H�o��r��� ��U������:�@[0���a�m���Vi��)������F.4���o�p��U�� 5*�V ��h��J'�[~���Lhq�����Z�I��@�f ��
�Z��I��?�=��n��� ���.�L�G�� 8�~��Lx]lE�����l����dt^`��>%���&S�� ��vq��=����$� 3�'�c%� G����Rt�j4%�A��@-��iMa��|M8����FHg��l���A2������E���o���x*�d���>D{������g�t�������^�|4���{�?�M�������� �!| �����1!���}uH
CS�{� "�����) �A�=K�P�;�a�! ����������5�� �r� k]Dr��A8�.G�Kl���]j�)v#��Z����� �Q�_�� i�o ����h���_��{�g m���~>>����?��x�A���7�r<
(�:Wc`�A��� d ���d���� h�{[d��Ms65 ����*����1�&�(�n-�z�~�wz���7� &�$� �����J�!��������\��MK���bzd����cd�~qz���c� Z�(#��S����>E�Uq��D�����s��Fc��.2 ~'�E8�LV ��x������-�.��:��$� �#���]��0��xg�l*� ������wi$t��{mI�de�!����Qg4�.�o.��L��/��>����q:�/��hJA�(� �Q/����wx�� �c� �C��J(��V ��z :��= �8��~c�D;B��lDni.In8��%������C|���oZ��e��u*��u��?��S��~��O��#����� �W3A�t�8�����U2�L9��/��b�-&����J.T��vH��sfU�20�����n�\�i��e"��E������&�(�<\Z�9���t���"��&w� ��#����z�:��9T| 2�]�(�0
pg ���/����"����,�E�:�HW��� >�|6�MVW�?���u��v�bp�+��8\�|9~[�r�5���6��[I�3 ����u��Dssc
�@f #��
����� �'N\0�y1 �����O�����[{x.���Wc
�3�n|s��J-I>%�Cy����"]9���m:� ���*��\��2c�H��o;�@7)����q��9�;��� ��t0�����} ��� �{�?9��`-����7~��]5����� ��/���Db�� O �����C^��?��~oG`��1%0E���-� �,��-#7� (���o�����.H�7����8!.D!� � V �F|�\_�&��qm��x���s�5��
6����`�)l��O��� �C�!�n�� �3��
�(�����
2�M
�/�a
�� �!�����PR��S������
���{�GZj.;1��d��o >c���}�rA��ik��J��L�%�����c� 7�[�D�F)\ r,����cri��H��� e�8�(�L����P A$Q�&�
<A#���A���vv ��A�+R��"���9����B�CTY3������} G��?>e� �� *5�2)Z9���- � �(D��$3C����8
*h���� ����] �+�s��.zST��
��O�<�c|D0 � uD�J���+
R���|���+Z��I�@� �5�I��8���E�+U; �,��[������x�~��A?k�v�jP��������&7�B�D,8�g���:#���^ ������zy���R� ��m�.� �) [R�*��g��x%s���jH���HA��5�[\ =�v���&<����^���9�e T8�'��9�|f�=��Jf�
����?55���'�{�X���g��������L0���-��-��nY���-{�PY��$��s8��
��3x)W&�b= ���9����s"���z8Ri �3GX?M�f�� ����d �#K�S�,+k�j�@�W��E 5y���f=Z��L%��p��}7WW�����$ ���y��F����&��q� Y�c{�i@�C��*7o�zx"�NQ���#L
V�{~���t�w��d�{v����o�� ���|)
�`l��E�j�!m��<�w�z���y%���#P�$��R������-��bp�Xt�����RV&)������`�� ��1/�(G?���w�Ajl�I?�!Puw�]�V�n0��}X�rt6#Q�ni�2�2 �-�j�i��J�����5M
�IA#��b Y�����Z���i��b�Pn@��A2�*����bE8� ���q �/S#��'��H��.�3i!�C�e ������{E��a�� 3� �������f���p��sPZ��ca�3<�.��V� �(�^�h��m��_tE6��h<?| ���~!9�� O@w�^
�x��?���J�����M ���'&!�] ����HNO�w��SX�1�c�(~"t������4�VU � (o��n!�/��+�Q���%8�0 �}��{����%�[i/�����w8�H�!�1i��_�������5 #
��l>�IH��)��1���"��=Ee��S����f4���=�!o P[��uuC%_[ ���`�mSqT����������I#�=����:$�="��$5z��'���2{�i���t��S2n����(eT�s���s��2�.�
E����l1~CN��i[�|�2���k�D~���l� ��b]�/��~�AwE �=����� ���� ]��e�t�=>*�/���;�.2�a���\0�q������gg(W>��q9)�*+=�����C�5����T��P#|a$(G
������� 'g�nb���t����hMJ �J(Mo����f�6Y-|���W��k������ �������Z�(��"��R[4���Un����nN ��k��(�x�:������4�����L��i �8�'������7�V��mJ����c
!����XIF;�L�}LuF�2�)�0is������(c�1�<���`o�\��i��e�w,�&iR|V
5d�J#����i�Z�����b�� &|��� & �M��H}Y)����[���+����� 1���@�,��b� �����hFs�)��Y��W(� ���YE����"^,�I�=����4������02n���| ���,��GlO��+QpLY14s6�.�����9��D���<�m����Q�UA� �A&��i��V$2g5��U�����J{Z}T0� J 8 �p����x��b�������������9����1��*�� �X�6E�W$���>�����\��{�N���;E�!�4o��3��7�q�V+"�y�!|w�6�{�����T}Z
�A]K�]ch���p<��nw��<�b��'�|�&>����]RjD}fb1H����������$��rB�Z ��h4�N��l�����H����H�`&JQ�qv"���?c)�&PO:�Z��4r 1 �� 55\hlPj:[-�w7@`���!����F������+�e2"�W `�?��,��+������%\��}g��Q� �^��w�0&�4$��`UV�@���|�8?���^6��E-� �~����C�y%�9( �g%����� !���� ���rM*�����)A�?v��_m������� ��m;�[������jX���)�l"�5: ���5"��_g�]�>NZp�+"IQ��*�Of3 �!�v��%�e���p�}q��3n��rG�������b��`{ ��,I T�|�����DO�_��6%�?4��te�D*�M��&f����u |��VJ�O E�� �ko���vy��T��N��Z�
���e�v]wEHb���V�]�;��%�Z>Y& n{���q}%h��?�{�Dr��b���4c^d)�C�l��l�����T7NsN��d1$�N~y��xO��i��p%�������5W�C$�������m�� e�}��']�������NAg��= *�})�����a�3�oN���*��V��/�k�A���I�y5fE��R�i�_"I�Z ���DO�0k \�({ �� `�����;�<��Y;nr��
"����E`q���NV����� z��.���� &x%H�����~��]��Gpo��s��� ���Ez��4v���k��^Sh�Z#��U�8��?G x F��7���`/�����������r5�c�M���( �zj�b�
S�7E~I�'�z�_>/����3�E���!���� �&�aL�n�#��;��Mi���[���� �NN���P�%^
�|���2*�������p1o��ja���9 �^��tq����E � o�U=
����b2 ��E� �a=*k��k�O��0y�<<}:_���o�K����q�f�Y ����n%���R��9 U\&$*�@7e1N�9B�����^� ���n)�Y�N�������{O$��Nh�8"���y�-���"X
�z�;
�����~c�uP��9�E���/as�:��{"���;"F?[���J�`w� Ro(����bom �W��j������*f� �M��I��LE���Z�IA�"� �� 0P�R1� �7Z*�r�U�m{z�i�[@�G$r�.J���$J;�DFA�c�XL�k�P��e�*I|d�H��Gj(��� ��#)�e��^#�����d%=����Hk(VU�4UaEF�+�&�5��%Fy��J�p��g��
��c�} ������>�H8s���� ��DO�`����n}7p<�+Y����u�.��+�Z�\nt/�� -�VW��T�
$������j,@ ���s���f��2�B�hRF�s^�4����.���k�`2X���v���D��mh� �=f��Y5�&�����>�%�+#�%/����X-���DwH0u�t~=� �����<����8��E�9�(�P����Z�F�a�n����;H�_%�zt� O������Ga��6��l�� :-X�nUD%�n�����t�z!+d�\�<�!#�{x0X�/L�C\j� ��2q&��?gFNu��c�6�B f�e�.�a� �C�\�� �*x�������D^��}<���9��:r�x8���F3��������K���0�q�ZUR-� ��Z��i�q-��t�9@v�WN��D�P�Z���Ev�hAf���
��za�p��6O��5���DC|����
���*������S7��?X������������JU���=x��
}�LO'����hP ���&#< ���;"�Zs� ���AmP��Zk�� ��B"��ISZ�Q��r������ ��������w�� �g'Wz���G6�z�u<����O*��7 �D�m�q����w�Y� RA���X����@���M;��K��2�2�i70q�����
;u�
e&&�L�l5]����a
�)u�[=f�]EI�,�������g��� "�� ��rF�r:��_��i�C,����_w|�*I+ ��j3�6F���W�F'[��z���
��]�v3�L=�n\kX ���u��T��?�"���c
��L��h��1�s����xr��S"]�N G'-XY��CK���x!4L�sx��Y�gC8������#��\ ������8��]o����v)~
��.���f^����S�3����Q���v��N�S[�3L}�v�)S�3h�IA�=���^���Q�������T�@��\$�������%��R@�����1�x�1Gr�E
[f �t������������ �d�/ wI�^��U}����� ����&Y-&vK\]\���s�9-�5��A����d8��?]$o���P(,��S:�W�%���1�����^��v�_)������T���r�8�� �"g���`��-W�IBI@�.�J�� ����a�1-�����t���� ��#��� �8>:���q ���)
��w{���W�D�� G�D�g�<B�f�-�WU���������|�=V� �� xhl���_�O�_�~�]y���}����Q�,=�.#���#!j��l����~������0z�� 2�2�D9������h�Vx�� ��f=om��� W������SrV�G�5a����lw����t�-������U��$:� � j\�,���$ �h�>A�,DIk�H\+�������q�,Yy��M�����[m����� uoE��(�6����x��>��;�)�S���)��b>��U���[S�{#��6�Q��� y����G�s���%3�d<,��O�) }��������"���sb+�E�����9���y�] q�������.z[h�R��9D ��?����Q�W_~���M������?~N��d����F��������e�����J�.��'��?+�z��j��nW+�V��U��l5*_E<�@�~�t�_@W>D[���^��0N��j��m���Z�\�7��v�S{o���z�\k��F��o��~��}�m� ��l��O��j�Y�W
?���j5���Z�����I������O\�>dn���S]���b|���H�/^���:�d�����-tb����('��v@��u%�[���z�}y�P�����_��Q-��=
��/g��}B S��Zk�[({),u�O�R��O�����Z���sY�e����al��Q �xH��&Q���I7
�kn}c� �O�F���6��A���vcqF��Vn� ~��K�:�e��.�}U���2�B���0���C4�`���J�N;�l �� �����k?�����n������w~V�y����2��# �$o�)e��(��4[�S��E����� Cct��� ��t�� � �7K���� �S�������������rUx����Z�����
H��S[�Sn;��4������nC�J�Q$SDK����Z�����i]����M( �w#��*S��$�|f�~}&� n��� 7���Ykl�! �h��0(<1S`/��~�h�h�.wvP2[�� !�qw4k0�zs���!����z�( +�F�s�<���v�lt�+�J��Y����g ���Ob��o��d�Q&w���'�@"�S>�����:��*vn'�})���z\�����c�8�d�+O�,��NR�0��hO��/�6`��n5;�r��� �F�@�.���
�� HA���
�`X�j���� |�
�~ p�O�.�;gK @@���X�$2O�����U�����p�'O�M3�|��8��#���|Y$W3�~��
}Q ;O�������`3�r��/z�/N�z�O�����������i�R�cl��F��W������7�j���ONdD!��4��' �E���3�������7^��p4qAm���
��M8q�>����eQW�V]��r�:N� �t3�dG�\�p��O��D%D$%t^$��3���K��� ����y9� ���Tn$���������U��H,�������@ PK�[���s��_�oa���A}���z��~�G����o�;3�Eg�64ZpLP��z�� )�|1�B�.8WA>���r%m4w�Y�� #�����v���j� T��l��T���LS>��w�� �mqj1��`�DO0<��j� W;�<N�SCra� BI�����,��C�/�q��'�=oK0�%^ ��i1$����$f�K� ���F��a�=��El��L@����)��N����?�bk� ���P��>��?�����M����s�4��V7s�� �t��������vf N�z����Z����iqR���PKIG�{�/�[ �7�����Q0�B���5�u}���!� )~
o&� �� �>g'����DnpG�w��G��B����C����eZ�V��X5)��W���j�r �$!YHG��e��G�l������X�7`��� H��+�������Q�L��2"N1*G�YF��U� �VgL!�:�Y��[���oJ) �� ���|~��H� �p��(FEetE ���|n�C�ou:q���L����r���%*L^��J�| N��bL���X(&~<r5B��u,�/J�N ��������� }�?��XL
�:T�� �f)���6DT\�����������)��0� nTUB��[k����Z|�wy��Z�*�d��L���IC+�YY�~x[��4� uF)�9r�0|�����S� ���GQ>����(WUi����H3�&�%���i{8�TX��G��=,��a�� ��D������]6��[5} �|�E�79������5��[�)o�F�����2+ew�q>���KtS�7� �^�c�3w� X��Lj�(-��8���lF3���~a������G=!�D_���j��� _�������c��a5����L�������r� }*�]���_~�I��&|�����H�u���,�K7�,���J��C��� ��j���2 ��}���n� 2��)S�iL����?N^��/!uFs_8�r��+�l�{l`�3j�����v���j���J ~�
^X�����-�}gEY��R� Q�����)���!�>���/���Q� #f����.�������1��x� qcg�khdF�O����1�:��C�FY� �8Rs��w�f�Y��0��ODP5c���g�` /�:���S �Ml*���C���l��$ 8�o8�7$~"����F��������� ��������p!�gQ�ZP�h��N�0�T �$��r*i�����e�������������+�T0|�X��}2� r���=�_�f��"r����&>�����)��"��LU���:�X���&�r�
�9K�v_|��TC��5�f�(�@m}LMa����$:|}b��y���0�����QB�(%�h�r{��N���8����_ }p~w���y��� �+���:}v|�X��P�T�B3����sFx�s��0_����92��-L���h���4��������g? � �}V���� N� ��B[�8c�=���>dP�T�0I�v�3m��� M�����`(8�] ���z��-��r�7�Z���GZt]|���'��� m�:0c��S�� �g����^)��o���5-��"�J���������,&7;� �\I�d�.Y��sf���2�� A� V(�t$�i�����?�
�c��8}��b��p����
@:��N_� ��f>vY1v#Iu�d�)���O���� ��r��G��$����[+�0,�@��,0���w����1Z�HS0[�p ��\�j�
�n�H)h���9���_��Um ����7�����lGDU��q���:]Q2��(J��+��? ���v��$��3�#�oK�����|��������|�{�q��SD�/&��(_��+���5�N���.� �)}#ra��N�X5��g������� _��B��N������]6tx�+��{��M
��Sm6��i6��~>����������U�������j
a{��p����#}��!���4-R<(�� 8`[Y�uwm��R��y���e��R`���Q�������t�`VMF ����]��N!�~p<�*��-�T6�o# N�dh���NJ�P�������VM�`�7o�U��e�����. ��lV�q�Y�
�1�`c pHk�����^�v� ��-���������Y:��t�,� ��q�����`����K�P�+x�\���u]k��� ���Q!/�� ��'��N���`T��I��k����s�����6sB��N�G�5t�c �)z�p �6��� �,��D���9^�������<:�t����Z��2t}��������$�<]� �,�2��7��)b��PH���D Ek�V�EU bW �h'�cZ� �R�
��+H �j]�7+w������6 �
��0�����3��
g&X��Q2b�%q�@�����c �!�-e�(G_- |RS�u"S�Y.p� U`��j]?�e�Q&%��2�`�6a��
�crc=����S�� &�E�0�
9Z��H�7#�y,9�~��Jbi�dPIm2_�"��:is���_a/��� X����8�w�hP���SLb%q41hZI 2K4 ��� o,A���j~2�qN.J$m��q7�s>��� ��� H��6�H�>����W;��\����4������ P��+I23��s���v�� v��� Lj6�)?� �l�[&HU'@�(`?��u���J�1�h�Bi�`�����z)�)�-~�H���U��(��V��MR�3� [`���M�L�����x2���ZR�p����8� 1�����e�As��rt6�~C�*��?Y�:j�@y]�t�%%��d�i� S�`��A������� ��Zk2��#j��n���*�����[�:ZY V3�Y-(�y�#9���}_+G'6B17�yJV�TL�01���'�����L(E�0a)2�|�.q�w>�'� ����� ����X��> ����=b����9����Y�����!�e��U�����
�z�]������u1����T�6q��r�iM>�M���|�5��d��UBF'x�v��9�A6�~�J\)Q����O�W��:��\fz��8������K*C���2��X�U*
hN��Q�(��&+�kp���L�o���� 8�H �=��X� �}L��?� p�m��q"V���r�VD�pP��X�Gh++G��A �V�F�.��o���F���1;������0�� ���8�6e�@���]"����
�.���WC�0�� �;���� _7����Q�n7��*8�N^� ��<|���o��v�o����YY9'����%�i���k6�� � ��LD>I5c�T����q�� ��� ���� ="3��D
,M���!'�`%��Y��������^y-+�B{=:�a9q�OQ2��� h�)������j���b���d�lE��/���&* ���)��<o��q !���4dk�C�!����\s��85��p�c�|"X�;A~=� �=a��!q�l����?�s��
��He��
�^�p��J�H�5�������-�����s�������X {��k�(��"U��3�����j�R� � �8���Lywc �H���6q�2W$�Q��6�� yAZ��o���hi��e�����}������I��e���T/S�CT�-��gw��[�`oe6ia�'�Jx7�� ������`cm� �cp&� $v$��
����og�� 9&��X���l�E�J�w�G C��7�����ssWd9;UX�n];��u!]-�9
e�e ��r-ax rCf*$ �9�� �m
;�N��1m� �y��4��z�.�k$^n�l���'���&�S�r}��������B
�����,�7�8�{�OJ)�a<�~�����O�"&9�L=��&�u# &������Z3���M�����&���}-p���c���K�0� !�[bx���6�71$��dR����U��J&���Yqf��w��1��x�/~��R^���t�c�8J�z��-�i��R��B���^f�@kJI����!�����H�.��y�V|B�Mk��9j�A��>A� L���q�� ��D����0���d���s��j��s;y��dJ�<bC�G����]*a����r'8�����P" s�\��O\�L\��"����j����J{�����g�� b?�a������QO�}5����f-q=�x��[� pf������)po�I��qj�ortf�4Y����������E�|�nu1Kmm������/jc��liysg��Ysm�Fz"[��{z� �� ���
����u����Y�E f��W�<�$����zD��K���gE%V�|$����-��$Fw��5���n9����� 95��e+�/��a���(�5��������������������u
�����!)�mP�AM��;� �����n�� �H>����|��\PH�B�=�$��{ ��J�������vv��
�~"Jg���S����w l����6������ �����z�������U�����'5��x� ����[��U��=�?�4l�J1��k>�l^#���+.0�4H���'{�O��g��� ����Z���\���A2}~�zn�H����(0C��
�J��8 ���O������~��<������������pSkY�gAA ����{��-p!������E������8Y(e��S
6Z@�M���i���C�j������=N��g��������
Q`���3���A��� �0t����<��2�Z �����Us�Tnb�8���P|gD6:��Q�I�nW��������:�Q��3���E��d�����H,�����h|M��
d���,��n\7�4m8f+ �� �� ���6�A�$f�e���m ���P�� �x�~�u���fw������F�1J�4/[��8(.R^�
�I��B��,zY�
��U�60�p��K��mx��Z�����izm��0���SP%��q����q|�%�������*'��[�p������a>������:L��e��_�q8��L�
p1� �2�:�<!���G��.=?M��C�mI�]��E�QWD��s���� �������_1���p�k7�����?�|�"E���N����������oY�[!f �J���
�C��J����T9� MB��7����p�B��'� ��$ f���x��t�0�>�I P���\��Yf%Pp�33��w������O�fJB��>4���1�7AHi�P�
�������%��T�����U�|gq���������/����+� J7 �c v����������P���$� �� �QmK���&��?�����L.���%z��<fd`Pi6go7����e�d��w�|v�k+���1}�N2��kb�RU�c:�ee���!��% L ���`���?l�z]}��< R?� �f��� v������N�6����&�-��E�������>�?����z �_������P��O��E�<X��
%�E�r��paU�+��bU��~������NxEYX�����6��#��l����S
�������3F�ht�q�U�`������g��xa�'<2@� \�m{���-�=m)�� :�&T- X�t�
|f,
��?�����Fv4K�SAw��l���<:�����!~_��.WdeD ��yv�v6��Pr���e�x�yI��N��{��wvi���>�>���>�`�@��6�'����������� �M,����l�6Z�M�����
+��G�<��U
��;�
;P{ ����W�)(�{^�t< ��xq�&K�3IX9B�x�� a_����x6��0(�W�4 �����~��|��K������r���V�� ??���y������O�}�;zuv���E�������o,Px�FK��v�j��t����J�Lu>���x�61*���hR��{��^@��#,���S��t��V�i�_=0q �n$�W�bIf6�b^0�m�B=�K9E���7 ���>��\#� �� (�����u��JK{*�1&��?�@�L01����S��5R�Q���V�� g �F_N�����n����)x��`.0l6Z�O)G�_qB~�>����dXrN`�9��X�XX�����������q dk���5d �.c���>
�,j!�����f: \�������&�(�SJ~"��F�$A� ���-��� }�c S�� Yn��5�2f�b�8'����Kt�P�>��P ,!�?�� ����s� ��Vf��(������@C%���
�cBP�}a=�'� ����Hc6��4��} �D����$+�# �Z�c�2��'&�-%���S�:>N��
� 64V�}P�d�!�~�������k�� �\r*�������ZC��v��L0b �Ng����e ����������\Srz�X�<�s �TY$C�T���.���
e�
����_��Z �hvT)a�����p����Q�$@@��U��C`���;���KDE0����h
��f� �`�gQ����n-n6j9��(����A"��s_yg�" {a���k�����
���x |����8�u���h��� .� L5%� ��l��e����?��v�@
Tl�h����s��������E2�Z��"=spX�jd}�����bR��������i���ImR�XLp��(�mw21����i�a�����av���� �� ��D���\�'�3�Q������dP�%���Ul��|��R��9����S�O���o����<G��g3��3�P��8Dv�^" ���|K�2�L��<�cf
w��c y�e��I
��e"@U&��L���}�7���C�Qe�f�Ez�M|�47�N%n��grtI$���{B
@)^��>=�8��`xB��<�����&3��� ��C{Y����m���]X����}pR���Fi������P�Oa�l'�MVP�����H���[���B-�8�5��T����F����o������C0l� ���W�j��<8�8� F���]���l���UA��l ��G�lr�E�Y ��D��@������s.t� ����dW�=uW��h7�5� >�S]*E�|�;�opPS�?l �d�#w(O �&� �t��d����?`�h)���j5�� z�M��u�VK�rb���q� (Paxk� o�@���%.foV)�����H���?��k�Q��Y �Xb_�� ���%FE�$��'����1���,\�+;Y�� P
�O���d�a�� ��!)Vl�R���~qx�R�,� � s� 4�8>����c�����p�Fc��,��)�6����������x2( 082����:���=�
�
�|��~ ������������V�k���a>�Z ��H���2�88Dr� �l�r)wD ����1� ��� Up�k
I�e((+�:p ��%y��\� �=�u��F��y=���o�������#��K�V���b�����p6���1��/
��da��R�V
�}4T��a�G�;;5���}�N�1�(�m���uwR~�����t�,�S[Z���h��q�E �m��K��|f��,
X&�N" v�;� d\ )8
+�D�d��)��� ��]k��zC��|� �
]�=kHdm��F���Sq��$�Y��9 ������Zk�l(��.�����F����>�"v���B��.���O�C\��E"e��?��jI�����V:e�� ��77��"���t��L��-��S�Erf�_��/)Q43�Z�M�+v7.�����I���;���#PYX�gy4�Zf%%�w�V�?zx�VK�W�T�'�����]�}������!:g�m
$�h<� J���Dt"�5��D(\�7�iB�������4�
(K�^%V�6��,�h���&,��~�{7C�����6n�������?������z��G"��$�` �� �b��way��:���{���i�_��?�����4�N���k���m4���R����,�� �������� -���*@+�X: �v�E���cR�J��������jq����x����>���j�^1�8��-Wg_�Z�_c����"��j�E�%���^���3��kCQ�4�g�|,�Z���)bt��� �����uvs���HF6E ���2��0zP�;v%� } ���4 �w�ws��:�1[O�Y|�$�����3���LAzJ%��*������X�f5i>g`n������S�� ���nu�d�f�����~ ���pt;��$���^D��&a��������`RX���������E�FT�c�����)� ���9���*��1����qArxfX�&�4i�6�{��/Y�H���^���. ���6�����" B
��}�V9N��Y�Q��1��Ma�*�u�t��q^�o�����V��&��S [�0�g�]Cp��'Gs������������3_���f�$�~8o��<���R�+|O\��4�\�%< ���h�vo� (�q��� ��������Z5��Z.�D�����SO��E������ �'I_�B� "=O$��Cj��/g3N��#GGq0��(Q�$>�uQ-��8���B���C�[��� 0vM�/WW���O�A���5 p��0u��XL{M*�V�G8%�
]��� N �8X�`� �OY|��p,X�o�HF��~��[w��a7��U��T+ML1�q8i�jo�i���r=�������Y�#G �Y�0co�� `;\�#�� ��v*��V�������� �=� ���!�����m6k�
l����� � +��vM�9�&��[s@ �����b�p �"��� W�(�o?uG
��N�y�x�@�@������_L���n l>$�X���������(���S���t����[ �X(3'��l � D���m$<?�G�����j�������]���_�ak"��J�%�k[O U�H�uK��}�� ���79�8��^��*Z���W�O��u;����{:O����Z�Kd[�(>�����:�D��
��S����� ��'k����}&p����������C;����2o���AG�� �a���6��L}�����\��y2��� Xg ~�NX}E� �@~������V�U���c�1�����.uT��G:Qy}��Q�1�//�`��%Uu��� ���QP,� ���'�D���5���
��{�I���b����5���w�%��OS�|U�F(�l�0�� ��R0�Ng7 t�H�hk���������~���l=|�b�&`�CSwP)J�X����+/�^9:_���m� ,g�L���� �p��,�MqN�0�������o��e �P���i?�]�E�
����_:�@2�V�Yq�� �7��Gw��t�����:��n�$i:`�&����"�u� 5c=�2�IwL�Wo9� %�M��F��DBa7t�$=�� ��s~ ���Dk
o�Y�~��c�Y6r�)�v����a'�~V�wa��V��_�P~
G� >7Z^a����J�m?V\'����L8�D��S���F�R�]a@�RxB������4�,&�)�l4v0�*q�L����F��l�� ��,2�n�����*�N��[���:!�s�MU>G=�1������ [�����$2�50�Z�����f��Z���4W�H����. ���� �������]���d��� 7�'�p*FU��}������r�23�8"� �����&��l��� �"����*�`JV�x��x����}�&�z?W��"#Y��|��E icLo�K!��|o�����N�]��A�d���!�zk������� ���6l �{!m �� m�� ic�!�6��B1�Fq�B��|�
�����C���Ao(��;�o��-�7r��~ 6# �9B.y�d</�� YZi�& 1�������,��y 8�=|Gd��M�f��-�����l�?d���=��� ����p-��� �����l�� ���:d3����f����S������l����f��F6���k��,;� �}����$��E7 Yf>�Y@���`���j}&����,�#T� $Bu������w�P5"����B*Bu���X�-#T�@�u��=d7F��o �]���]�����<�4�Z�.�L'${3�m��E�����f���7����4+� -� 1���;��k;B�~����R����������}��-�#|�Y� �5�Hd����>!'uf��������� Rm~r����N^�6���������_ �����ld��� 9UH�������ku��>��+�>8?�e�}����eT�'V���m;���� �X�-V��������
����� �=�J~���Y�����
*��L���tC��'n)���D�#�[B�Z�_80����=�pn���$�c =�l��[U4.�|�� WC�.��� � ���#���g��%���X�H�A�� ��LM�����}���nB��U������j�<�AQ�]�������5UD ����1>�A�!����LB��nJ�-,��t���D� �C�d�v��:��"��| r�,{1n���_Swv��[9X_�����AV���a�fP �<8�������Z
�J���N�7���9�c��X�
������5�r��,�m�NA��U�����l���AoT��,fi*.�+t�� ��&��<������I�j�LMT�
������ ��K5�cH:��xZ� �un��� �I;5M��V��6"���X����X��z>������ �k ��< ��X"�n}"tO����d��TLc$���=��*��$?@�>��� �d�2������t�e�����&9<�����=�����,+6���2�yC� sb���Fv4�����H�j7;���?� �'������7���j�P����H8�����l""E���S''7�������Be$b�N92q������^�l�f�u=S�sn�����~R���2=dJh *M�C�������ZMMnKY� [��3�h�GT���F�4��F�\�@���< r��xX"s A��9�9u� ��M�A[��N_���w��O�#�;|�����H���yC�~ 'J�;w�p�# f��}4��{F�� ����'F��7���- ^��L
_H=��H���qL��E:��`)j�(��@I�I��G�����r"a��[c�?�p%3���LO?���l�����
��]�o��3��bV���~b���k�j�?
�B�Ed��u$�"l��O���L ;�M&���|����B����M��&�b�
?qS�����;�����Nt�MPU���0��{�H�����y�q�h��Vj�%�:�hS��:��_�=�e�V�Ew�5�E?�Bn� e��a���������������h���� W ��]qz��W�����Be|Uy�w�U� ���q�G �o���p�.]��K@�Wpy�/�� ����$ ^����C
�I�V���Ag|���c���5�w��Z������4!HL"6�w��k�q�� ��^�sz��LidJ ���w�s�o�����2\8����{Z�Z� \����|���1�&7'��H�k{A-L�
~g�}L �������\��nZ� r
&��<Y-�U
����W��G�������|l�� ����Is>��T �� ��'zF�V_1��"��j�P�������������S���x��t�)�T (Q��6,6e�'��(��}f����B8�z�Z��]�Dw�^�O������X�}��R���T��]*�j ee��������X�`�������GF�s P��[��x�p���[�+��e���?��x@=�0��7��������vH&���
�H�HJ�,.�gD�J� 7*��J|��`����'�E��N��"���d6�M�<2&���1(/TQ2��b:��q���$�]�,�C�S��&C���W$��JT���M�20���go7�Kb1��������9���b�atis+��0F=c�������yB������)5�Z������/��i�rS�" �'�U&�y�#:y���4�^ ??�q����9�s�=D�~=?>��>��w� ���x ��E"H������������D�������%&f���Fc��h���\2j �!��*q����|'%���`�_t���O -������ Gp���0X#V:��������Q<d2%�D����K���8bJ`�lQ!��r��sM��� ?��F�t���L�H,!7�y-yl�9���m�0�z������K��]7/G�����*^�x�K����<]�����\7EC��h��]�g6���%�Ej��o�>
:?'N ��� ��z7nT
Bq�z� 7�
k�>��t����lUn��� {~��^� �jl Y������Z0����J\��Rx�E2�.Q���`]�jC2 �R:������=����8�*��!N�1s���h�����yvM9l(� �y��L�^�@<���J�F����7lRv����}��:�� # ]���" )j-��������`quv���z�������z������X� ��
`P �������5D$�<����d����\�3$��xq������"Ar�?�/{HC�>� �Ghb>�_��&W��Mo8��k������([�����8y�����1�^�?�@s�:r���{�q;������ {z<5���� +��x4LF��g������������G�q��'9�2���q����i^��N��~�u`$va6{U�� �,�FWI�e�7�8��R�G��\aU;4��������S
��"���Y�,� H�j,����(��������o����� ����U�|Cr��v�o��w�����ZN�ZW����]�T����m�Cq�6���4��$�0������r!�m��<.�p.�p���d8������`C.x�7����b������9P�H��!�M0��d��� �(%�1��X����"}��6��&H2����Ga��MX��K
�����w��V�!������� �B��m��e�z4���}A� ���a�#���n?����S��%
H��vF�:>@�r���Z�� � sTz=Z���v
*+�X�6!����w�x���z�����|�w' =�S�<���{/��:�]C�a��{�Q!Jx�� ���$��A�.��g������6����re�����$+�KF�kS��5UA�����W�O�+D�Cuy��)��P\���P�1���2�+���I� I3 ������HR9~��y=d�v=�����[�G���8�.� o� ����XiA�l#p
�E,�%�4 >��f!����,T�*���T���,?�js���R���h�3=Va�#y��}jm�$/3Rp��j�(2��1,�5Zx��������Nkj_��`w��b�H��:i%�z�R��(#�E�RmW
8z���w�,��������w��� �6�f�@�����a�r#]���AMd�9Z�}�v<g-��* fWpy�#���Bs�T�3m���|�B�>-�J�[�����xLv��Z$��i�CY��1�K��92�->�YJM_U�����(�RW:�C%���o��?J�7��n�ggr*�'R�����6 �q����WD�@�i;�������� e������]q��Uf�JI�3e�/��s�����!-���*@Ak�e��^�s!%�i�P�IhAZs</ox=�7�\f��$"d!���j7n�=u���?����i���-fY ����� ���j�](x3�(�1�4�2���T2l�7��p3g�)�@\��l��B��>|�T������$� ��
���`F[p�[��
��JXR�H�tM��,���������m���Jw��SS���\��� ��i�Y�P��D�~��,#Z �JgS[�G�5�R����*L�a�yV0{7�����@��*�,�:]�pv�O"�M��t5f�D�� 9���9�&�!��{^q�2�]����s/��`�H�0���C�fp?_�oS�
j���{c"=NeO;0^����
�d�~��A���+h�N+��G�<�=;~q|~l��6�������4���`�����0��s���BX^�d��1�\v=��V�Z�N&�7�g����z =f�2Z�q�l�7q���^;��� �_��qH�m�u�D>��|V`�������4� �:_u����"�b%
���� d��E"J�E��j��3��Ab�9!�N���}�@���t����������r��� ��6��F��4�:��_l�h�b�38���xM��$�,�y,���� ���dR�26oa���GE$��qb�b���.S����d�3 ��A3�1����(�� :��������&H�u;6qK>��(���5�y��19�|hn����}�����a��~o��n�&Q��> ���%8�s�K>��c��j�E&����� �����A�NZj2R{�(�5��v��u������G +1 Iv#��Eb�,�8U���d�v������}B�����w���./��PI��"�T��cA���^5\a� ����/-�<��nd�8� ���ujL!�����vM��<t�l�
�g�����x���v�}��i=���eb�k"]Y�����&�x�j��*K����Q��X�6��� �o�M�&,B�����]�������p�D�����
�d�f�ih�|#_� �A�5
�%[k?`k�Rn'�������f�Ub�A��� q�� �����X�,�;���r^��0�|����dy�����cK�5}_�H��ymGLiP:�����Ofb���(:
_�E��~�)r���([����;�pL]����NV���[f������pe �����n���I6a*��pf� g��;:3�c�� s�r��l��hC�:=e e�����C=���SN$�T��e1����.>{\K��������!�L�Z���o|6%(�3DC������n�=b�IT!; B�6r�0��H���}xc���s�8 (�_O��(���y����y����1�G���j��i���"mH��j91g�{|���y �����grx����^�8�F�l� 7�����)���}���. �� n�kV���zfa:g���� ��aV(V�`���J�Q�7O����>��(
b �jTg���U�>z+������Ho�����%D�\����V����'@���K>�x���
F{�x��w����&����r��~������vX������u��3����j������� l� )=��a��mDA��>�����Z�6@�Y{5����"��C���
���)/x=~2��34���`����tx4Cr���X\k�\���X�d:D�'D$F�$M~��� Xo����`�$��F�~A�l �}"���X &�c�rLA��#�yJ��Bn����� ���t0 ��S����6�Q-��|9�0V��v�����3�9��f`��@E����h�x�P-�K���s�]p���U������{�J\\�������M���J���l~c��� ��(;.z�s�-\���bI�[$�TO]�Q��j{N�+m ���n,� ����5�s�G[T�FI�@���s�[z�(�<��! O9j^���� e�IEAg\jj���r +
^4
���8.F��T��m�YL7RK��7�Y�������b�q"���U����#�JU�wj��`5%'���"�����J����?�m����5�c��I9�=�g`���oy���l�,?��= _�)����'/����&���\��w�eu7�,*�.]0QE�a!B����l�����N6 ��L�P0���v�� �6w��Lq2��]�O� ��6����� �Two ����kv���/7�jT��v����� ����O��S��r�:�a�_��<������.��
P��� ;!���8&���k��)��k��)�\Y�!� ��� Dp��Gzk< ����`���O#{��lZx�������+��SF���+�� &����������S��E�eCx��t -����&�����]x�s#/�Ey��h�ao?��6KuY��V�;���m��>�����J�a��G��#_�qj��=�j��C[� }�J� U A���Z}���Te-g�,��!g���nP� ����� �C
�"�� �� 6��� ��7��� �����J�ra�����eP�����c�/�?�WP�G��!��E� d�CT��H�������a�� � *g���=!�~�M�����X�h��
��{��] ]��t�r6LJ���������T @j�����<Fa.���� �Lo�v��F��2���U��@�k�z:J��1������:�Fp)��B������E� "��J2�kT�@����{���.1`�VaNz��o5�0�.��6��V �<�$�����'��X&s����"�p�"�?�A�� �� ���L�cfF�F��V43���R.�����d���W��e"��Q���,^�� �Rj�T�p��xu����DH
��������Iu�
���I�������vf��se�B�D.���9��[�������k�����H� �l�HCB��}�a& ��3���C��Q�.8}�r �YbX�sQ���Q�����_� �M�������3�r���n
��=H
t�$�2���t������Sz B0)h�I�ZA�����C���%�����?�5�N�Y��m�v� 7���BpR)�u�G���rQ��h ��L,�H�G�M� bD��WNY�u�u
��Oh� .����,��}�u$�K�P"����-�����WP�Zqy9 78�4���z�L���w>����y[;�V���<{N����k���$��aV��+>��[������8&tm'; n�rT:D�V�
L���Nj��Q�����$��Bzvp5�D�-��f+q����[]�F`�3�%�k ������'��� �:��!��FFP�<2q!����>��_� T����e5�-���k�N��v0���Y�����M�g�M�= ���R/�@cTy�bI"@)v�TI�(�.��-T�t�7������J >9y���������}_h��
���}` �D9���7���.7���X��v�@�]/���Y.�q~��KU�}g��D��������
��6� �\�n'kz���pWu�{b6k�v��U�V�{ �/#�c�\�6,_�����y���F$a��wk49�� !�)����QB_���M����L3LY�udM{���� ��`�}��|6�����}����;F�d��-�`~?�
5w ��������{7�Hyj<�h�� 8
���c7���3=���.��� ������/�F ��9�����s-�����z&�s|�����kQ LE%=d�B7��lo7 �������Jx9c��%C����y�����U�+n,c�.����@!�7��o�cq~����� ��Z@��.���:�|�vkL�Z��]�{��6��TT�M���t����Y� *���{�A�E��PQpr|�a '�o��:�;A�s+��!��aB;z��/ ���[Yt�bD��x� \����>;[��� ��' M����3�
����&��.� Lc��m��z�����MH��q�w�r*�e���jT"E�(��V�Hw-f�� ��(f#F#��$�2�L2������(����f�� �+��YX�#m) *�c�"tc���53O����&�����Q��.�)� �G��w���j��j4 ��������\T*�r9iw[�z������-�]��Mm�*qM�O��u>��o��������QR�|�� ��q��H A��M1�a�\��0M �>�����������~ ������y���D��9��A ������ ��#^��G���[������������O�S����?�y2���aSBOp��d�D1�� �5��Y���Z��/a�9����,���!�%�� H1.X�����[ �Z����\T�U7j���XQ�3� �?L���� c�� t�Z��|1���/u~��n�2����:���`�V(qZ�V��2bE�!�.�� ������7nJ}e�N�2W/��4[q���i��4~7V �N'-]��\z�^�L��{�-����������[x�h ��� M �� p6���9��I�a��8�f]e�X6k��NC�_ ~�R���Afp�B]�Y�0e��|���J}8'w��n-�v4�����e���� ���K=�D�SI@L� lK��uG���� .o� ���;����)<[]�3w�-K����
z���T �J1nB�&_�����Y��>3�
]�@4�v9�t�){�Z��������0�Tk��Zk�C�Z��������^$st�T�M�3�M���?1`i�d��g�������(\�S-
r]-����R[��oj��SW�>�[W�T ��-�=`��E ���P�p]]� ��]�{Xb�� Lt�M��O�-�����X�'B �M'�JB���f\u�����%���.V�8�����Dz�IB������0?�]*��M����iGi����K���;��a� �M
O�B�"�e���f���,P�d`07��O�y�r W �
��t�q�R 6�_z:�7.� �}��w�L����[��Ogp�^�h�x��{��LHQ
_]lWn��rPkw���r�� T/Z �YnW/�-�+n���� i ]��M��0����v�t{��i��&�K&��y�����Mr~19&�'�ebS��� �rD(_cB�1��9C�����@���#V�I���,�H� � � G����_�j�247a @���49��a�����d0*Iq�5%�r�s 7�@/(�PfT<'V�7�`1����`u \�{� e��%#.����q��vvj�;{�$�SG y�&����}�����e��,��E��s�����0Ix��L�XO����f_G4�
\]"l��u W���Z zW�s� �Ga :�����M��pI3�k��;Q-g��a<������w��S�n�Qm�o���b~3p�A���H�@�V�x�E 0V��� ��G:z;:�����������g���&���ff�w ��� �%�6�7i�h�5cv�c
�@JaV�����X�w}?�j�Z� �W6��������5i�qx�S�,� �@XC���gD�d�B4f��/���)I��5.�Gq�s�#��8< v-%�0�1^��B2���.�b|�����y*�m�� ���E���r�����n�����,�*������K=���~ �5�R;�j6�U�V N�� *B�dC��wn��=�V��i(g���&��<f���nQ�l1L���>��j���S�����5:���7�$@�o�$X�����s�2���o � �����8�i��/����?��B�~�����d�[��1T��?�Yr�bF��R��O�U�0��zL :�=�#���6�HU�{�i�������|P����,�=��G��z p
�����A7VoHA�TX���}~SrE�|X������A�������d2�?7]�]�u����:���r�����h�%Y������� V�Jr*p�>eAbb��F��� O��hr�FKx$e�f$����������N� ����"_���t@E?D�3�� Oy_�����}��l���z+�� ��Z]���������?�Z���B*p�]4%���G[Oj�`��� �C��{Q��/r4������n��.�}��O V�q�-x� �wnI������d|�=�<`6-�T3�����OM�������F���7�r�x�h8���r�Q�v+��f�����U��R�`7�l~�O����J�S��:�����{�s)���-����J�y���:�gs�y�OP�%8��3 f�V[��9z`$� A}��C����&�:��|51�~tRS�;� �Z'��Y�7W��7u��� Y y�e�p�x��]���q�q�
$O�`��
o����}�����KXr i
��Ad�j���z���y-���� ���A�=�/P�=������S�&:�io9z�����g%������<}��d���,�r��TI-��$K� qt��{���U>�t�.-�{ ���n,# ��J�E>>�K���3 <M�d�\.�$c���>�o���3x0�������qf��,��5Z�f�Q��j��_U|�#�a0��U��wq������--���f��� �=]]a������t��G�i���Y�N�} �G
� ���O��#-���K�f�L�(��t��D$!������&WP&��t��m��f��Q��&<�G�����f�� ����pg���x4��*kQDsV-����B���I� �L$?F���a�)��4r\����K���6����j���6��Jj���������_���_>;|���1..� �`]��u�9��)�]P *, �yI�-5��y�mI��A����>�V,u5p�� ��@�6��I��;����t�� nO���<"�1����������f�EC�\1 �����M�?]���\����Y�z�cf����xl�`������y����MczRU����m;Y�l��m�?��E�b���[�k;:��. ���tw+^�
y���<�P zkv:5u>u��U�� ��� �A05����Ch���Mj�$�� O���� o%(8�~���=�y�T�%�^�� �T�E�:��~��������g�����)i� x�@s��!�7���)f�
��M1V)4���v-��ue�X�~d��{L����T��L�7������ m��M����I�C��I��NJ�y�^���*K
�#��G��$�����v�[%lO�p�����'��~�b�,�;�����o�O�j7^&]�5*`
�3|.W\Eo�j�D^�YD >�-G�S-Ke�Ap_y��/��X)
DTQ�����y(^H|�A5n �k� ]"��� �H��r�
���z�@
��G;�g��Y�H(O�E��R���� ����u]��o��� ���� ~c� =�H��Rw_���G'/� �W� �����r6��>��4��v����.��v�����������S�}W>S���K�q������<��|"���j� ��[m?�0 ��CL �0f���7���2�QR�c� �J �`�0� 4cJ24[��=B_�uA_]^� ���-H.65�U����;"���ry9[ z��F�
�AE���-�
i(6��;{J��SU��=5���5k^���� ��k(x�`[���AY�U���� ����~���)2b���"/����m?��rX��&*{���H,#�JT�Sa|O������ ��^'�VI�>@3Hk��tjJl-�����E�����=��&
3��bQ ��HL� *=�8]���~/�3�z�;��a��}�� s�P�� h(�Y�n��(���(��=����fRv��Cv%X���,>�Z<��Od�������0�s�sc
��(��q������Z��r���P&n-X`.�=#��gW���D( ����o���n��S����1��/�o��n�j)E��5)���� ?c���Sf�[S��
_�;�:�ZcX����F�;hTr��Lu��*S$�:: 5,����j�W�
o��<1E��JxK����N� �%�����sS���Z������� V��;��� �r�� *B�����j��w1^^��P��� ����;s|%����l�p �)|�` o��Vi� M���)4=����xGi�1��U$���\0�,R���l�\t��Z���h�Z���<����I�2k��|2�V������0u�]���m���B2�td@K�`O���|��
� ��#^4�`T���������2h2�] ������ �^(s���)Q-���
3�]G
�b� 1����j�� 7�����LO�2oLyM��F�D D�"��3_�W�3_��>��&� ��k<}�����?Li��h4*O+� �M\%e�dwj�R!VE�/��R�������v�R��U���W[_E<�Xs+D���|��>�_�]a~����J�]�6kmX�6��t���m�{����]�~]|;(��c���o�_�� `�g~j�W��*�����W ��Z�*� {������r�����g���m�����/���
�z�pZM�NcX.���F��X�&!��s��"�hJ�"�4�i�#<a�:�8D5�$4��n���� <xT�Z�� �E���>�J*�h-��
�SV�6��C����m���&�<Z����*� �� ���Pv�� ��$�����cs�����.M����aB�!�J}�b�I#����� �$y���&�4N�#��l�$�3����q�$�m���>�<�ao�Q��;�\������Tf���.���c�=:1l� ����{�u��,�x��8[-�\}��!�-!`~��"�N��d�� (�M�0~M�b�.��\�\�� �
��cS�7��]����A:����t67�Z0�<�D % 4zSb(��MX�-������ ]/ ?�?@�E�����z����������F\��Qb�i�t��0��QhG2���o�7�AD*�I�U����B���=I)��E����fL} �eEn�\ ��0Tw���T��">)��]����J�7��Y�D��Fq%�K��dR��d����\�B�X����
�j'R� �*H
�L��Y��%�6 T����-Ie� H[aH� ���R� #�����(��� �K����sw"Pl�+�$�� mQd.���c�3��xY�����)g��Zy��� ���B}&��
�;&����C^.q�`��b�
<�?�f������s n�2�a%t m� 1�ut����b����~��;>�&��������D��0`��L��Ykq���%�CW�= ���������1������3�%'l�C������:A\c�� �3�1��f)W��ca����>i�0�=,��a������p��(4�l�t�p��S���F�^2M�Dm�����T��o������2��%J����#2��B�4�acO _�� {����-�7����q_�0O���^�� �%C1� ��y�^$��I 6j
��������*�-BD��.3��8�1nF���,l�K��|����_i6�m(Wm4�_�� ��������QH�kY�������*���r��D@=�����r�Z�������������{����_m5k��o�j_�� �wB Y�#{�<zU�=����.�� ��#�A�&
���0-?�/j= ����|�\� ��OC �����O�%�GqW��Ab|� $@�t���i�?���$q���;LFc+�=B���
N �����.���~���=� "���A���D�7+Nf9S �Lz'c�0����T����� z,��^��{*V��� �����|���e�_���n� �h����-A��) �E��G� ���~����|A�*S]�2�J2Jw�U�2���k�����:�{�7��) P|�%������(��%�������PB#��^��j9���:R ��n1R��`}�����lk��]_�/�/�/�/�/�/�/�/�/�/�/�/�/�/�/�/�/�/�/�/�/�/�/�/�/�/�/�/�/�/�/�/�/�/�/����� �� H 
Re: [HACKERS] WIP: Aggregation push-down
On Sat, Nov 4, 2017 at 12:33 AM, Antonin Houska <ah@cybertec.at> wrote:
I'm not about to add any other features now. Implementation of the missing
parts (see the TODO comments in the code) is the next step. But what I'd
appreciate most is a feedback on the design. Thanks.
I am getting a conflict after applying patch 5 but this did not get
any reviews so moved to next CF with waiting on author as status.
--
Michael
Re: [HACKERS] WIP: Aggregation push-down
Michael Paquier <michael.paquier@gmail.com> wrote:
On Sat, Nov 4, 2017 at 12:33 AM, Antonin Houska <ah@cybertec.at> wrote:
I'm not about to add any other features now. Implementation of the missing
parts (see the TODO comments in the code) is the next step. But what I'd
appreciate most is a feedback on the design. Thanks.I am getting a conflict after applying patch 5 but this did not get
any reviews so moved to next CF with waiting on author as status.
Attached is the next version.
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de, http://www.cybertec.at
Attachments:
agg_pushdown_v5.tgzapplication/x-gzipDownload
� I&=Z �<ks�F��J��9U���!
����U\Y����It�� $�$b` P�.��~�=`�$mg������9������n������;.��^dXg�j5����>�'��o���g���_�)��
K���Lch��>�����>��}�
�����'R~���
�1�$���h��5�c�����������3�8��z�&�W�t����>��?������_�������1�
�l���|A���w���={R�w����C������������@7f�%/[ �\o���=��3��i ������w�_�����7 �����_ 8k&�Z8�h^�G����C~���[�{�G���o��}�l��Q����&���1;=]y s��hq6w y������E�}Y��E�[�y��+���|�60� ��+/p�'6 ��ra�z#���c�:99��������{�� ''�U���E� ��2
7���a
~
.�m��!����c����� /&��������7]��E���{|^��]�8����� �u�>���e����{s�C8��L���"� �^5���Z������}�~r"��� 1�����(g$|���e��� ����N��L���-w
��o0��[�|�����DR}�~E^����-�G�q>�[�g7�&�g�W��z����"�"Sg['J<�o] F�
w�k�Y=�z��#����Ibq���~��_�[��]� z�U� i� ~K����s���`U�^R�#D8��Z����if�Q&\
�F����0��
B���E��p�y��~�� �$-�������;�� �����bN���f($��5�@6B �y��g��n�38�����Z�z���m��5�e�zh�� C=�����9������B��AS��mt. �w>�PR|�J�@�%\�O�x^+x�J�YKX ����d�����*C�m l$��Ae�@��F_��=�C� ������%C�||�'{��� �-}Q����cg��Y��� ���o��� T��f��`"a4���P��40���t��0����a��Xup�������������D;5��t0IGY������QT����r��v5���?^N%e/��|������y���
e���saO�%�R�F�4����K R��a��Wy��<%��tI�R��m�����-����Y��t�������m9����T|M���T� ��Rj��X�J
hK� k����D80<0a�����
���[�]� m}���Z&&����d��x����u�U�;�,������d�"��G��c�M8�|���._:;?�|�~��^><�=h�h@�����(G��`��G��,9����������� �s�<� �
�����f z�_v O0�nPP����M�?Vbp<�<�'u�)C y2
��s*,A2oB��/����B��U*v q_�d_�5S��j�:��V�P�;�$7�}�[�XWB�*����y�2�6���|�S�q�_!�o@�{�I��� �p�Yq>{v��<����4~ ��Lcd(�c�EWBPM�o�r�n��G�s�G�
%�S��/��%#Q�J��T��\�G<X�K�o�-�(�[�N�[H �/�}2�� O
k�0 �4s<�!�GC��uSB��SMU�$0��`��U������Z
g�k`����}{=�b��+�4�z� J
�s�a:�^�`��VO��/�7l��D�m��0��e���L{�3�EK~��/i��~���E��}�*�7�
wI[-�O����?��E��u�4��|VA���* e����l�?&���V��(�l�$�n�@3���[��\����jz9��zS�
/OWz�5�K���
}p����(w��0
�p`!8� ���z>���5�&6M-�ij� *�P�y� � g�'
�E���}l"}��{M�;���l% |��Dm�-?����_v<zN� �D��}#_-�8T�j�C_o�� g��E��Vl����F��p/)s��6�08�f� p �� $(ea���w�S&�?K������!1���/��� �� ��t����!�n��� $��0Y�u�'������5w������ �0�"\n��A��#8}��m�.�@�j��Sq|��E���L��-X�
ry��&�I�G��{�^����)��#������������M��}������m)l��&2v�o�JP"���y
������ �{��p��~2}u���(k�Ux�~�����&�f��-�-���e����]��f��#q�I �x�v+�AR^� c�W������i��w�6��Ta������g�!�����>Uh��P) � ?M Zd��Xu��X�U��zgV�Z���,�)�^��������jr�������������J������m�6���d�d���A�u�]���M�eG_�4-���i���x��� R�`�i�������'��5�lJW��{�k�p��l�}[���l����B]�V�jX��O�oh;�B��j�Y�8sd�3d����{��S)S��JX��6�U8b]��S�I���yY�)9�C�Vi��sAM4}�� ,P���B@;��"��FEj/s� �R5)��vj��!�,��JY�H���S���V�+�mU:���b��zK�t����j*��2]����D%Dy�k{s���r�D����
1 ���-�����r�fv}w1�~�{,E�
!�<W� � ���d�|sw{�U=}�J
OuAYT�m
�����A��~����8g�@����]� 2� ��J�����C_������ �(�?* ��z�J�'�Kf5y��������S��7�����5b�++���
R������f2�x�=�|Sc��q#�Y���Z%e�����D E������}���R��������\����m�m�<��������������}�;���|� t^����V�X3lYkd�H�w�J���Da�h��C{��a�5��mG0�.9��r����!��d}2���>�!h2k�U��]��u
P��U0d����xT�F���� 8t�� �M��� ��e�g;_����.� $sZy���J�� � �������s'<l <l_9
�������']���v "C�jL�X ��������Y)�����\/u�����J�� �����]�6��~�V�E��U _��b��E^����/p �J��/4��j��*lk&�y<�������]�jp�jV
|%�������`�3���o!�������z����z�6
��.*Y �F��!$�l�Y�2�F�� �t.GQ�v���k ���3����o5��b�9��2g<�u�v ��n�A�5� '1 ��g�M���G�9;�b��1P �i��\���~�Ow� ����)q������x���x�'4�X;Q�b���`����� �J"���}<� t������j��E�l]�l��h� ��e����Y�9����n��p^��e1�����r �A��v ~���0��%,��mW�l�1{w9}�>|�����\��+[MW�8(����`�L� yC��$"c(�M�-���P�>��x���oc�`lkC]�, ��0`��&�,�*����%���<Y���P��H����+���e���*��;;�A�1H���}6� �!��0*d@��I*M�VQ���=�s�w� �����������s�J�L ��[���8K�^\��� z x��x+/X�;�+o?� ����w���6 ��
c�5U�+�M��e�A��{�����l��h��
���r �"�Dy���!�i������x�I��
T��d������y ���������b��a�+Z������L� �����S� A�sr��(gJ��q�b����b����Z�a��Q���W ���
}�}�$��Ou���iS_e�M��a�������h�>�n����*L��An�E���9 h��>b��O���9����7,\�E�?��* �X��[�k�%k'�����u��]e���!4�<�W�� �GZ�Dg��n5����O����CW�_o}, ����,��S %FZD�b�� ��Z��Q�6Z�6<�����c#Pr���W�UD�>��� ������.�{��� ��`(
b�C�#~w���E9bL��L
s����@��G��oIy�
���D��� ��"�_H����gS�dI�����$i�$���t�:�y�U rJ�]B�E����w 1hv��)H �9 �������v@�'�dB�s���pI8���,�j ~vj Q��c>�t�o���7��|��s1 ��n�z�bl
Fj�����1) {�N� epBp2 �����{S(��H�T��KC$z�l}��D����bl�{�.e�����d�s%��{�k���(���KpY:H�|��s|�%WX:�4s��{�w��6q���
�s�6��@��B������=�Z;0"&*mH= i�1�7�q"����!������`�����O��YZN��C�gd��J&���V.������o=����Z��a[�-"��� �m0 ���a9�cI7�$��n3��T Y�����wrk��$�%�*}�����C v�/=BI����� $������+ k�?��v���k��/��)�6*���/��q��'����5 ����-%�,��H��7%��"�Dc����n��x:]�Jb6��a�O�r/��c�r�tV[�.��g����^�- RHw��L� 2:�����`w ={1�i�������C��V5 JDaFA�an)�N����G��A�@ kC�� 0`�m����������2��������*�@��K6�<,TRA?w� ��G1 p8��������v��+��@��{
bg����u�0�j
��t��Hs:�����l���P&�A��i���3��d^��� �Z�R�����B�=��K�W�:Bz��<9���0���jVt������No�K\�A1���b�� r �\������Lh���������oOq �@p�u*� .>"�A���Z*������/��[x����p}�"����)�� 3+�����vL��������WiC�8,�S9*A�� ��<����*^p� �"0�C���9����� 9�-���s���D�ko�� �3�������� �����o�v(U(H@D]���0~����G}u)���$L'K����Ul����� '�3|� � ���,[8O����g�R@��� ��6T�%z�\��7Fz�I�(����X��x��dE�<+����S Q��m�� P�������&�����<@��E�a�8`Z� >U�
Z!�_�0}v"g$�4%��C��. �>��JW��Xi Wy6|��qmZ�kz_�t���O1���
��< ��2 ���L��*��R��LC�_~�|$�;���S����Rpn��8(�{������wb�( O�R��>�U��,MG}�(���q������XX Z,���59)L���pt� ��bQ,��)�� �{��+�*a*f�;� 2��\5���%�e��N_
��Dv"(�M�_�@�*E3)F�;nr���<��G��`�V���(k�� i{���(�
I( �����#]*�?UL��s�����3�I�+��'I�e�=�D� C .t�4��W$���9e���!�����1cl��R_��:��z?�Oe��]����r���������2�Q��Go�Q$�� �-w�)
gg9K��E��xtysD[t�0D�3AaX A�0+�#?���S:���K�L��E�A.G�� ������ ���D�%s�<�&DT���b!���C�M>�R@��Dpo�A\U��'��t<�/<�� d"\���9��� )�D�J/xrb����HQj O"�3��ig�������2V@�=�EuV�3�0����Z�� �g
��P�!���x ����U�, N��@I/'���T���q�[��a@��8�8��B^-�D��B��p����c�#��_�����#� ���hg�l, �/��9����X 3�g�{t�R��A��% �L����KU=U]�jY��L8�PWW�������a�����N\��]��E�&�M
������4�}�����Z�+
��<z���W����y�����"� W��� J��� p����
�_������O��F���?��x;�X���5 �Ka%�)I�H����"` �
�6k�8��\������3�)���X.!MF.��HW �|`��� ��cE N0 hn�������WgU�(0 ��� �f t��� ������+�������2�� ��<6�B�'�����;�:�e���9�e$�jW��K��|q � �|���'� t�V�����_}=M�Oo��L� ��N�4(����A 3��O�qnX�]N���{0qG�B�C��
��?�)����Z���� ���iB��7�m�4�c�>�T��I����t�'�,g��+���D�U�+[�:��+�+���Z��j���5qI���yr-���WC�V��1���"w��8t��o���G��r6���!����q%##���0����d��NX����K��G�__� V�f�v ���0;�y�P� ��|���G\����w�H��*=]F K�Yk ]T� ��6��^�|����
M ����zd-� [ ���h���i� �. �p���2� � �b� ���>�:`��q ��������F�I.��� � ����������2�� ��O���������?9��[8���R��f��ow���n�~����?U� �M�����-�-�[�}t�qo0��g�V�����*=���Z��@�� �m����?2��|��+<����8� :�s��P6 ���X����&OV6*(!G v�e�?C��� bX ���(��c~�R�iJ��|c�� m���d��r(������'5����+�M�����5��?�NO�� rlS c� /
K|�K�-����o L�b�Y����3�C���_�U�� �@#��W��R�aV�.� y�T O���u��� ?����HQa�!�h��4F�`� M'���: '��q���i�8����l1���������Z,W)���
��+zCE�j(�d�c��}}�<h���r�a��""3!;��4bTO_�V2�P����� T��D�dk<����������c{�Lo?�����5�l1U�8w���=��|u=[��"����� ����8^�
:��Dd<���`�R� X�.�a�~���G����2���S�dZ/�:��!b�$J�j�A�m��F[��������Z]� aH�t�}!�q��5���/l�������q30��fk��??�y���7o���~�����?}����X�� c�6���X�\��x����%j���!�l0r ��7�;���#� �����������tr~���CM�,h�����B�@}����w�-6+>�_ G�y��&1���j#�C��:=#E:�K\J ��Gv��~ ����Pg�h&���,,��x|y4m�p$����^�O��+�Q����3m2�t�?�����V����o����h�`]E�U����k��&�8��34~�S�/@<v����/�4Gx�#�
��p���8G&���{��|��[*YG��x">�(ps��[� �f1���f:&�Q��7�;�b��Q�;�Y���$�O��k@O�� ����� ��������8�������}i��tc�,�=�z|l^:<�e#u �����j�f`�d�X��7�~�|��H�:X�+�u����� ����r��B�� ��/�)�K���?�M�U�>�&�E��3$s�[K��3������aQ.�E
�W������/1���Jz����;�~����`
��^�G�~�g�+�������� Z�@~��+%^��/u^1_���3��Q ���]L/�`,�*�_2V� �.�l�a�V�e� ���
�|��<�X����&�C{��g���.l�tWE^��t�?�.2g\�����d�@m����� w&F�N����g)aP�$&�JW�S���~�����:��F�c��:���J�&�2��d���<���<z�%�)�|�+��?H��d tiN�PK��Z����dZ��O E�'�b��6��[���4 :
[|���L}}6"Q��,����< ���3B��+W�`��iY�G/ �Q;���� o�'����5~��5��<�=�$ ��Xd�(4� @�0�u)z����]�t��y� U��7���D��gm������,������\\E�UH��G�:�����z���y�#��-�h��jNp��/6:-��M�[]>��<t)�k ���t����"ft5A�� | x��"��X��.�S(� r To��Fq�7�p��eZA��_�.����4�o� JP���F�
�-����� 1�I�v�H
�1?C� �Y9
Tn2��|� �"L�n�I2g ����w ��[��j/�+�'�����w\o�W��(�%�2KQ��#�c�)$D�
a�Z���&+��2�I�dZ���p���M .# D�����8�a{���M�������bQ�~u�{]��mH��hi&�+~�3�|��<�8Dg��:]�y��$�f)��L}c�l'�T|%dU$I5�:��-�5
�6������ �D��5 �U����2e�W��`Z�P���Yg1}a��@�+�����=
�!��0�j/k�1�R�M���
�''��|�� 1��������+KK��.`[P�T��.(�P3N�y�,���6��hGh�s%:���� �D�����'_� �w��:D?��o��8����) �?��PX#5���<��W�|G���B��^�_�]0i������J�0P�- 3����u.�$WQ�����lo� ]����h������o118��cB5�� �����B�����"\����!l,�/]
��O[y�b��X�>s������ ����`�^���J]{�
������� �4{6ZM ��>�Fv�@I�=t�J��1���a���$H9����6��$�x��
u��y���M�U��{��{��es���K�3CC����JtDvC�O:&�!�l�v�" ��$������,]c�� m��@�v8���*I���j�����<t��-:u'�Xv@-�VR�����g����d�T_������=E��kh�pB �\<)�N���� W�[[� <oj��$��f��������
�jpA���k�A~@ jA������� U���_�-���}D �c�B�����W���#���ZW�4������O�F��M�Z���
?�����X�h�o����o��M�c�3-��;,HKo���SrFL[����o5� "�A|K�
�>G�~]���(��A�_{�[3��6�U��4��3�����`�t��������������l�N�%Kx�
�Hf�7�7��R���b�M_Q��}��Y�(�� �����x O��#M)yL�7��I�s8g+%��G�s� �\������V+n�f
OC+����2�aXk���P_��c9W�t�!��-�m�9������?d��x O��/�%��j���G��?��tX�\
�Z�% ���&��P�^�+��3�u{"q@����{N-
ub��G%o!�id'e�j 9�U����� ��S�P��S��F�G�5��!��$D�jY�W��U0 �����M�Ig{����C� �������$>{�6hMR�o����
�y:���-k�Dl��M��Ma�&M����@�9��^�P7�'�LZ���`1T�^��=�j%�q
���w���(R-��
B�b@��QC�Tu�pQ � C~:���Z2d"R=��?c7 �fx�m���B 2x0�{�"%l��1��k| �|�!r�n�S���3�i��(7� �3����� ��U�� �������R�9 S\� �,��<�����8��:��+�C�_����wE��k� [Z7�����p/�� �T�����?�f����U�[I�ZE�O�5�'rj� 6r���������B��1��j=�T�h�~��d�+��,}AV�����pP�r�������Sz�l[�<��r�Wm�'��B�2 �&5Mp\0 $�2���3��(��)CK��f�� s��� ���f�j�
��Cp��&���_��MA���$�2oI��b�j� � ���~>G>�D��,�$TA�����:�A T����e6p��*�
�&��(��%2��J1"�Eu��d^/l��Q��I���{;��&c a��4Lp���.z�N</�xZ��ca�P��N`�\���������F>���A/�7�I����Q������]r�f��j�U����(��������-�w#�� ����0 ���CE�Y7]�E�)�5y@ ����~����R������bsq���^�L��z��R�����/[#%����7�PC���#�S!�@�\�T]�z�G��? H��q9/��CO-�� �����_���QO��
{� Z ���fEW��`������J 1���7@Sr�t�"� ���C|*�@I��\S~)5�z;�;��G������G�I-� �7�3�I�,#�GP
�g��$hH�HF���61 x���qz��E?���0�:_@�"J*�@ ��?!|8f��)����#�9?�D��/��z!��*��
H<���5( �t�
�1���J�JVN ��x����w���\�7##]$��ai,�M��m�6��1�"C�Muk* e�$��������ls���O��G�f.*>u����AhWN P� >YN��� S-$y�����3l@�������$N Yg��"�7�td/r�����a�:���t�"�38�fk�9p�U�����,�>g�����h� �� �� �U+X����� w^� %���I��?���/���
�03< p�%C �
@��U ��P� ]���7�h0��z���K#LBW:�R:
��}�������o�1Q9�%N�����x�^�#1����f���x�����?�����G��#�f��" T)��p 6d�M�����Jn57�;`7�����#/��WKDk��tt���^�\(�L ��
��\��x�������� �R�������~d`�� Y\T
4�|���� 6���LIk/�C<�"2� 2$a �����HVT���������wr���A�!��N����)c�Z�h�!@Y�B�[Ba���%�+ �C��R�9@�%��K +C3r��\�s����[�������Jg�,�K�R+����X}�����4p4�� e-�� / �����&��R`�%.����Q_���H={�� �'C��]^�gD� 1��zh�J�M�toE����q��Ey3L��,��������)��R���F�?F��� �0Y�����|@�����N�z��oT+�F�P���&Z���v "{r5�sJ�;�`��q c���X ���x��9<�������4*�{N���F���4�/��[��C.�c�o!&��M5�x�6vH��Z
��&!|�����9�k~����f�N'f������=F(O�Uz�(�� S��c��!G��j�X��@���-��6N#��Y�z�J�0�s?j=[�Q"k:^�/���E1���tg s��A�� �=��������K[�����?��Ar�{|�
�T���|����Q��
��6��c�4FC�
��������QC�y~
�v��9�����38� ��i��������&����I#>�[�C V�Dp�����B�-��< 5Y' �+���,�����D�v -������Y���PU.��h9+f�9�D�J�;����h���� ��/�B�B��:�2 d����s�W���� �C�g1�
�C��p�&����o2��Ap�8���~��\)V����� �m�������?d�Y�-�%�1V&� 9�8b�G�����4� �e�,tT=c�W��,eO�'!��(z}�� b��9[��N���uR��]wZ�HR��7uL�J�`b�*Ju�VZ/�����o4G& ��6�� 5w�f�F���g�#V4����L$����d�C�����T{�4B� D U(�<�\ ��X��@�Pw� )�y Rw�y�.bH�Rs���f����cW��\��V�c���0�T ��J^J����u����������CO�
d#�����:0[�p�,)���KaO�e]�5�>v��K�(���=��@ �a����V ��&z����'����<r�(g8�8�0Iq\ gcz�������g�M_��Ut�QTl6�&���fLq*���y{b���{�4�x
�N�@�3��0/��]��H|�`�
�K��B@�G�,0�5�m�X ��o�FBn������a�0
�X��$��K��R�������#R���^O���*�$f� QZ���K���4iJ ���T6�D����e $T F�N\�� ������r��>�#�g��mI55�a���v��v�����\���O�����ELpd�� ����se�=�Q�TA��,��: ��L�!��&�K ��Q�*��hA8f����:C ��JV��W���vI���Gm���r��
���e6j|(ZI{.5Q�� d�nK� �uP�� ���J�Q�(;�&x��E���cc�%��Y���6\[��������{���^.����1 k��| �Q��6�����g�h�7lL8�K,�#�s�",Q��qm�4�m����� ��f�h����%����]nh����#f�0`�3���\���q�
��� ���N����������������[�bk@Z�%�Fh�- ������
vUMwq������e���]�����y^5��cQ�(p�L�a�W.��3�� �_�LO�u4��]��:���td�8|��d\`��f0Q�/�'[P�Eh3�k(� 8 D�5]��al���q
�"������$�e�����#wd U �i#�5�K����^�`H�A��B�
����l���n�hl*� , VW��`�~�8�R��L���` ���Au?�. ��4�5��NX~&:zJ�5�t�S���F�k$~
Y/W ��)
|f������l�*��34OZ���xF� EBI���f�$���$�M�`#���}��8���������� �. I�n���� �&~+�H�d>�� �3�u�Q��0��8:��sk��&�;�V|�2c��8J�v(��@
��-3��'_�" l�8Y���@����d�Or*ln��@1-`j#Ck��`�=b���^0��k��~�]'�#��|VNr���i#`����(wa�DP �7� l4�& |j�Y��+Hx?W�.�v��3�\���J�~(���/��:Aq�<6� :3���F8�U�L��Y�<������&Nc��Ar)���8��I��,�|�z�g�m$i( _���b( �=o�(���)[�W:*�\rj7d<��5��Fu��Ed>������J
Q�r��������� Q��P���e
Z�V��l��%Z���X��+�H�#����Hy!;�Tj���[� ����c�x���E�a��&O/ls;���>�rog`�� -��������c�� �uOj�dus���� h��1��� ������_�����H����:��o��eq6���d���1!c�
T�>��}�MF����>hc���cg�9s
s��Y���K��A������1�%�%�DylA �w�{zw��30,� �
� 9G�Qc6/
�E�"������.h
<�?��
��5PR��������4D�w� �����.�7+�^�<�9�*=D^oV�Y=����e�2��b�O�A����H�tuluk>*������ }�F��H��@�B|�I� �w������\@��g�QS� �:�OL�4K� | u� ��4�F� �A�����0���;�G���qrd^u^�G������F~\$E3`X�L *N���b�S$uq���+.(H�L�!]��d��xq1GM �J \�� C�
@W�'t�X,��>�9H|�dEIjb l�c��@��p���}��n ���b�����}Q �M'���5�Yl ��i����{I;re_�m/~UE�F�#f}�^&�������2Lt�6��n�f�������.U��79�l1��� h���l�1�F������XB�r% 2\A��d���t���0��SN��4�������j�Z�����e�.uJO�IK��]g���P�I+]c!�tM'�!��)�������1[������lYG)���!D�~Z��a��k6��f���I:�9�b�\rx���-h���.$�?V����P' �N\���� U��&t���?���M�=WQ��?T��*���C��L�ve S���I�>����#�K��o��m} <�#�@
l��J 4���� ��04��� �� ?�� G|���$�s� B�j�Lc��C7t�9���A�G�K(�e9��lQ�^c�P�}��] ��%�P���8�v�/��Z�����x��X��k���W�+<d�h���< ���)�9���fY��l^�a��*0 g���
)S$u��c�G���sg�yP�\�H&�`<x�1�Z%�� 1P��W9ruk>R��B��s�� �^�3> �����7���J��m�y���,�a �L��]�O��X��� ;�#�����<L&{tI�� *����!����Y�Ypr������YHG��~�!�HgXk5l������k�
���?��@�sO [,�m�<�IM �)�DlM;]=�@���A'^�fE$��� �D��f��+��fyh���������K��.VK��� R�:��� �%Y�����O�=�XR�{�W�i�
r�� CZ A�0��Di���\��r��u���? {�$��`�\Yv�Z�B�"V=��S��9w���D3q��P�(ruB��A n�q����{�#�F�,U�j�<��3��ADe��[�^|�hy�����6��
�X��o�S~� F<�n/,H<�[D���W �\��}�X. �e�Xe"^�y L10D*#��4�ji���oo%,�����`��}�lb���I�4% -p�Z �r�+�qu'��7Yp"y>tbO*�*����Y��h�����k�+�����j�v�p�0@� �X*d.�;L0 z� xS�+K 2 ,��������oj���t\�� �������:D_=��������7�����B��)K�y-���x���� p�� s��L ��x�%��" �m {?u+���������X��o��@J}�9�*B�3�T���������Go��6C ���R��,�����oH5Dx��'��w����^�zI�RZ�0��:Y��A�� �S���3��jW
� �S �!��c� �4������@� ���@� ��H���q� �I�3���N�+��X+����\�e�$���J�����` ��k�L��0�3h5��Yd0Aa�j.�[�D�1� Yu)L��$��6��hW��� �tJ fn����L� @���tl�������N�$��5��+P��(hC6ro)��i�/�� ��,�$X���2"2���� ���,\�#M tk��b%W\,��/) ��yBj��`k�a�`k�b��;�2V�X��9�������29o})bB)�2E�sQ(BL��L���.��V��WZ��aP@i� <1u`?����WP���.tjaJq�fk(��T��m�VH!������T��t��QI�B .1��^�<�rX ��U�rc�� �w��l��ED�����,I�"��z�^8��VD-Ti�s���P��G�d��(�7���O/ ��&��ne�$R�*4~!� &�
tm���UCf���B��cG|s�\J�1��-�&����q"����\Dg?:7�}����3�d�Ug�3G^��KE����qv1����=� \�c�3���b���.7K �K `O�n`�x�T����LL���C�4�x����5����� ~`�pD��N�� ���|����-X��3�`����N�^*2���,K���qt����q����Q%W /�5�����B�L
�U�X7�����Wy���8Y� �; ����{g
�&.�H�b�����T�;�c`�E�� ��$�P]R�6}/m���ik��St�0�����U�j+R.,=��.dT>�<g0��Z_���i` ��?���A��V����8��3)�� 9xN��xxd9:- �^f���z���� �a��"D]td��3��]��)�JVDu �
��"-K��@���O mE�a������~0���IB�=t 6:��>��9T���u U������w�� }L,*0���y� j��qRt�!����j��Q� +��j�V��q�/1?a�A
����\�7�D.����c��4��f����)=@���@O�����)@��A{$-C�( ������[��U��{b������4���� � F��E�$�G���C3�T��X#P*�al�K������ �B��1��Q��9W��`z���61% S0��7��'y����"'�^�:J��� ����
�[��.���C�9��1'����bj��x������/����D����� �\b M�����B/� ��25-�*� rY���B��� t-�Np���������� �����u]�z$��k��P&t�S�Q=2S�� ���-5C�����8�V(A����.����:��!:�Jj^Z])V�/�kL���{��}������#��� c���*\�h�8��O��q���,�������5
���R|`]��.�@bG�,���A9�k�p��� z�����6n� �gv7qj9�P�x�Zd"Z��1z�U�/juJ
�L��xt' +]��d��^_G�# ��� @(�R������L' ��� ��C� ��#Eq "�z�� ��x*9��X�U�Hk
����� J~Ni���?+����\YB��$�e��l1�9�����z
�X�,?��Itr�����|+C �g�D/�/� $h%���Q�8���r ���X��� >���K��
��R����H�������Tw�%��,)&�b��1L� ���d��6�|a�� n}Se��F�3(��w|�k��7���v���Z;X������![�� � ���x� ��9�X����1�������x��gC���e'�f���2�}�U)��)��o�����E�=�^q.�T}5NJ]Hdq�8����sv���r�� ]�[�p|xli} X�F@,r�$z(�������q�Pv�&�?��J���qc���T�B��&������`��vH0i��k ����(�K�\������+bCEHb0R� �+6b������a�%K�QYC�
�H����O��m�j�_�@�'�V���2E��������O/AIl3r��q`����P(���a
w�CG�^���XE�������-�4�`����J� ��L�[B��QY`������z=>�������d����d����E��X��4>K��}����� ��H��v�?��]�a�5������{.�]���S��I3q��l� �p�8 �<�A
���1\�T;�!�y7 RQ��/���@�n^\��5�E!�C���,��"@��3�P"�8�!i�����[ d*�7
�I Q40��}�r��D��n���=���� �z��&���H%�V����F'���C�
G�"
��AA���!�BV:_�������a TYVI��� ����YV���������"H��A������G����)��I�� �?�pm /4^M�d������J�w�r�0�x8o��=a���D�G�^�u���~�3�&����h:���gp�}�4��Fo�@6wU=9&����L{����l9M��&���lu�������8�@$-���9��i?!�p�!�����N ��<�^$��$nl����|���('�v��j)q��6&��'�z��DY+_�����
��(ex�
�^� ���#Pd��� 3� �s%�$#T�-��YT��`�����bE�������sn ���?�}�v�P�Y
j����] �?�s���������I>&�����mY�
�
����)�6)#�vr�l��!%��lN�IUtYF��Y�:�F���u�1�
�+p�F��nt4���4�Ry����S�(�9����s:�V��Q�`��[����jK/gK�n�_ ����`YD��������o�[r��2�w���x��\�O
]�����B�:����M�2�����v������w���������m���6W� , 6�����j�����?��';����"�1�����dJ�E�1�**��}a`���H39�6g8�Z��ZhV[1Z^�p��$9;_����3���s�����X�k�7yN�]d\�z[�*��$18)�" ���p8��� �y�9hL��x�������s��+;�M����@������[�Y�4k��=�����24J��� I�3�#���=���, �t��@�
��R���{���o��|p{ ������k���r�Ig��A�5���O�%�-�F�]����{X1��^�����am��v:���ohD �5����ts�=bC��h;c�Ec� -�t#����]���0?R4*�`����&<���j�A�c�� ��x�� �>�������Ir��M��09�� /���5�� z�j�Tu� �r�K� ��C�?h�d���<��}�:���$�$���4��m�t����W����h ���U��m4��?��o*�����v�^���C��M>���
D������>�?�F4���f�� v{���� t�~��������^�a������3��O�?�����������6��M�����r�����!��3��|�����%�}����q�u��O� ��=:�\������i������ ��6�& ��z#A���/������w�U�^��RZ��i����n����,����5������E�k[4�-nC������%m)�+��V���T��$�Y�Cd��4��������� E�6��rqU�IZ� ���z��l��g�q a�A��t� %�"�8�]������z�������p�;z^��K�e-}����=f������O�J����^cw��f�-�fj���*� j��A�+F��",� �V�Y�e���H��hUw������V> (l��Q���zM�����I��|����������l �y�A�P<C{����e��� ���y
)��m��U"*�}SK��rf�9����+7����m�����S�N���3���'G�j��� �d� ��I���q`����=��?���xy�_B��i���N�� 4B�]��x�i��� ��`v~ �������������3�gG���������� �0_��� T-H�:<�rA����%t:����}l�g�Da����z�C���rjS��%-������l� ��`8�!���1dI
v�U����������J���v�.�d�������[IS�A��-En���D������R�a^=��
� � �h�V�$!N���'���Z�@�*��a�Fo�p�� ��B~`���O��<������U�:�poO��rC�"]���k����p�~�OW������F��g�4�w�+�S��q#��������~[((:u�aj���bEi":��HP�����sY� N���\-��*�A5� G���@�/c&F�
2�G�bYl�D��
�����
�B��y>���G !`+w.���?%��� _��� �k4�H�s�b�D������@�\ ��F�]�Nm����:pZog����z1�AG-p��n�QS����V�m��tm �j�3���c�.jhur���&&K��{n����W��6�2�`�P("��9����N�����w��z�x����L�g`��3^�����e?��5Z5���d��M� :$��:�����&E�����s@� �Sq iip�{��U�9�w9������s����������!DWf^���u�S�\: ��{:]5�n^��|'��vt��5�
i�
� =�����H�B�������b��,��j���kZ9 ���E"�b���:y�F/���rl�K
� ={�A~b�tY����/������l�8�t��
O�+?
�� *�`� �X4G���C�)��m��>du������P|�@a�lf�kWU�c��
{J����1-FI" �� �����od�� J�I H��R�@6# �l�L@��^�@DO�������o����F�{�*��*� �[������M�D���{�Dt �#��[�~��j�ps�� ��&pD������}s���Q�yDA]�;������iB�K���/`";9Y`cu����mu:�V���^���V$!h=�_
�����Msm<0>dO�}�A����� <��}���N�`'�����KW���I��`� �K �������#��.�q
(c����(F�6o����n���*�1l����qHn>��k>����c��Z�vW�������P����PCU�����XT o����J�P��TI�Ux�x�5����X7��7��{��|�`�W�:h��� T,����|^�t��#JX�M (��%W?�E(5ouEB�F�5�,����:�����w3����N[��N�{�����[_��%�"H�#��N�����6P.&^� �5J�m!G���1���D���sT]n����kY�q��`�E���o,,���AhWXT�{��I���J��� ����*{RW���F9�Bw���S�n���F�x ��_Co���!M�3PR@g�����5�j6Y�<G�w��g�2�c�U�v�Sk������-�}���\�$Z�� �S ��Z��7�E���`���x��i��1U�� �p4�]�veAW��$���z��1��
�����AW��G�� �
E:8����`�nE��x]\��"5T-�}l�!�U��%�!�?\�{��Iq��c(}
J<��\P�y�R]�!UB ��rt$�9@@\�f���\�g�J�$3����?��s���������3)�����k"W���H����g� �QD���bs&������<�������L#��^������zj��b���/
r�u���r�_��k)�I���}p ���O!}���k#���3�Q��a~9�|��tt �>uI�
�:�?�� ��v7�\5�����I:�������Dsa �kjt��H�&/�.4��-) .z�iO� �� ���������j�C��=��PX��|(�b*�Q4��J&�F�W�6:�OG&����t�8l ��&;�=���P��������(�R[l�e"*���5<!f]i~��2
-��k ���s�����r
j�#|t� #u|���Kt{����z�;���t���� ���h�V+Y�/EIb��*�S�[�8��TK�h�7TkhsP>�5� ���0J���yj �k�� g�f�h���Op�g�`�#���4���������
nL}�:�G���9�2>g}8g}k �ad����� ��z;& �$R��k�r.��sqrb�.���l]��JQ#E�#�75#��`�zoq���fA��*,P(�`�!�*Q��8 �A[-��
y uI���� ���GO�����-���5P?u� $ ����;#T&E�^���e��XC1���S] o��+������6��E�aZF UN���<L�PD<��&����W�F�@ ���w�k�A��OF1%���o�� `�O�s �V��� ��Q�����}�]���u]#��zN�����K�����>��>I ���� �=�_f�g)�8y���b�~��B��
��Ub�/gR����]� xG��m��g{��=�De[�����du���N_����_���#x���������
�� ��&��g$h�P'���zrB 8�z���6j��}�� ���n��#n�a`g�},��;��+(����uNPzt-v�*��e�L�����`�i����������xX�!��W �\�L,{�9;Hr�8�.�T/<j��� %O��I1f_]���������
��`�R[�'jO�S
V�/����J���9����T��l0� 5�iB�w�S]/ �.�F�.hu1��As���L���q,����j}ZF���> ����� �� XNkU�
�l��~�!����E��a� a|��@{�������OPl�W��6�mW�Oh�{g�y�> �M��l��o�'���V�'GQ��T��j��vG�X�8�.��8���D=���
����^�o�t �1�����
�����M���X�S�oF0 H�5�97"7�[/�a>_�"� ��Z��(��S��>�.�1�'h)�?��]���F��-q {��=�������w��1�i������������)&��<���� k��@�B�g�p�0��-���5@�7~R����8SC����tZ�����y����(��u "���������� ���r�b�y�;L% ��1�p���J���������������J.��v����u3�e��U���Ze@FO���(c<~{�?�����m��_�5�5�mN- e�TZ�;-�N;)� �z3�d����������TU���Z��h����T��i��z+ ����v1Z��U�m}�]��G�$[����X��i��=,�N�fi�9Z��1�r58d���p��e<N��7"���N��� �*�A�&n���i����wY� 9�o)�#Wdf�;MR�?�$5�~����
�J�5s �
�"U�
�X����4nq��9�g�=S u���)�)�Zc�������l�Q���F���T�@w���}Ch5S�I�'�"�(����U��C����>L�f��bs8OlmW�0B_��$)D�<������ �HAL�"��@�E4�(;�`�.��� � q ���`=1*C$�8V����Z����*���B��>����!�^s�f��\/��&����[�^{���1{O���_L��� �� ��0����T��V����M/��e� 6k��W@ 76w4N.�f���Qo�����k�^,�(�/6��Q���������lM��A���J���������z�;�M���^��o j�f �7^J���_p(=`�fC
���>�`�_q�qv������Z�&_��=��(�p���w[��Vs�����>�Zhj7��^���� �Q�� �V=��H*�<�%��%��.Xx��{=5�\����M������4���p��t�I�#G[Y�1a�{_ X��#��%tZ�R5��X��'�K�Sa�Q��z�{�n���1����LJ� ����c�f?H{������]���4����z�5���z��Q�~+�q��O:�x���������3���J�!j ���C1��G���q,Vj(p���z:�8&�"�t��eD�0?x� '������6�_�������4+���u��z�z}�� ��
������o����!�����(�*P����r�z �Y k� '����T�� ���s�v<.�B9jq �e����h�kXF=�I� �����8�u;g���w\J?~�
r
6�B�M��zJ5�u��5�0t"�8���"L�z^�����[�I�$B� ������7hn"��drv\��y����b���f�7&lw��g������n�6���4%�q��h��g��#o���h����&&������� �*:B QK_^��k�z�F$�8�����f�,9��2,��R�5 ����f��t�e�E�&":T�LjWF����M�q�>n�� �H���D���'NFm%* ���*�Y��
iv��)|�������t-w��#�h���S#�WQ[%�ot9A�7"s3(���2 ���PPZ�#-�.gF�W�#��J5�����M�������[d8��J����%�_&�t ��I��a5����,����z��l��*Y-�~RJ��kNh��� �Y� ��dL&�U<C�:���i��K�1� �1��������u����:���l�tG���&N�S%�d�� F�@);C��������,���\>'l������*|Y�T�hC��v?_���"������~f���_.LF��|��l�x�D���/?$Y=�zD����P����W3����"�ow��������|�-� ;{���|��P]#��}�,$PNMS�&/7���3�L=�7�1 �����J��5� L<W ��M�3 �1� o1�$�AG��0����F����/� h�n_srb� �Jh��8�~c ������������ �V�T�W����#P�[zZxI�V
�b'�Uw:D/��������):y��gXrS�-a����d ������Ds���N�0�@���O����i6{�����a����� �#U�H�L�3�o�Ro�����k�h�e�w;��i��5��~���mb���8
���
���ay�k��RYG����� ��~�i�!�;�=����e
:�����@��K]=� p�q���Yx��'��V���e<� ����:m�e>H�S�2� ��d���!��0�^M��|�m;8WsH!��l��^��l���>�������� ����E ���g5�a��s�}/5 5��q3t�������t����q7 ]��X ����* �� E����
��}� �������I�����:��4��AR*�75J���,Yg�}�
i#�0t �l�.V�~�'W��b:U�� �U`�mp� ���c �������i�e�v�w�![J=����O/�������L�l�K��h�� `$!��GI���K�S�k��^i o_�y�O$�.&�_�K� ny�{��� q ��(�/���5�,�
�iE�p���a��1����C5��\��X}���� ����i & 5: ��\J���]O���l� �����l��8�s�������~���4�_p/�_;C����t4����fo�.}��0�*�}�����w ; U�s�hn0p����Bh
n�E��y}����Z�9�~G'r���O��<�e)�M^�����X��� }�~ <��d���b�j�����q:%O-`���������9 D
�3�s��0��=�{"E�
TTK&�[�m��dC��}0B� �N�G���o ��^/$����x?i�&�T���UR���6 ���v��� [e�h�e�C��A��\�>Vj�~��^%�l�,�?��.J6���9�d����l��6� !� �F���}��p���,�P�JZ���ZP�Y�?��� ��0��ZK����x:�b3�� E��� {���w�^Gx��M���E��T�� �������y��8���5"<g������u����[ �.��� zL��$�����~y���@�P�
�Q �����ve���������q�,�����~�e����������b�FV���d�u��Z�[F�7h@��K�m�h5��<Mo��'7�
[��+��m�
�D���T~���@���f�q���+ ��&y ���OQ�u��x�m� ���_L'2�K��H���<��� 0���+R�`UAlm������1��`����$�V ��9C��y��,�O�U�����dh]��,k`� 5���D����F ��� g ����Q|1�4������o���o�'���`�t ��Ri�^6�E��A%� �@�r �@�;�������2��"\���[,����W51��\,����MP{�.�V� ,��:����# �O4: W��� � �6�-����D�p�@������|�d1S��:����NO���pi�������� y� 4q���<����~BeR���Qh�� G�o�]�{S��_�i�gA���%C������F���Jf�G�hO��| ����R
2{����a��=��q�{�Ew��u���E��!�*WT ��^�I2U,f ��\(��Y�x���b��� �c�H&�prH
�oNpWk����4�k4k��HY�4��Y�L��:��v����zm{�5�f��B�&������q���7���8#�b��-���SN!/��H�U3��� �{)u���Ja�� FY���X����c�5(���+W�q���5�w�a��;���N~i���n���� ��D� fR��6�RAb�����"NO����a�%�k��Lj3��ql��/���l����
k���C��|w��ooM��g�J����m�QH:ugi�RG% ��d��k
]iN� ��Le��`�P] .�T{����������n$�����7O��P����S����Lu�f�yK4�-�v
�B����_n<"U�<�]�*+����$OlUVJ�����+������^�� ��6�,z�Xg�jW�c��|��,5�C/_�B���/��"�`�'(C��~!�Ao���i�P�o�&G���������WoF/^�{�������=�FV���!3[ZS��jq��(���Hpu��/�8��E5�:��;b��f-��b�h�L~M�@�������j�#�������v��[Z���Vw�$ �����H����w�<���q,P�o�,Q_��x�1|1m?Nx1
Z�������}��m�D��
��r�S!R�`���W�Z��E��� ��[�+�[y���� ��U�
�����|�8V� *0�A�Y�`����V��&���&� �u�u��I�M*� S^l0@C��!���=��Uk���D��^s�I�#b���
�q�* ����� ��*��P�wkx�^v�o�� �0�a�����3%��g �A F����Pu�<m����~?���e�
�6���x�s���Z�I��
L{�a�6lt�6�}9�<5w1���[��W�r��`����#��B..� ,\y�c�s�PG�z�YE %��, )�W�s�w�Q� H�b���������Mx�D 6�5xWS-P������/F �H����6��{�SL�+��
����b��P�%��_�\���$��x ��
���������}bM���C<���3�B�a@h~��P�z�S����t����_� ��c �~� zTv�X��V�"�p��z���Q<�� �hMK��� 5*�g3f� r+
�Y��Qj��8�G�i;���+kv}�{��`c�o+w����USB���k�d�S<l
���S ��� �������]J�-���+�D�2N�R������C�
�j�]���T�n��R%��p�!.Cv�MJ!z��^c2���������n���Cj�#�igT� j j��V���.#����TKGH����7���Nl|]�= �?��10�\�c+ ��z����#�)�����g�S������zS����{���}���3� ��}7zx��0a��Z�j�CsE�z+?A�����.>� F#�A����M��w�<��>��S
c����7@�b�o(�,.���y��M-������~EB
wA�Q2)�O�% QE�`�L�5�q �<����I��c|�%SiY����4��I�^ ����%�Gi��|�6���-�� ����a�#�w-{E-@b��7���*�$���N��*�h���H���3� ��F� p�U
]J��kh��&J�����`N�{��s�A�W�?%_��;�?G�`�,��V�bY-��HA����� E�H���V==�3���o��� *L���l��'%�
��H��>-��z�A��UWS�5h�&g�W�Vv+Q�� �V)i�8�������i�!� ���Q�6 r#��?[�3�rT���� �����k�YJb���L��.�y��y>�/2�5P?+!n>��Y$��<B`{D�������� I@�9 ��������3?S������ �� ~���A��Mv9Y\�3�}�:���$V�AV� ��G���E��L`�o4x��� ��v�����V����v�F���C��G���g��]|�3��nD3�,�'�n�=��:�A}�l5��^�sO=����a�������t\�����������m�w��8��~���n~�Q�����V����N��>����v�'�<�����S]�Z+�z��o���R���]��l?f�i �'�j2�Ei"J�������i����� ���Vv����fv�����R��el�� ���E��G�
�� ^���?�VM-�P���� Gv �3�Hh�5������a =�� ��8�����myxj����T *���9o�������P�yG�� j�I�bZ~v��md~mdgP3h��u�[k�I��*��5-C�y�R����t��w� R������
�������~9��d#�i1�OM9��f��j�:�� �.\��jH�_�e!����2T[��E(;b���� /�mz�����
��n�Ukwm���{'C1���jMt � �g��pv
�\�#MD��!����-�[-�������=�+v�N�����:�@��:>f�5xG ��)Q��������F�������"Z� P:���#
����f6�(��03�GO��jf����/H�����9���Lg�aH������;^l ��� � �/�87��:�Z� ��R��Y=z���czR��� [>r� ��j7����z�wh�<8�)���k�
��������&���(ht��",h���� E�{�NE�#���I����>����/1j�2i�t�~�o G'8X��8�)���O��'HQ>C[N7��wg�����*9���X�T�i�O7IV4��'��O�����n �a@��@�_]��a����O|M F�w���7zHH�H.�4����(��� "�V���< +��%�(�X.�����BG#Q�)��w���` �6��5vdN��x3]sR4������%V�� �����"��� ��K5L8��$�0���T5�������O��)��W����0�1�,;��~(���7ZF]7(>�#z��&�B��)H�'�
�A����9C ��0Jt�����m*&�-��v��o "u�#� (�5��B��!�')U1�b�@��Cd�
�9�A�&>$����E��,� � ��;�y�V�� F�-�6p�y��@/)I(���UM y�(�����S~�`Q�����
(�����oSl�mRq�Y+q�%m� �&Tz Q� ���L/�M�$���������7� �]h��.�9`�%�^����� m����C�C
�F��3@6J���Y�oS�C��& ��R�y(�m�A�� �6�
!���Yo�M��sAn�ol)n`|F<�N���Y8��<����? �������?>}��{S�����/���{�����o��}���� ���5�\c�&���L�����^����q Q�!�v�"�_�%�+�+��{�[c�J�d���j��X>@������`*� �����']k�s�<V�z`�����-&�qb��$�� ��@l9j �p�K���P2�d���;��1�?��1�_u6j�9/�!�����6gA����r��b����[��% ����VS�Y���+u+�qU��\s���Z��+�������xU*M��������wC@.�#+0�J�buMt�� bI]�G�P���c)^�N��:$������/���b����t�����XO�J��W) �t.^����Sgnv�a�x��CEU������n�=�%rV��(T����q j
���H4�o��T��Y�c�H��ta���<a�������Hd;8
��~��f��7�x ��WQhz�r^�PQ�n�$XlYZ��� qc���QA��A��*j�
��q�M)9�� �4�5a�������Q&������,l3��C�h����(T��P�w�� l1�o )�~�-r�\���3��$�v�r�gWh��P`��Kq�-�� ����X�:+�d�� ,��s}�jq����=��I�13E��F��K~�a���}b;�(`Uo�����H�y��p�����P����j��a�l �N��r��|��#eoc���*����L5��P �����i0�kj�d���vs��6����1h/X �o$\���z������AC����^��
F4��4�� � }I�7|�Q�kZ2�$�������}7.l����|�����������F�=� ��J��LJ�����'$:������Xg
&��V����z��&�Op QI���%������|=�D�LG�
���`k�E����B���e&g�wu##�k 7�P�Dn��<�s�+�O
o�������p�&l�h��
��H�a�\`�V4���#���;h���B���o�q�� �3�YA�8���9ET|�����n Z����4��.J�w���x��~nf����������Vn������ 7�s�$^i��
� ��'������6��l�5l��D`��}L�zyL?��sfk�K%�u�hm�(7^o1Y�^���k����v7W���t"����#�K���N�NfjW ���5QkR���Z��7O�����obc�v��H�.���7C��]�/�
Wy����3�<<�^�����{����r/������\��b��4�^ ����~� ���?@�����qs]]��d3��*+���r
�������i�x�� ����E������w�IA+��Of`�.BO���P��i�U��6�_0t�E����:�na����c*��"vey� .��7
-�N�7s���V����d1 G� ����,���Y.Q,��,I,����A�^ �'���$���'��y�^Jjvj��P�
�-���o�/}5�
/. -/F�����_�����������8�5o���#�hpj y��9� ������e��=J�q<��s�t�=���Z ��7};<tt�Q����z�\���5����H�:���a� �fPJl��x�|�}}�W�hu�X[���[�Drr[I$'{'���Z��Drr�I$'��DrR9�D�)oi�n+{�������h �G�E�&�#'7� ��V� 1C�A�Hnh[�GN*g��z��=���n'{��v�GNn3{�d��k>��=rb ������B �(�%������n��p���z=iu������ �������.*��Tvi��[�F����k�~��� �"Tt�>��iN3�.E��j�V�kJN������������N�[pj�L"����-��~,O���"YU \��j3^�p?Q�jv��j���$C�U:����tO����H!���V���#��n �3�O��� ��n����� �O7���������zf����'��s ���S���.�f����^��;��]�)4�8����.��?�e"a�{v6lu�������'�N���='�� ��G/�� �I�G��O�%�e8��`X�E� �0h7�`�E{" B�rF��}Ov�?��?� ���%H"��WX��4F�WZ
D`1���8�47���Y]�,���O~�6Z���Q���C/��GG/����5�W�X�8�/�� �����c ������� �l� ��D.J��D���X)�0 }���\��.D ��������-��&u���w�� ��\�j��� _�U�� ���U(4��l'j~�P���9{�q�5p�������{�1b�!����&�Fo.�.���/�5���p ��� >��|.|G5ZZ�n���C
���J���C�?��� Y������q�����!�����<���) }H���ZN z���zy�I�J��Q �<^]G��������jq��4X�����X���d ���S*� ��p���|n�%����g�����k3r!+7
�8�$�%���^oM���3.����Fn�| �h7s� &J��f��x�>Y �\����Zs0h��'[��_�WGt�� � �c�R3����� j��������`��Jls�I>�s�����o�^%�^#b�:
���(:B] ���|��u�N�E���3�Q R��������Vo0����d<� �V�qZ`��O{g3�nf��v���K�A�%
��?�:#�
�}t�*���EX?^;@D"}�=@�9��86LFe�C����]�}��-��wXX�s����@.��O����Go{'�|Z!M}Kc]� w�� p�� ��)~� ��mW
�>���f�^o��g�m�^���^[�P�*��� �"�z Pg0�u� Y����MH$� %�w�����LUA�=��"a������{��'O�7�|������o���R�Y�H����!><b��EJb�$Uw�"z���7%6�vIp�!���e}�O3��WY7�lkc( Ems���^� ��h)�wO)x=����-��q�4��+�����������s��=\)�� W`�Z p��b x��8���� ^~M{�� � H� � �r����EgJY�O�d�@ �n8�$#q����q��#�x��VY
Z�a��o6��}DB��3P�a��xJ��2�, ^��FQ0P6 �Y� �%������h ;��>��tuH�n��Li(2��c�#�g�� �
��5��$��FM +'s|�[��0-�rW���5�.����v��F������kP���0U��9�����^}`����wIB����%Hi�t�QR�3�fEix��t�^� K�d�drP#S]�
~�,2]��1xdu�I?Q��<�2,��Vzz��������4�=���[�Py��c���f �1c`��x�u �ag���k �JQ*�� ���4�gtg�� x �J����� f��"�>�gK�^����)��B3��I�v�^�i�����mR
�05�tmK�s�M��s�{����e(g����vs�C�j�'�n��C_��7�aGt$�-� �5[��Fa�V�g���������oIp�Y�w��U2��"����J��������~����#V�MF��#�����d������f^u&�,-u��:�&4�,����M�R���� )��\��TW�������TYP/{� �j��^����>���]k3y\�4�����of���`� ��~�zTQJ����
`���
��J��rK���� ������Wp��\|���\�o��7�~��zP�*�zf��� `�Ih���iR�@�Rt�{ �O����k��a��=�J��J���i&��'�W�kl}��L�e|S ����� �����+��W� �%"*������O�����!�1Y�@���)5n�5��`�o�����b���Y��U���]*! u-��P�����!�
K���^��e�3��W�0\�N�:q����ek��/ �T��D�� �1� j��B|����A�������f2��t�T��8Y���UbW��9�G�����A,���b� ��}!��aD� ���(t|�(�U�#�a�?Mh\3�8���� �vI/�y��� �A�c�n���c�/������� ���Y�>��b!�������McE�ju��!F@7H�'��>U�V��|*4���������L�g� ���8V�S����U��_ 5�JmAU�]b�Am���9� j�b��:�G%?����)�`���f�Ig(�����`4�"X�y�^Bz�����&/��7�e�Y��1wy���[�D�o: ��/ �M ���k� R=��
��E {�@���U�����i����N�$:2?znb�_���_C ����� ����tt"��: ]\+dE�Lv ��4W� !�� ��[���_$��`�5�����/�d@�W��B�$�'��� cO
.��!���Q ���*���O'{T0X�Tv,IwU�w�m��:N����Q`8zj������"�o�`L5����T���g(U����qZ���Tx����'�(��n/�
����am� ���G���t����pw�?��?f�{��ea��� ^�z
��,�>���
�F�&� @-���b��-�C,a�`�7�� �+ ���=�BqC�Fi������8��X��o�����G
��^����l��f�#+���P��%h5HpG�(^�W�T�D �qLWx&�p;�� �\����v�JJ��*�kE��cKa�c9) ����� >�$ H@���6��������}.�s���^���2�L�QH#����0��q'U f�z�X��2��$�� y�<X�LJ�L_���P�!�&�q�l+� r7��1Yd�D*P.��lR��^���*�f��Eg��}��q��7|�!J�� �,:�*���`����]�@��h)D'��>C�s�!�(9O/���������x��r, ����q���)�����hX&B�M�@e���F1���T�� ��� .����c i#����2E0�h����� �
�� ,����j5��/ s��S�]tN�s0>p�����9�r����Z�9 Q��/W���c�@�������"\(j�aqZ�Z��
�R�� 3� u ������\�_�F��4 k^���t��(�!z����&6�g�|#�-���J8.��)���L-����` �l�
� ��,�������D�srW�����z!���uAWE��S�B�)��n�QS��W��p���BP�^��b^��y4�;
��N��j��A�$t0�*�}����^_���&� ���i��)�'|0�P���m�^�f�-����fg | �^
�E��o���`��>"bh��_s:&�f T�3M�� 0DYg����7����*y]� 5����D����3�+�MF3��*Y��2�
�1)���T V�����2�����������rO���< )� ������ ���x����u���H YS �x��m��X&J��`������i ��]|�l�*v������fx����X�&��w\q�q:�u�e{��=�0T��c�- P� �K2�p����d�B���A���FhN��������J��� 7�9��]x����M����$���������aR-0���PD��G�k����mbBG �G;=�Bp��S������J�of�p`U_~�I��D_p59G�:�L��h����3J!0��� ����� ��M$�����/
���o�J ����
���C���14�����n �� e������5�����~NCx��B��
ieq��m�� }�m��7g�!$�Z�������K����yDA�u�� ���xfA
���V���4�N��g?��� �5K�����N���g�%+��K���������!�>uMa�/-���q�;�����������$C�����y�?��a�� ���9��~��0Wu
O�����)P�y�1K~��-����
��������Q[��'�k���2���Qu��V o# �f������{c���b�������n�������1��^_��A[&=�{��jD���n���p����s� �s��'�j1_sg��(wD84���#_jD�fV���jE�4�C��7�d����|Ep��X�v�3�:8<$MZ�/����cd9�����uM-����[�k�"MV���.�FRW��/i���[=hL��� �C�C[�5��Nln�.�~�#�L�P�4f��-�9������
��n���
�w����# ��{ �p��| ���?��?-&����T^,��a�����]A�������k���6��m_�])��S�AjWw5�"J���J�.���|�&���%�j
B#��u����I�r����o�������;f��D6�����������C��l��+�W5a�c�]�$`��S���l��%�5���1������p�FR����I�����}��Zn����F�6��i0��t�-k7������ 7��\�HrI�,�A� �<I��5�Z�mV ��3��l���e} ��n��^�'Io|^��~��;�7������ .�Gj�1q�E.*|S�C�h��B��h�s�E����e�Xp�xg���;��;W�f�J�t������� P?��/6��?���?���CQ8���w�������.E�J�6����a[QI�l<NB ]��n�����%�[�o4K������~��l���V�!�Y-U��nm �"�`d�����@�c�1���"��5G���0 '������D�S��>gq����ve:n���Y����;��Y5
�~+�(��i���D��`0d� �kCA�F��eiJn��n����������O���k���������]A�U��k��e��� �Fq�z��SF]�� �%C�(�������nVc��������f�p/�~s��(I�s�< Td��m�sCM�c���D�Q���� ���m
k������Ft'*~����g�����{��\�����Q�e��R�����%�x�u\���0� ���J\�;p���[b�V33�piS��v
x��Ef��E�A&_F�VGq����r'� �$ ���F��������a� X'������G��g4�Pa��6Q����
���YkC�r��!����5�(t� �M� ��C��$�T�����)�^�:CE�1}p��q�U�EQg�qz-*�/�#���d��gN�^��j�%hWW�����r�� ����T�q��_���������~K'��,�S'��+3�����-6k���� Q��] �����3����
��r ?%��Jp���8� � �;tLqF[v�Ta!S�7�>�v.�BM3q��]%��mr����� �0�=&��U|]� H���\JHJ���#@H@����� �9� +�'\`��z�T�I�1�K(}���C(���ihQ�@�_c`�� �����b��J����� [����4��(��\9U�c��:�E�" �t�� �E%���/��a;n��Uk��� �apE)���z� �\���N����m��=�����>�������V� �M������7�ksx.!KA�)��`��5V�L�T��b�����TO|����O ����M~]p�8(�����0�Q�F��t<�F%��7���9��m|��:>�Z �1���K�5��������� ���p<zy�e����&���K�Q��w���h�w5�z0���r����T����� Jl
��H�i:��Ll�Z�-����r��3��m�<��\�r1^W���T�Pd ����qO���������]���� J�s� _�fS�&���g�4�8�����O\=�j x�8"Sk����WM��'O"W���?"�e�hiXw0+Av@�jA�I�Vo��"����\��nFkT�?P�n��!A��
� � �#���l��F�d~����^���@~�;/�>������ {�I��gL9���� AH�r<���"�����c��`�g�� @1 1.35�h�2]�� �.��{P _C�D+��E�b��A( �R��)����-���X � ��i��t}��*�"��������$�.�|��� /~�}R��$�+q���cu����kj�%Iu�K�} (%���E���N�_��?>_:����Q�j���5��)8��w`� ,O��� ���5��W�$+����@/"���G�z��Hv!$�Q(i;� ]g���fQgW���-s���G�{$:����S`=��� �$t������k�>6 `��CK�� `l�y�9@�`'����x�\A���MV�Z��Rr��&�(;E'��o��&�2���](@�z,���H%���.�D�nz?pPQ2d�'���G��� ��x���> =����?����i����,�ca)
����}���d��\;����r~U[k��G �����!M-�p�x���e(����8K����Q3 �����&H-<������7�]�~R�N��� FY�g���n:
�[:!+D� �L@���m�_�h+?�-�O���o)
�� (
qhsbn}YB|��� �wO����� 8����u�����ZJ:�����X���&�|��B��6
�m������P�|f��'�Xep����L">[C1;��I
p���N-� 5 PU��l�:��KU����Szr��h��i�����.�X. Cw��3-����������� �T��(���;I�� ;�a70� �E�eJ� X��}9p�n�� �U���(M�����8e�0*�+�0��>���� ������ ��a�X��cr�d����J$�(*�ki0�2�N��N��>���)&��Dh-?�� �o����mg��U�]�`[�����A~cg��|Y��^3W�9����� �v��"��@�4��7[�R��bV���1���0
��<��wtf(�&����-,��-���l�������B,�X��/���|��{9���`��"�J��A���Dw_(�����0
DKGd��.��n}��Z���
K� d��G�^ ������� ���i
��4C �"����#o�� �8�D�������B
B;J)#�S�"$�[�]����xI��J��c�/� '#~62 #������K7�����V�Fn�nHK-d������k��/��pJ� Ap� %�hP��F���`��O�i�2F� ���:\�YD���p �{����2�$_V����<�PI����Q��(
]�����x����C�SL�6e���v 9T$�M�P'Zu��d;� ���2�9� d� ��:�7�n�ABt,�(���j;HtNTV5 N����l�T��s G��t ���jE����S�>���k[����b ��h�����l�o���o� O m��/'+�t �GG!���U�%��� =%9�~4����fo L�0o�n�1�uM/��3��iQ8�=�w��u������/Gh1n�[�U�~���{jW;m����$
P�x.�N2$6n�H��Q�:/�� ������_ � �b�Aw#�EPMA,<�b��Uhv�������o�Y���,Y����B�N����X��z��{� ���S����lS�\��iR�j ��U���/�����Q3XM�1.@��r0 �lf�_P�:��9���&
vh���]r�����_WoS�L3y
�� S�y��_�K<����2����"_u><W�x� j �/� FW��i2��Y��g�i����Y\���,C2 �� pb16�.OU����,8b����4�m@��>7L���}n"�R�D�/��z���-P���������g�3�w�x�� ���� � �����H �q�B��� �h�~8����� ��U������c���Z���iO����������g�����U� � ��Y�����qj&A�v�q�/�o��?��@�$}3[�2V`~�#��&����UW)��{S ������aN���q��x��H'��c8�5'Q�^���
�O �*I�]���J^�^-8�,7-����1c\���v�H9.I:���� ���o����UT�n�����������'����#M���q���������
�sf,� �KS ���f6�N�&��$ �
$.���!���`�� K?�LI�L�>R��X���^/>_#�9� ,}Eu$�Wkw
4U[�b��:@�s���|�^DD
�HG��kY~@��t AF�z�8Xt�.a��|*�\��#l���U��*y� 3|�]��jY������,t���:
w�����4|TnBp�1����0I(��jv{h_~���!��2����@D���/����*J�h�Z��0���A~�H��A�`8��Z��p�J�W�1P@(�;��<({��'B`sp��s-�9)����
x2�
t������h9����O2���� �JC���r0?�<���]��52�3�um����
��������r V�3�g��#��� �=����r[p��JHd��h�Y��������mF ���
�P����O��4!��!}N)"xN���# �(M�K�����C $G��_iE�5l ��_{�����G��c���W �M]�C4s���� L��%�{���O�/Y�b1_'��MApl� ��E��K ���tS7�� � -|"��d)���%
9����������Y8����*x�.Vz*�Z8k��Bk ���/��x�r�5�>
� 1�'BG��v2�Z�NF��q����?\^~@����>$!�A�m)���������b���P�h���5� uNH�n���PR�|����@<�76��[]S a�@2��_]�� ��F"Dh���Hp��h�K� ������y���� ���a����[�Sku�M��~!�f�;����V��=q�N���/��ve��y���v����x� �Z�~�L�~[���)��d������-���5��\3%B�]~U�B ��OA�Q����� ���{��+��� jbL�#5� n[���9Y��rn�I � Y
��H��~g��\�96��9+ � �p��B�j�`\��{Gd ��j���r�&��j�D��F �]S���a�Z �*' I(S������8�m�������^�}V�^?}�C���J�~��-9����q ~��������P9u������� a��9#��k�����iY�V�������h�o� ��?BFT ����(�a�hy��=^� *�����
|v� I0PTV�FZ
����
�@~����TG�o���
�j�� nM��i����W�Iq���3�+#%�����q�<[L3�E�� ��xi<�`�x�����!�)���]��u] �����A�0M����1��az���QA���� ��Ci��iF5V H��k�����ra4��e����/Go��������`���z���wH�H7Z�#S��~��4����;��� �4=$�p����~���>� ,2H�����a�*f�=P�~���S1��q�-�)t��������_�E �V9�O����X�kA�����7�� E�]xrO�Bzm��t���� ��������$ ��,~��,4��K)y������F6hyGn�� ��:Wk��Y�k���i�fh �|�w���*?���������1�0{���\T�<�^>��ZrzwK�`��;�4��Z"f\� �J �=l�n]_����7}��Wl��_�s�p`�����v��� �q����N4��Jq/�A �#��}W�+��J�����kW�=�����h��):�t@5HV�:_F m ��Xm[�"8 ��R+��]A����q�4���t��>
82���<���c�Lr��.%6Q���lf���������b"�����Cq����:F�vW2��@�� �R�Bd�D ��n�!Z3 ������u��Rm��0GER(�Y��g ��h��[L���$�2�ld���9�� /����66+,�M��fCM����9M��_�<|�[_�����j��FU$!��� Oz.o�B^G~��q�E� �M�o����$���������3 ����wfVm�� ��#�[tiyDF����0mZ��&��t�nq[����
�
��T��
:�� �z���Z�v�e3[cEl����+� eux��98���X�M�+ ��%�1�K�AT���q/ ���l�Q��6a���%�BF��cS���r-�1t Q���'u��o�+uK/� ��Phdqg��`�W��D�h�d����L$D71��]�< ��o� �.S�rv��������� ����y7��jZ#��m��� �.��]��� ����)�P+5!�I��$N�-
�V9��������'*�L��n!�}���I`��C�w X���2 ���N��M����x�8����:�&1#��1^L7�y���lz�"�������_3��R����q�����M{����6��x�)��%=u��D��F���`v��g���?�}�:K���i��:�c2��������|��(��
Z���������q����Q��`5!{�����TB���L������Kn"����M�������$ ���<����BM \{� ���B�C^:5���%��G&L�d��s����j��$� ��i�+^�,��f���1G�� ���(8k����:)r��[���g<U��[�����d�r��E���+�TydZ�d��TDK�P�,���B:K|n�� V�/ �7� 3c&���������%��Q�i>KWM0�$O'
���"�)Q+AO�#��'�E�BDu����`c���1��2H��z����5z�!Wj1.��"J��/��Z���n�I:��v���V���+L��fH ��-9Wrv�]d�UV0Y��G���}�M0Yc�vwB�3}�[|�DGS��|a��X�-h���j������Q���Z�\��@��-�l��KK�R�]���W�sC�5�fi��r�jQ�;���t���/�����+� hI�)b#�� �$@�VA ����s��``W��0A���� ����|6@IO �Q��gb$��_
�f��b
8);K�w�&���/��EbJWg����T��Sw��i���fE*�Y�(�U
�v$Q-QiQ%���V� ���x� ���p,�J��6z �k�pl���,�?FC�^M�@+��F�]=��9��>��������������m��V-<z&}$�@1 S#X]���i�@eD<*|��~���,?���Y<=�����c���O��I�������� ;{Tx��eR �~��A���qv)1tH���f�7&�F��1V�\!�k?������� X�B���W�(�AgG\�7��������(���#<�&�!,�B<"�7f���\lVV�C�'>?W7�+�XT���\� ��&�H�����������c ����eS��[��3��(�_��~�%.N�F� %�������Yn}�� k���������N���W�y 8;x��kB�(�<���
�:��-i���L�����r�5��LCV��V)���f�R��L lVO,����)*����
�gzX��K &H��7�� 9�A\45�M�E��r4� �� @���� ��g �V\(�6/[
��D�`R�Kw�&��N�*OYI��
�:�,�1��`�H�K&d���
D��L^)GvA(�^�!�X*)��� ��&R���3q�"���'Q���cM�`d �����%� P9�/T��|������Dm�!7��d�IQ��!!��� &���*�(�w���);��;C�Q�C@�;��
�����"h} ����K-�T�3� ������m��>uZlH�Q,�;�C�U������������ � ���b����
> ��z �����W��X����� F�)Ba������ C���2Y%M�������������7�j��T�& �l�V�W��zM� ��8� @�.+�,�� i j�,k"I�Y#%!��)��l���RG}��0]�!j�N���j��.��e:��kQ�� �;�`����"�X��!= =C��5��,���ZZG�d��Z�e���tk ��u��de��8P�u=12mch�}�*�|> �SRL"f��x�m�, Qgd�^"���.^��JV�tRa��o�
S(f��h�2�P5DV������N�S� A< �Zm�^N� "��r��l�9�1�zI��% �0��
0N�������j`��)������t������X+F=��aWt�`�'�e����M���"����Rd#7��<$zmf��R�NjY$�y����X��^�_���,�������5�L%>����,�y�k1f�����:���M�?�����5��R�f�6F� u>�I��\{&B��^�����##��;C{�� ������9-1:y�ZB��z�?�����T� t���W������Vg�������Jz���k��%�6�A��W�'�q��l��[����vmSO���>����)��vi��NI�[2iOn�I;Z(r��L�G���@s�A�d�AIy��Gq{\�f�Wk�|��Z����w���P�U*x��(�t� a wu�o�$3����� ?���
6�0�*�����{������S�Z)�q������_�o���W�?�� �o�Wa � X1�,��bFUxuqe��~
P���������yAz/L4�v ��N�?� <��AbQ�c��3bVh��H�D"��z�U�Jzy��O '�0�y�\�QP��S�������~b�v�(�3t?,��56�KG�����;TuO�"=��"��������N�Zk� -���y6��)�Ji{���/23�
[����^�C^����S���� ��pW6�KJ2���g���Fl�o�&F��� �~�e�G%W��&����m(���En�_*}d����A��J��R��o��S/�����3���o��wP���4������EK+�[x}�@�
-���E�\���!�
F�*�.� �%� ����f����v���`Sv�����+;�����@��J�����/q�:�(��d( wfO��yl6��~,����9 2jTQ��)n1��5��0~`�o�g��$f��t������6�����j�a�be����-�!7*�s9����a�~6 �&U�B��V�P7�or'�-Cx���f:� ��i j�\�s'v @��h�q�:C�_� k=�Q�8 ��'� �k��u�bp*�avJxS"��YB� �5�v�?����Ut�b�����������@,� M:�b�^p� ����!�(��dPa@^����7���9Fu���
�q�%��p�h�Z��-�N*85�s�pb�G�Y��P2S���P� �����a]X�C��4�����p�l�87�-|D�������U%��}��N�-_P����,�z 7#+�5N"��-7�W49e���,�����\��l�P
H�|*Wc'/��u^R(M>c�����z���,i��kE�����+%����G�6`e���`�x��q-
��B^�w%d��'e���_���s� Nx~%h�D�6��=����,k��E� ��G<^� ~R � :�Xq�����d�q�m�^� ,�i���yi�
] F��,$�Wm�^lj���b��o�E�4�����������cnQ. ;*r_<:�H^�g���� ���Z� �y����V�F 6����p�J�
2j�����f���rRE�M�P1�6;�V�^�'��q6 �>�2E�]H���(I8��� :�C�h����{]i ���G���.��Zb4�����E�]���"�� 4�w#�AtO� v���0��!�f� ���7����S~����\�o $���#0� �NiX��#�`��
��2 u�����B6M�����sD~,���8�GG��~��32' ������(H��
�;���Gj35;r.�$5#�����x�l�j�f�i�c�"]����Z�c��*��
e��� ��@��M�6c6��p���� +!��|�;58N����?o��V��fg���g�,ng)+����8���@w>��������a�o.bx�����)��w~#,�m��������kj�f�G�q�
e �]NI�5l4'����o�����ItWt/�&�Qfg5�@����#L�x������L��x�[������������s�k�f���W� A0� 6�%xr��E��6����7o���E����7�=z���/� �����t���/�%?/�G_=��27�������)q (U��C�'�������D�>qP3��O�X��D�s�^�z� ���P#�o`� '� �h� B��\f��[�~�R���&��a�7���F�o�� �,���da�&��\Xr�/ ":c�_kv;���?u[����"�Fa(Jg����J�E� � �w���.`��|�MI�� ����a!� a�������������0�X���.4]��[�� ��YK{�Jt�&J[+�6�~ �s]�! �1Q�[.��!JP�@�1�1��e<�L9��L�b�y�����F9S'�"^�kd���� w��h��2$��1���Q X�=�d�&�Z� ����t T7��g�y�2��Z�!=���Z�'Q���Z���A��M�a��AP���MGQ�X8�t����8��� G��v^k��w��k�YX�����Gh9u �0bTH�/<vT%_��
����d�4�k�y>(kd������?Zs0h���\� [��|�J��fk2h�;���.� sM|rR�F��y����%J#�-Da=�
j}AQ�O�x[�>?������(�a��U��X��$�R^����u[+��0m �M�1�pc�WEb��
�0�*�M�(a8nt;�V�~�l�M:�RJ]�Q�hV@��� ��2�i+�b���] p��������YIP��� p�4b�jA�
��|> R�z��U@w&��������F�F�%��(]4z� �g/8�����G�S+N��;��fU�����D�.sS|"1}�'�;��"Q��,�S������4G@� O�m.�`��`�_p"����0<�F��n}���Z!-y"�U#��� (�����CH���~y��%\. �?9� C���^�m�`a~��u��2������
)3����2��_
�9�|���� !V�-�AV.$�`�;����A��t����M&��� ����.��s��8�n�������Z�z����K "F�[���IC�D� 1��� ��H����%N� �FsR����VF�=Y ��Z��]U�/�� t�e�����)���Y������&�C�����%��N�b�S[�����E�/E p9/}M��d�~E�$o�0��k�f�w�e���B ����B���Kc�����9��'��a��*eb��2&��xL���Z��yr��}3�W&����
���Y2;S�F�52
J �����(��/�~�8�� �����p|�|hN�'e���-A�v��nS��z����4�Y�x6�g��S�e���mUA>OC�w��\�����[0F�l N8�:9��"8?L��HOpy�d���;f�C�{
��u��C<����y�������`zC��)S����#c� �7����yd�o���k0 ��F�k���3%�F�����G�E�R���48�3�LW:6fu��O����
�h$>����)��I��]A$E�W�}��F���>������$1$������Zq�9/��G�Q2V*���:��&WY ��g�]��xp�m��� 9��qQT�����#K��*G�����������u R�U��E������sG�`D�$����;c�kha��d��� �N�?�D����S5<-�sb�$ 0�� ���UnCB���W).<o�-j���\�I��o]/\�A�6l�jR��,�� m~4�$#���d�N�� FG�nD<�z�.AH����*�$�?�KF0
����gg��C!�Au^O��)T� Q�� q���s�L��(�`N�
��� �/�$�[N�����j����}�K����|�$�}�>z ��CO�UV� ��G� '����7���h4����U��m4��?��o�[�V���5��N��� c���
�������>�?�F4��^�n�����V��������X>�v��Fo���6< ���g���*�����#���m���� 4;�v����^����N��>����v�'�<�����B� ����(��8c�kW�����^������L� ��-������������r�X� aAV� � ��}-��$9�MpN�^?}����7�=z���#2z�������x�k2z����D�� ���Qy���h�r ��j�^]��*��0s}��?%���V)kl\<L�����
��k� �8�=(�� �q���Y�\ +�)�=��?�|�o!��!�^��.�� ����<��l3��K�<� ��,�/8���_}��x=� m����\�n�a�o����Y�7D�R+f��p�"�����G
J<�2Ya��O� ���`� .+P�?��������3��[���)��3��aG��_�d�l 4�����D{�D���E����y ��Yhb� �jib���r3_��gk�d��b� -�$��~�+��"���\S?�8���Da�����5f�>�|�����<� j��<���/�%tS��������{:���:XnqR5���9� ����l���ZD�e��+& �td��MH,&�<�L�d �h����������u��JP}/���u���0�|2e�E��4@�O�;��+4���
���@�U�������V4,�2�'r�;k� l.�� � �t
��\���f���-�@kK���)�M�(�)�:J2L��|���Q�h�`9�$ ����:���.Z|�\�9�C;�je@'`��%��W�D;7N�y�!�l��`��b�����B����n�*d�a���H����x35�t)����z���5[-��M
�D�9
�s���u$t�5W�{�%Xi����:c��A���k�#�?Z��:��l��Xut�%T
R>l�s5��4���gS��?� &����IW�5Y@� #4����;@����-���xjF� ��F��Yg1a��C�^sW�{Z,*%����W�!`B�6k�a$���Pp8dI2���c���c��zl�F���`���X/HUCk��
+|���6�#VE��:�Bw�+U��������5X���nTj@������U�wOig|bG� p���S]�+���ak���14L��'hk�@�; }U�� T���M�0�8p��'�P��-���k���A-�/$�m���TDp�_���VL���l4�a��P������"��y�B� `"���O�*^%���J�/�?�/�H�ZDe�J b'~~�93����&�a ���}\�5t���A}:\*�5�Zc�q�w�Zi�lq6(L����������� ���H���d�JC��K�H7P�O��
�1��,�U�A�S-���W�t���������e���!R�|�I�7y�`�����mQ�;��k��#
r���7T8W�+�����j1_����}{p�*-�2�F � s�_]�7�����S��HGB^����0������#��xr����-���kc) �`�G��]� ���]������
n���p���� �#������I�@�8Q@����� ����2 6��j�k��LQ����:#��� X�< ]YN'�����v���( �a�:�53+b Fp��u~���� �U�(����o�;�Z�(kn���<�\Vcub ��U* �ZOg�m��l��$
3n� �� �e-K�% ��fp��+�l��*xM�I�u��0<�
fh�*f ��W��1Iel.]X��x �vC��������n�� �ePj�����Wb�+t��� �n����P> ��� ��������o������n�<x�����`�p�+�7
�l_vj�ZJ���]���[e��U��� dn�y� � v��o��2�3��M���Pb��0b ���A�f��%�P"@ �
�H,T� ��� �� �i��Zo8����D������-����;��d�nSgs��pV|���9k�5��^�B����M@���t�Z�x\�����r��0��K��YWq�x�G�'��d�| )qrh���@t3{uv�Q�;c��_�����}�X�t��� ��7m�F/����0�724�s�L��Q(o��o�8����� s�x�����>���qb[�Z��w3d�7��D����Ee7���ib�?�Uo�����;x=�� :^ u�!��G���9~>�
d��(9.�.H�O�������Z�����~�v ����w������D�a�>�$�����E�TZ�\�G�p����,\�� � �A E�JBT�S�� S��nA�Q�~���KE}B\�F�����W4�������w'������X~/�^�������aNP)��K.�*7u����o](��_����-a Z� ���d�n�,���L
\$m� �82*�2�f�6�5C3 qC��N�[S.3���`��\} M�����q x�A������#S�Y5�<�G����F����3D��{~�� m�=* >*�>*�l��0�*�#�@�F!�)��P RA RA$R��=8��aH�� ��p��c���`������{*�h|��\d����
<&2p� ���VQM��>�fuJ��3�OiC�9�n3�0����� �y��|���ez�;����)x���� c��n�c\���H����(��l�����h���
������� `A���z�#�e�� �!����~@ EA9X�E i|6��h�%]|^S1��X�@�P��#��h���dx� �?P�&�Q=1}���v����������H2��,-�o�����6�N :�\@������*V����X1i
���P��(���9������YshQ�I�y�m�| ���*�<'�_ �eJ��bE�A
=�(�b-�&++^� 8 #���y�q] ���� ��C����� �����[r<�a��^K��vK�A8�N�ld�8&�K`�Gz���; ���MV5��[��>��x�����>��x��=���}����o-���V��o)�[�j�xo����}xk����o%���6��=b�q�������>��x��[��>��x�������o#�[��=��w��>4���{ :BfqLl��������x�HG�����0����`���L�v~�s��d�n������o\��q�fqw�v6$�
�Q%�
a��o�47�����
���R��
�a��!�z�/�����
�����v �� � ��d�c��v�1�[��i�f�������"�]*��"-�]�(��8�\o`���1�y���^��l� |�[R
d�j����`&�F��<����������dUQ���(�"Mf����a
��)�R�f��� �+u�O�q�d���%^�j&��b��z�T��^zd��K��C����q���*���Q����LI$�k��#c��X_���/@�#6�-��x#�.T���� ���fS�U+���5��v+P����:=_w;D���<���Vv�(���n����-���
{l��)J��:u�a�r�����K�J`�T����� '�*���A%��N���t[�,�Q�7���4rX�|9 ��L�� �}Q�D8���=2r�iQ,�1��������������6��7��M�P���`����!����x������'��z�r�����������=n��/^�|�����p�>������k+��mr��}�F����h�/�3k�6$����H^&�{���}�j`t,��Sa��e< '8���r�Z�~���x�;7��/��N��� ��wE���J�9����w'^��~�d%
.�*��J�T�-��H{?.��r�M�N&g6Q�=R�)��=+�"^a�fa6�V�(����l6�$�8���\��k#rV�d� �e� ������������>|=���QBU�kC �|�y*�s��Z=E ����mh-�
8F5z%n�i���gA�8\�aE�=/���3*���xJY������#������E*�D\^�����Tc7jA%�� ��r�D$F������:�L����`[��������� 5�=�V�<r
�r��<w��:w��d��Z�B8w�M���Y�rE %�p_�t��5e��@�����Q�L*�<�@��|^��������Q�>�K�)�9c
�m�9I�1���:�us�Y\"^�������<�e�<����<����B � ��
���qGc�n����z�W����>��z]{�����So���{���<�����
��/�0B,�/�->$h�� z�<��\D�-{���������� z�L�=*�D,�T�U� ��$H�j�S|A(n�SR#���n k�v�+��g�@���������J�oI#MX%��W!�T6�
VIj��+@�y,��9s��:j�9��W���(�i�5�<���W��6�H�S���v|������5$A/W Z|
�Zm�c �� �#�K�. };�������b��:��s ������BGvP��s��P�R+��� � ����Z���Z��u����A����������3����f�c6�W�C���<��u�i� �3�_�h�� �k����w��
.|�}g�'7��4��;H0�[���.�>�w�O����B%��A�d�g�9�G�7�����st��!Y �X��(�B��8}>.rq�%���X +F���� ]t�� �����?�������,P�]h�����B|��D��9-�6r4 q�������z�T\t�?
E�� U�v��3�{��V���zR��d������T�$�����t��'��� :i���u"V ��pT
�#���>�'��n[��e��Y2[@ �<:K/���^%I��m=��C�D0@k��C��[�D�3�nt������T|Gl�%�)r�Q�]A����/�����k���(��(�9�7E��� � ���7� ��v�$�u0����V��3�4���m���v�|��Rx�v�� n�m���>*��
o��+)p']J�[)x-�|*��f�� ���g�� ���0U���� �>�ln���Ql�r� !e$���&j�1Fw�@� @BA2[N�h�L1�#]cf�H8���zCy4f f���Ei- �7c���,:�T��Y��0j Lb�w�l�y� �@.'.�_���X���M�q�� �g?O}E/��8��:!���sHP�������6w�1�ds��:��gM���+����N� ����#[Q�k&�S�E2Z-�! �qIl��7O�&��6}@(�emBq v ���6��&��U�ODD
�;T� ��z�z�����*:�r��!i�2t&�:I�4"�5����)mH���O�)�9����D���&"��I
��A ;$E� �4"�J& V�ef<P����C�; zjS���@��|6�=��!
d�l
?��O����JTvE�[��UI���C���(�:�8�j��k��GY7�2�JDt��h� 2���_kvBZT���EtE�"�DL3b}�q��j�;��Z#�Z<!3���pBV�� �,#��[Nh����Q
�� ��E
��B/j�����v[�����BvD�#�5�V#`�8�I��kO�Z ���7�Y���]��p�,�{b������W�4�{��-�T=i��p}� N���4��o�E{���p����~,��� Y;M�����b��(� ����� �G�+ 3yu��z���,c��+h�*�umYP��,/#l�
X����e�����0]A�Z��v���\�����B�a]��DR`!)3��M �pMv^.O]�������Hc$8S&�pJ����t�|�����r�b��("��(��;@���~�Ub% �lE&��\p�CG�=��Eq����-Y��>���M�� � t�}!-���%��x�A���mY^N����������,/Ek���mZ^N�����ky9��� B5���1�Z^Nn`y�C�by9�ny9�ny9�ly9����o���N�,/��9����~0!�Ga ����D�� �V29Zd�0����(���� (a�2�X�x�f���P��#|�=� ��J�o �&�GY��-��M��k^@s�+����&Vv=9cKmvo3��9����'�J�7�6���? �Z��P�����X~�e�����<���=R���(X��Bh���@�����]� �� Vg����ze ��5��!�1_�h}�S�Q�H�uM��Q|��G�$� �,:V+�%]���$��:[�!p��u���#!�h@;D������A� ����n�{R��\\���2�Q��(S���[ s"�^%��,!�T"���$`
�GU��q
S��FAS�nb�DC��j��� Gk����8��'e!'�������7����l�O�H^;��F�� lX,� o���wNR:a� tC���d,�^�Xl>���(C
�N�!�Kv�a�LJ����~�~���O�o�{����GAz2�y �� H\[_$@� f�f2��
z�� ��,y 2{z1�MwV���S%N)���(M��K ����K��V(�Y������� z�[$���B��� )dA*a�9<���Z���!���
��{u0��^��<���w �hc���T���QT���=��{&��{OX+��{��:�4����,��[t��f�ce��J�W`�/��+.b�*�� ;���4O ����EC�������12�CG&+�b�� ���1��I"����S��=��[q��`<�`�������]p��'..pa�{� ����u�{q9
UF��� �����t�bJ �!W55�������I�Jp��T���#���(�\,S��W/< |��; Dz�;�OP3������ 7 ~l�p ����l ��ZHu���U�F j����x9�����j�E�p (���m�C�)���F
���� ����+�Vp��|=P�6�ZH�.\&��H�,Op} �����=3N�q�<������F_�
��?��I�0� ���ZK�!�9���n�^��<] ��]�m9������������9�%�N���$��S��3X����#�Q��
��25��m���5)UZ����T9����)�5( ��� 4(a�C���)��������|�ho��.��|�����B�1 |�{�������w����U}q��&X�h�7���d>��+� x2k���C����C�Q���3O���W�v�9rG��Y����PQ�~��P��5�tna#�����
���{2��_
wu���P2� �_�� �������s����~:�'5��@M�J��~�^lV� ���.��O���B����tm�� ���|��� �Y�N������x�]��a����w,o����&�V�R��I"��l���l����k;�)8��K� �`N@x��y#��'�xK#7�;�6P����A���rM�;��� �����, #����J�Ld����5�2�K�T(N 9/m^��d�{���c"�?�c���e��d YG��w�;�_^;v������TPm���U0Z��%H�F^Ny�Q�����_�\����"��o���������J�\� ��)Qr=IEKw#�-q��s������
j�a9T
����J��x��Y��f��U�o��f��Y2H.���`�:c�%��q��M�t�#��\��f!��=�u�|�>/�R);e#S��f�A`g"r�3� ����q�p��u�X������v����4��+
��
�������������mO�o�6 A3&��>E,��2�F�[k5��[*�wK����������n��o8HTo���|*�$T,���d`'
����u�����p�F!0h!����T{�3����"<}Q@!P�[ ����y�r�JAv�:�� �=E���^k�o5j�aC�g�3�B�����9�r�~Y9=?_l�����u:�zt�X ���Y8o�����wWo��2l��V���/W���C8W���]�%����U� E`��[�=��C�\5[��{���U��zK ��.Vm���d������������VX{Z8�BI7[�l�B� l+��|�6�]���v���`k�>�����yQ+�Q���v�������(+e�g��.��d��%�
��^�����������t#��7�{�n��sW������������/�}������K�����%V�eh{0��U]ae^��+��~C��s��]�����N ��<Kv�ah�n�@����� �� �5��B)v�5�M;�]��Mn���h��
]���p��o�h%��pw������V�;�; : ws
�w�� � y +�uGen��B�K?���w���PTs�4�NJ�[������[Vt
�o��pK+�X�8�,�\��[H�hGwa h���� �� CW���g��4�^C��;�
��� ��"��"_��V��3�z*�Tx�� �-���}��V���_h��R��a� ���B�� yM?�+�'�J~A_i���7v� �|CS�~��C
V�9x�������'�d��#��z~�a�1h�Z��(������=�P�( uHN��d�
I�v��E��x������?���h��.'����C����� �(���zg�l?j����!����h�?�N�W�����l������o�Z�^��f��n�!��6'Z�g���]|�3��nD�u:K�4���������Fo���{��=�4�O��A�5����F������3��O�?�����������6��M���^��������!��3��|�����%�}�����}VW�T�z����@�W����R m%�� ]$�,�?��C<�����
���+�m+�i��$Q�� �DIc2 L��z��;�!NT J,�G��/bL��l���"��1 J�� _D?��d K�����������Q�V.�ChdM�h���`kO��'�_1���X�K1�d�h��6"���U&3�L�����z�@G��#%�������veT� ������^�h�R*��-#"��OCO Y#���btn�- ��6L�5�� ����&(S"H8��Q�T;U����t:Q��@5%z 3��N!8+�����&�CN����{(<J �F���O�If
��H�6\��>�_��:M��<�#~^K�S}<��T*� ���)
n�o��l5��7g�������7K��� �X�N���6�4�QR��7h�{�_\
�b� �(��X���0� �� �g ����!��j�5����� o<��=��Y��E%��Hw��������2���?v#
�z � yY l
G�"���~�=C�_�����9!��w����qM�kkZV_7���.S\B�-2l���������8s���'�hu<��M?v�+Q�Q���� �v;����Z��S���q/% ��m!+�aV�k��sa�V�}u'�J�-��A��99^=tE-���3�f��]_�����}�C]��[� PY������y�����Wo���:���B|�A�{����<db!P�|8�� ���4E
��\� �6�z��%���`~��G�^_,S]�!��?"���m���(Y����C��?Ey���� |�7��C� ��s@���Z���0)�]e�r�=�5�������������X����3���#4��d�#�2O�?j>c��kX�-r��'��&-�2��8�6��t��=�v'��x�.��3e[a
}���"�~����W����v�����r��& N�����5d�X( �[��X#.��m�7,T;�u&��-���DB��D��f���Q`�y�r?o;QED!z��6�����H
�+���X���,�/"z�`_ ���
�g�����d���G�TR�5� F������o���}-���h�Y/����l�A`vX����[ 9HUK3��1�e����+�4�g��prV��;�������,��jim���n�V�%*(�?�X�o��D_�}����_���J �K�shql�����TD?,���ts���E[1���Q��Cv s =%���{t�: a�P��Q�s���]��������t������Ir���( �N�
�������U �}��w� H�n#���/��&�t���B'?<};�����_?�vo�=������)vg�%��=�/�{�d�@��.��n��B��4�<�wh�i��y:��QT��*�p�o�c�� �0"
�2��+ �w����Z�(�z%KaE���!^����V��$[�i+Z��q�S�@ @X��5����5 �F�U�7!\��*��e���5k��+�}�e,4�� L���b�\�k�K]�2�MA�!��z��G�H��q1��H�<��E�[���8>������A��B�v��#I��XQ�� %��(|!{Rv�*~�Q�wk�A�"��A~�o�.()�R��-����wD��vp�[��W�O��La��5��U����6�p�K�N��t��j-c�Eu�`�rl��� �f�%�XL�yX����pE
�i�ez���gK 5��ku��w�����RTv/��{E�lA�� c�������D��72'�te2q� FtIAy��S8'Mg�����8������.�����������!��rr�4�S>�*���.��Sn^ -�";V��.�P'���n�� kr�R,v��l/��6km:����"�P�,���W3Z��;E'�)�Vj^j��gq���8[p�@� �� z�@Ld�_��Q���� �&mb�z��s ������r�&l{��e�SY\�q�)�1�q�G�9?3�C4.���� ��a�A�b:�$s���������=�'/��5��� b.�t>�"��:�}S��bm������^[���%����?��]y�Ml����-� K��{ ���7��'�7�c�>[d��,V��e<�b|�7u��n�������!O����|o����C�H��uZ=�jzD�{�<����T��4��,�o��a ��>~����E� _<�����p�'�CI���3*��� %`y! @o ��G ��#�j�����;�;�����O}O?��p��g U���5�7#q�s06_��s� ,��k���1V�S1 �+�� ���+�g)��������������X
P
�� E�!������ �Psh�<�& L&���_���s��7��Nc���!E�\p�����%s*x
g���0���2�R
P!��r���U|mpN @pMV�� r+=���$�_$j-��~������0�i���'�3���U?�) t#Ic��Kt�����&�/���.����\����tl��}|$� v�${�9�G��1�8�0����!��}8��^# ��[���)����v�%�cC�M�%�I��a��N�������/Ly��0z4^�J����n�1�g �������=� C0.# ��m�}#��f�(��� (�]�����U��r����7��3���u< � ��)���0���b�XwPL��� r ��B�!X�r���E�uA96B����4K������j����u"�BHb���u�����Fp�#�)H���X9���k�1���,}��8����4|lD:E
�l�mi�5�B��5+[��3��C!�����4�p �����[>�E��@+c�`ru�T� �\�"0��5���r �����>(��(�[��FcE~J^���$W{�����L>>K1���r��FM� K���#�,es�n~�!�� A ����4 ���Q�Q�N�H<��]N!f��a����)���B ��(����85��[���ks�e?��������s5w]n��(.v �v�p��f�f�=qsv�l�g��z �F�k����Q��e-�A��q��S�A���xPQ��xh|L�����o�@
3 �XKp��Q���*@�h^S�Jw'� �J��dM��0�sA(|��nka�6,|������S��� X�'W�����C<����Ez�LY����Wo���wK�W��Xq3Y��P�Q������_�b�2;���B��&�29�d���8�C?j�,�-l-5�V���/= Q�K�����������7J�D���t�n7�w�+ �E���K�m �(������ldp�L�|�+x�A+�.���xK$���������
K���-�j����
j�3�0� ��&��d�o�2c�����v�m���/'��]:cn
P5A������u���| ��P��
��~��0�; _E������k'�V@�R��O�V
����$J����%X������
��t~���i�t��:�u7
��3o� l<a>Ii�/ ��[M�VF� z�K�Hk ���[k��e!��0����J3 ���M����3�^.��kri� sW����`�WJU�+�f�� ��%�T�/t�M/���*EU ���u_5 �
;�� �� �Q�HO�q
_�a��m?D�8*�9 ��E��bKg��Yl�,6vV�v�;�~r�� �f+��45� ��
C�B�������P ^@�
q��x�txb�4 �d�a8"�����8�s�@`]�:�t����������&.�Kb��0T�O�&�G��[Q�DYS$�f�����a�wK�V)� ��5�W�����?����Cz�;X���*^�A��2�b��-dT� P
��j�:^O��[�w���� ���${� E��`���"9[~W��`��8 �`����l� *�������V[1��o�[�����9 ����g�� ���G2x��T�T�*�+��"��<��~{�� fG�;/ww<� ��!gg�d���M�/���s�h��+����B�Lu�<J��
W�?���3C���H�)Ek���������w��}�����m��� �pdKo����M7*�$��<��,s�,s���(�@m�
X#`S?��Z������8A�)���:f�g��Su ������Ic5�|d���1j S5Z��� ��-��!cE�)N �aY<������7�_�l��:�d�Pz&#\%zebZCY�������x;h�k�~����k�M�1� !h�����#��I"�?��P�`L,w�����U�l�7�P{��V��=��AL������,���<�ws1���)�-�\/�n�M4�V��w �
���.�o���
���Z��m�H�/f
��ZE�ZQ!�1�j^ o
�3�h$4t�pk��`��
�y��[��)W���e��m���� ��CRpL�J�Q�tX� �m����rh��&���(��?�����>��Yt/�G���rGV��+�&��W&���������g� X1��o��� �T���+� S��2���/ ?���G Tt����1���Fm�����r� D�$�Ej(�K<�D� [��"a�P�+���\� W"�U�*Hq�:C�{A��\p+Uw�2���"�fc��]����NW��N�'S�~�����hCZ�O��D+d�-��!��1�E2�'���� ����kCRw� �k#?�7 7����Rz� ^�E��v���{E����g�;�A7�U�U . l�9�����M�
!��@R(�� E��-j28����\�Oa�Oa�O$w�P��ra�B2 �j�f��K�wK���" �C ��+������c ����8_$}I���* ����- N������;4o��tyy�[4�����K�a�L���[0�qkc�fu�|��^�6 w�b��=@�O��D�� �k\`����K{��
A������+��#�~���=r� ���v���a�F��=��TDTY�X!�z�>j\��?�f ����9"!��T��+��
�9_+
�+��t�5��j4��K7 _xgt�/����(� ����?� �qW!� ��4��
���y��^��Bb�b���� P.4%?u ��3���'���^�T��� 7 ��#���������( �����+�[����E���O
��l��,Y>p����[���1��� 4�����;@��~���p��� �R��o�t
wc�B�t|_K�Vm�
%I ����%�-�p����Cv��E�m�� D�"��D �Ep�f"�R �
_+��=Wk8�5�C���@��=�;���f?Z����Q����c�J�x������,|�X��
�WN�D�o��#.��� ���+cFu� ���y�!N)v\�hT��O} ��S�A�6��Q��b�����O�l#\��5�e� �Qd��bH���� ��bI�� v%:�1��FO��8��`l��z�o�
JU�$����oT���S����X�8�1�3� qN� x�) j0�+�!��T��A��Y�^�9I����: '� � ��m�I�h��E]�r9va�\C����[r ���3X�o�57�p�E��lf��
���^�����@���0�� ���Y� ��uO����}�u
��8�p�
�^� ���:/t��J<L�2&w_=�=���
K%�"��F�Lx�����D�-1�K���,.�n��p�!��#}���Q|�L�2<����� �p;��TG���|� X�b�r��+XS��o�X������T�o�� C��Q<���]1��6/w���0���Y������Q�h�����c�P�/U/����5���Z��Tz��g�\��F���h����o��+�� �&����X��q��V���� ;�*�MV���zw�q�l'`�b��K���r�n>,�D�( �<�$���q��@�b�6�C�
B[���@�8���I��n��I( w�<ci�%���z��] �"4N-���H��.�+�$'1������f������vS&��kd��Y7��@{.�I�%�9�D�$�>h�?+ZS� ��\VJ^)��H �H��� 8����b���� vy7��u���?�8r b|O�Xe1�n4I��Q����]Js�A��<�u � vv�q��y��`q@�1!���~2�p�;g�<�5���W�cf %w�-W ����Geb�A�X|P����E)�Tj�fS?�� �-��MzG])��� �2M�#w.
���������d>y�X���zg��������\�������w���2���:)d����-���M/h�����S�>��a?�.|��7�V�09��� ����.� �������<����������oGPa���^�cV9������� ���10���/�}�������tH��2 ��z:a��X
��t>1��w�[}u��| ���Z�� ������x���?^�x9���������������)� f������c21D���:�)q� ����D }��D�t��s�.\pr����*!�3,�����M�vw �KT?�P��x�q ?[�c� pG��_?M���� KA�j����JPI;_-f�2��� ���������K�~�~":�(W*���4�D�>����.6j��kt��!��,�s�GT�:�O$yr-j���k�[���D������}����� M�,b� �{��;@�CI�-���������2�|���G�L������W���CR-5s��R;��R�Z�\<Gx��$�����UK��[ �2c�4v��/~��7h� i6R+��� �p�ts�2I'N��a��_� c�+p��c0�n�3v (�MY|N�������E�E5T��T�E�T�/�o�����A�-�����L�$� #���GSG]���Qb{oh3>�����#$���O��{���-��v*�Rp����|���^��{�W�~5Z�`��lZ�r��F�I���.�|�3v�k�|�E�4K�Cy\������e7z�o�U�����~��'n��3�F�����am�����-�� ��o�c�^�� ����
��H��o����Eg����r'7Yr������n������
�n�����a�P35����w���p)�,����K���97��gi��.�{��<h� �^�sc����@��� ����o
DZq:@ �~S�F>��p������N�.��^���r ������`����V�� H������ +mP=D���[���"�����uh������ �VC*!���EeG��i��G)p:?
6��(z� �������c�d$���g� Kx������rL!����T�G/&�"�k����v�~�m��dm�S~���I���d���~�� 7���}�� �<��)^3�W<��OnS:2�� s�55$}E�;����";�#��v����E���3�q��vx�)+��9�Q�]&�&$X��^�Zn���v��d��+v�FdP.�( �+'k%u�+~�0��5��ON�������m(���UE�zB 3�d�e��`��O^�$8�EA�x(�B���)r`������� KZ�7\���qBqk��\ A If�l��<���)9?�^���Wb?�h���k������jU�F�g 9:bj �A��of#C��x�u���>=��m*B)j�`���q��t���q���uj��l���.��R�o�C/0��3�D� ��* �%=,V^��/��A+���Uf2� �~�"/�+��7� �B�:��/&�y�L�0&���Q��dz]���"��~�f�R-���u������'�������&���/�� H!�;%����
�rV���m�9e�~~���� G�^�}Wc��+BH����6�������l�-g!����
���������=�[_]��A7�����"+a{��U��w\�������FO�86 a���k/���`���G� S�Y�Q`��#b
3g#E$&����@o�=}�����
A ��-����X�ml������a���G8����w�;M%�wZy�N�M!�Ny����C�7��+�������)v6����-O6pFn>e�I�s�0�`�|����������:���/�6����� �v(���v�C�$�=��_����1)���j �=�������� ~S��M��t�����dlFJ��h�,b%s�������RB�����
����@��_�A��j4z%��w�]�E0�����|����y��t��^u?A"���P'�Zd_�8@r�>���H
�H�"���h��:��VI��F#�c O�Q�R,!�C�8������`t��'% �� ��6��
]�5U��W���x�I.�0�Oa0!�0�_�!/]?����w�a�hH�h���D ��f�x��'Hj�qg�z����E��g�T-�R�Er���Q��� �x��-o��>���8�(>y����� �m<
��#�M8��2jO$��%�Z�k����0�\���r�H�����5{fQlp���U�o�p%�����"�
���!�e�~��Mt
�M[��\!&�}A�E�����0��]��e�AL�:�����q��V#�c3�&Y��N�@�#�.e��Gl����y�0��f�� ����1����U|��U`
�W��\v6F�������&���7_�w�KG� ����PwG�����)����; ���E�Q�m�P���Z������C���Jt�x % �2��� ���j<M�l���$��s�4����5����c��->��e>�D����OR�$�Az�����'}�u����<��r����"Zm��NA�FU�!2����
���n
d��fS���S��#�e����^�
��i
���n�%Q?�%��<��[t��Q �t�I� H�py�) �Yn�@aR�����Apw�����y������w������ �Q��W]�sm�~�������� ����8P���(���Vv� u�j�1���d�%[j�7����x�,�(� D�X��\�@�eOmh��{��%�/`{�:��v�t���� ���Z��@��V-��Tbf=�"������(F���2� \�� �R9�vy��"�|i�� �J�
\�4wr���yiN BD�����PW7�T�����&�m�������k-�����.��?�+��z: ����xR�}<��$�������^�]�� w!��,$�B�,'�"� ����)����oy�6���[B�(t� � �kJ�@Kn���c;�����H?2l�F���n�l�j.�[ �.�Ut������� �}��Bh)��{� V5 z�#Gj4�1���'� � �/��X��V_�@[�A��Y�� N�����4ZmH� J�L��O� �|$�M!�+��m]�X������se����,��+ I<z8 �
�3��=�_o^|��;��H�� �x�5u�"���� 2-�8����W��o0I�����7m�(MqD����� �I��H7��� ��K����|�� 7ks�����E�`�k������xs��u��[��u��U��xd�^S"�|���u8���k�3�\�]��U� �P�,���j��
��W,�/�N�'��x/� ���W���B�� �� #S ��v�I����1*�M�{��]�p�t��}���i��* *�������,���U\h���G�\f���F4����m�R���AGX��VM�g(�/ �y>@�aJ0GS����y��� ������<YpK\����E���"@� �!�n��r ��<�Q�}�}7\����t��3���9uf�rR�Nq -b��� �S�?����J����/��}���v��������|��A?�=�H�2 ���z�1G���8� }��������[:��y� � ���[�N2��H&�6NkI
g%��@���q1�b�g����R��zX� u1��;�-^���w ��w�u1�C��.��
�,��v��TR$�:M'c?�.�)�a��@Vp�����)� u������9���O�� b��� ���ql1��@V*�.�C�@^H������n"E`:S��8���<�K'�AP(��b3�P�&�T (��dA��@�� x���.Cvc����o i)��z�I�T�f6V t����H1&�����c� �F�km�:��%��e_�:��/b��T �~�dg�m���~���p���L����x�%n��|��� F[��m��e� ��@D�� ib�;�>:�a���x9C�pO�FdK��-�X&�H.;J'�G[���^nA�9 J8����I�9�#l $��dnC�9�_�9���Sq�
$�C �����qe�]��Rv?*v�s��������|qE�%�su��������x]?���0��Q��P���J�t����#^�o*^��;�:���e@�mO~?24�T0h���x�-%������+*�E2��� ��AKm�P��w;8�Y��V)z�&�o��� O8h��n��a(W�v7������L��G
�����{R@��:4 �80�nC(� g~��p���������s�)n@@0�^7���y�8V��^��2��m���^���m*����|���V��p�f��W��S{��V�_
|K1iP��a�� /�����\��R.nK�e
H����|6gc?�������v�p���l����>K �v�I��8�l��z5�Z������(/5��TLM
�JE�Z��O����!�g����p���(�/yK� vV�a�~m� �f��^'������\���t�/<1 ����\�� ���G|�G��pq�-Ip��?���w� �f0�}�3u��o�O��8�|q��R���9�E����-Q6o�����g��98E�@�0
�g����-3#�� p�d�x�e�*+����X~a��
��0|��X s��v�]��k8�/�� z
���=����������Q�u<��g��� �W .��z6
�������t�N���� ��iA��.����5�F9F���7e:�����v�^��R�G1�m�5 {=���7
�����q������?Q��� �s���;�jQ���"Y��P���{�����D�n��o �+���"D���|��E����o����^��*�m�
���l: �+�������_�i�K85�i'|��U��R���u c�WP�%�aB�;�
��N�*�Rf� �y����`��^��$�>��{�� 9�V��������j+'��>��-��������F����S��p��l|S�\���@����9����l�Y��2{����h�(�E�A�"J�1q^EU����424L��j!��2��d�R��R�l� �{��,� #�~`��iO*��+��M� ,x>Ih1�%�K:�"M�x�$���l�L�5���������DK)�����|�9 q���a'�������=�~���� 3}F�<O���CEl������vjm ���2 e>�����t�Mz*K�=:2�����B���W� CMA_� $ie�y�|�
,.& ����x��1�����ET����?/�E2�7��5��C��}g�50 ��y�L�M��������O��������\I�Q�Ir�q����q`����?�BG�SX��C����sal�&2��~m���b�����q���?�����4��~H$og�KL������F�i�Q6������j$ �9��I��) �W����-�N�� �$��D���g1�G4i�-{��XB ��@��Ir-o1�Es���p�M����x �I����)5V�������U�/���~ z�I�&)������Q���d�n���G6-���&n��
��: ������/dNo��~��Ge��F��� ���� `����(Q�s�CSP��m��i'�s�� x����z�Z�j� ���� ����c�^�p�j�~O�&Bd�R�k�"::��o ��5���(���?��B��[���������z��06�f�v��������'�R S�����4j?t��[��G�A����d��h�-�� ��'�.��h�|@E �kf3��y������ &u������9I&�`���l��1����pJZ�� V���v*��c�>�E��lO����1�
�T��3 T�-M���p�����0, }��@���L��d{2p�c>��#��fX�9���5�48�M@��m���'���^ @\2X-g������xp1�i�6VX�&���Y (�-��}L�c��i6"��cU'j�*QTz;[Z�Q�h�����w������m9D�?�Z� �r������ �/Y��x4�%<�����b� -E�t�|6_�/�6�x���s�3ie�mW��4��Fc�%�WT�V�����t���'����U*�n��$
b[�O��m Bv�w�:tl�_�S��e8�O��V���i�q��yv�x����x>� @k�x�\�9(�;� 0~�d����}�Z��Tv
��#���
�T�� -h%�02 �� �/�y��#�������9�goS����k� W��8u���YW93L�� �c��`�}3�2W����6�l+ 2���P�mY��%.�������EM�Sh��#4�R�P�b�P�!lw�I�wg�;�4E� �:��uGR�xt��P(�$"�@ ����'%�����t� Y��m8��sv�%���G�%�q�Z\ 6�n�rV���ao=�����OdJ��\�?��N��x����#�D�c�T��+w�7#=�+� ��YLPW"F+5Q����e l|����!��s���Oq~�&��X���c��_�'9���Ujo�"��u9��$s<�t�Fe
M���t�Rd�e5<��ctI�Z��9 ��rp�� �tj^����38�Fc<���U����{��i�L��s�� O�p�W2 '������ ���q����|����I7"�� ���5���LS��[����/��Na\7�Z��Ke� �8j[�:dIIu'�\l&���Y" ���OWgs ��_$��t���2�CW���s$�r0����rH��%,�^��E3� ����c� (K�Q����2���\�����/�S����Ch��m��f���'I�� �-�8�� ?�$o�W���D ���p���:�[��x� �;h����`-�
�]�y����e��:gf�p�����.(KB#5��V�\k��/�s�V&��GR%6jn/�}Jw�?� s:�t����~�O�K���3SI�G� ������LU �i�u'��ld ]��aT�6���P^]p�7-kN���c7���Cs����J�����)�yQ�C�bC�^������8� M}�&g���S�������p�q������A����a��V����`CO��������r����""9�q~]<%/��)�.���0��K��k�+,�oK������-j� ���_A��E���cc�>���/5�|�EOW����?��"������X�(�U�C��u1�_��tK�Ssk��F�))]p������ �����_�1{�}O���% ��A1���#�8��-k��|������ai������b�� H:(p�P�X������o�3�� x�4�:��dkA�[�+������ *\����`�[�g`����6|
��!��=)��L�\�1xC�6Z�b&����S��l�}k=��l��j��u�t��Sq'���q
�mO
���u-W���
�Qk-����T����
�(���1sK���1p��5�H@�_s�g�C��:!�:[� n�!����ycQ�M'/��U����%�7o���S��W�������M�?����hd�nM��+ ��5��M��w;����3�O�d��n��Z���3�y>�j�mg7�l`����������L���=r�O9�I>�[�N�l^����B��4-b�w�G �x'�
o��^-=��m���;�z��-?���r=���e�
2L1$� |��4�o�'��?�m���Zm��IWk+a����_s�Vq �����b���Is�9�*��(���5b
���Yk����J��C�"�jt��f� ���� E�&��xb� � ��][�%�&tB���i&c��1e��U{Y9W*���<�������7��v���r�#���4A;=���g����{ � �G�_- � ���N~�Zn)�&�W+�[..= �R6H.f�!�a����|��Yv��������� *s�g�1uF��#�#�s �&Vw��(o�'t B��?����A�M�r*��t�"T���j���� ��$d� �}S���]�N�%�a�B�j�xaR!������T%P����g�$0���_�.�0���x��0G.Ef�P�;%����� (X6�;;���q��]������������ll�"!��9�g���/���A�)�� �%,S�� /v����f �P=@!� �K:��9�B���> �x��s��w:�M�l�G���HI�BsU��r���H�� Y�2��Y U������2��#�qI�gXj�D��� �o ���"�E�"�"��������3�4 ���M2���Q�+� F�!���M������� ����d�t@i���t�G ��e�~�D���x����Z���]�k�c��:����w���`���p��^����8��k��Ny��cO���4�F�u����
������������$M���e��
yTh�)hF���l]�eLO�K���J����s�#������%��|�l�������`#�� =@�c�?Q���������e�1��~���^W&b�A#�
n=^otXA'���� ������4u�P��TL��H�L�t����w� ��\��-�;�\����.�l9��^��ne1{�'t����/yn���2��w�,p��{�
Oz�0w�z�������@��# �'HY����X�Z��[�>�7��^���*eL����+|0 �� :����J]���6
= ���%�3{y����uh< LV�$�����\D�Yw
:�����N�rv6
:#u
Z[���-.b����C��n� �8�uj�;]�2����F%q�M��~8[���/��n���O�� ����gF*����5>�S��^��:�U�i�]OaTk4��f��3[>�� �bcCL�v��y~j��?CN1��vW)�
h���Q��`k��NA�ll�)������Qi���{����]{3o�FYNND��aR�/T�� &��Lt�\q���Qk�u%\5j=x���f&���8�/D�s��e�r�^�Pt2O�x��t�X�g���$��x�{6/%'�A��
[� �gT.s�UX.��\��'T�>j�6t�H��3�h��e�Y�8-�/�s��?G'��g������*����������:�R�$�j�6L�t�e�N����`�P�j7�0������� �J��<><8=�N~xs�*�b�)�2�&v�_b�8Kv�K U�o�~Kd'��+�m%��)g;@�?�p0F
�����W���{�����M�������'������wy����9;9|s��4�������l���p� 'i�Z�:���I��6�*3��rd��G� ��8����m��"�g���5����r^�7w�^��);s����������S�x� G����� rj@�� ~��K ���",N!O.�hD��%c�<<e��x�u�#�.,1�)�A2��D����u��O0�����������XE�>���T��-���JM��>Y /���0R� f,�Qe�0��5 -�Sk?���o��F���NNO���Z�y#�x^�* wy-��,������$�p�B=�r �G�����g������?#������4=����xJV e� �HH� �-���������� � �z�����O���T�vF���������c����� GtrQ^_�P������s��+�;�v�+~9���J���]G�+uYg��mE�F�����dy����� ��� �n�����m�����S��h�u������ �����v��&�p�l�:}Q�/ Q��7�����k�'.Y���v�n�q���6� i���^���Y���N��������������?w��z<���(�2��|z�U�����'�� �;2rJ1{�p��L������{f��X���zx�P�(�� 1�;��������g�����f)w �����g=qln#dJ����K��R��n������o"0��F��J��Wa"���<=�H��:h�n�;�l����tK�q2[��|�������v,�U�?��O8`�s����
��mf+[C�5��:�v�Fn"q�� T�5F�#*a������*\���CB�C��l�o0�L�����dd�)K�!s���������%�a 4S��<�t/���n�;�����<����},��Q��x��w6�mF;8�Xq( ���� A��9h<� � )bLj����:�w�&����%����1��N9t�s��������~�cw��~����"Tm:,p^��v���Ft�s#�TGG��^�,i�
��f��$T��Sv����aL���x�3�ny2����'��?�n���<�ZEB9��������6��!��v��_ �}bp���f��x��R"Z� ��0�~�;�� �
�
����W��t�>�� ���P���yc��h�n�� �}�VY�#�� �?x����Lf��U >� �����\� TW�?�4vU�.����^��7l�-�^On���� ���?�10���f�pH���5�a`[XPTz��]��9�j��|����e|-�L�r���n��� �������������=���o���)o���M�����y��c�5u����.�V�?[��]U7:���&��vS���bn��8��^���������������F����0�;�9o#��Y9��4����O]���f��������n ��[���<��y���7o~8x�� |/n��VCc�f�-
��Q����N�R�Z�N�[b��t??�"[0�G���F x�F���I���p?:��K�_J��n�Vm�� �����:R��eu����b�L�c��c���4���. r$
aDw�b\�^�G��ohK��T�AOH1nP�0����!K���|u��@�)��
�9��-��&�����;(�n�{�AkT�a������.Z�������/4��2�'�I �FS/��zP����M�&����]�B��,��W/����u�G�����)�)���4�������h���n�s� s,l�����:���E;=�q���^?|����
{ZW���|���O ����z����� 6
���.���1����|�p����&��Q�o��
n���zn~����;T��n���������L�I�����������������L�W9��3���z�����_��|Q?��l= ��S# r���5�W�|nWP�p�6�URw��'�o�����_����Z���j�]����v�#-�YaJ^���h� �iT����2yQk��V���Uj�v��i����m���;�J�V����n�����#����O���c��?j��:��]3��V�6��p����D�dO�� _l*<q�����A~��������fFH�k����������&�J��������:�������Y���10 n�'� ���N�[�4 ���I9��c8##
J�i��=G8P�� /uE����2����W\���K������*� b��'���� �F �~:�-}Ag~�$U���[G_����[�V*�A������K����T�<�j�^nt�*�i�i�M ��'��E2D
N��Iq�y�[������rK�jq% �E���3�#�(T���^&1��x\��a2B��j ��[�*�N��s9�8
��`6��$n���MV�SL�q������rp��
� %�}^�%�>�I�~�84J?{��$f�i�� �e����NYu^]���AhG��MT�4�\�UlU�z��H�Q�/�H����=�2� �-���f��>���_���G�&:[@ v [T�5��
D�P�*s���}��,���L@������O� �����t��j�1f/���r��0|T{v��1��� O�K{����`���`K���v��m?��Q���K��Dq ��?���K*OM�.�{��]���l�~��h.��h���w�^��"'_.�$�j$���g��� 3�Oa8 �~��f�^���
_
����� o5~���bFh�Q�����^���Ks����
f����a����b6Km��tx��v�
����C�J�9j�/Q5��-J���L���r��<��V�����7�T��l\|���.OX���
��� r%2� b3��b�N��c����y��*�
oC�_"�)^R�g(�N�1�}'�
.���:es6�@d�<�J)zBFg�%Y2��+J*8�%�������waB`��''��?�=��9��uz��`^<�e2]�b������j:��������h����xw��$!��E�,� �}��x ��M�b
����8{��9�-� <���VBt �;���+�4�7�z������r8��G#{�%�q^6� ������rv?+�������o����4��n�e�����}�Ywa�x*3un2N(�,%O�\%�eSs'5�P��z#�� �b�jj�H ��� � ���s���pS��e��7Y���}nGz4 ����,�R
g�C���,
��$����_���yO��FU�����b������2����N���U��z;���Q�t�M��K ���a+]��������2�I��6�Qq=V�� �Q}�T�i�Y�a#m�]�Q�l����7�
���i*�j�L��9�������-���@`��.���C�)w�����0��P(�/,q���u�V���=;q��y������y�r�U�j]?�Q��G��\�s�Q��'!����:�$$�'������ � ��J��|�(dL ����o$����D����!��z N ����� �49���T�c5��/��-��i*�m)���LN�<�P�'��5������:�E
K�� ��)a��_�l9��DO�kV����DDl�Rq�v�^n�\�/z&��z b>B��Vm�F�Z��q�f������������r
&� ��c��E�2�$b�.� �O �x)e��;�C�M �z���$C���m�:Z��������x�A���N�� s��5�����+���L&lb2�P"��t|9�$��Gm
�� �XO1\d��������B�2�<�8��� ���� ��`'����3�.�`��(\��(���#8$,`[� �(F_at�l�$�lAJ��A���4 �Q�[x�du��,�� \/Y�0+�z��=j6��-xwr�?x��'/�/�Y����Fo=�6����x�I���4#��K)�l�����:�� �0����9�n���H9��Vf���S�a�g�A������@
X /�I�2���A��:�`������a���,��@���l�5���� ��~�)�
(5u � Bk�bU�������S0( q�cx��h8R|�� Q��������K����R�G�r���Q)3�]������n�~h;z4r�L��*M�t����b��-=�����Z�����d����S(*��B�FY�Y� w�U�=��B7� ��KO��� ��tp���+v���S� +R��� V��^�z_�� �b�$����`�'��O�a�&�W�E �������K�)4��"�o����[���������Jirg�!j���!\��`i��EF50�����"��N@]������)�����������������Y��uhjS�
��#�����������@�6������(uI�H��1�X��XBJo$4������r���;���4~ �[Kt[�#��j��}�[��P��������8oC�/������[�����������n����nCt{���._g�x?���K10HM��[���+(��� p�k"[�xN��l�O�s��3]��Af���s D80`����&�X��A?M7������6�V{�n���F@$ ���@�? ��|�>C�yTi����uz�PXtYD�B�f�uZ�^���e����Y���>�2���{��e�0Z6�"��mk�4��s.K����)���bg����Z1�e;)34���Ei2(G������dL|��� ����t� MR��5L�:��^(��� ��*�������������hQ�/� C��o4�`���14;�n���_�����bL8�����d�� �u����k#���h5!s��,�e 940C��`��j*�6{=pOa^�2"� ��EL�\�� ��J�f�i��m �^���(���}�j#��9�J�b����E&]#V��=���7��.�p]Dl��\W�7R�q���F�� E�����}"�j��Jn�fQ<$sX��w���c�]�� ���D��uY���y���qk������yIBc�W��z����"f���0���
��cLO�.� W���EO��U�d� ~����:>�\%���T�d� '�] �� }y��$��*Z�����cX�]j�AP��p�����*;���I� �u��i7yEj�( Y\2��Xq�����Q�`vR��%r�E���s��-��S� ��>�S���e-|F��J�����^�Q� W|��a�g��u�,J{�q�:U-��o�O�����p0���A�
��]�m\�8��q�]�
���� k��8HMar;� �Wo0SOK������������������G3�&�QL�S��^xw"�o0��?����_�c���������J`�gQ�8_���x�����wEZ� w#{!�}�M�8��.t�U�Q�
� ����L�G��Z��5����G%8��Dc�j��8�*��V�KE���j���Y+���� �C�hEX�������#�!�5� �>��� _7���#�*W
���F��O� ��q`�Q��1��^�"�r0z���N�j5=��� �����~#4��/�O �@�5,TT�9o�P���y{�@�tCbh"Hz�����+���z���p���[��1����BLb��>�f:v,�� �L�%~P��y�J��1t� 6�n_�J�N��v> ��g��+ �lO�b��b�#Y�O5;g-�(y
^n�R����t��U�~�b�b����� :�Ob���i��6%����>����Z�.�UQ�<��@��{����;�l�{r��V��L��f}G*��TQD�]��U�f'���V�Ivg���^[����T�h���n�`����P���jZ8�2����?�w��e�LR7�!���p���SINY������G�����%�Z����m)���$x����x� -���r���[jC4����ji����"����� n�!o�#���1S�������2a���3�B#/�(�Fq��j�?����Un�������x�o���,���A�Gg���1n�B���@.�+�Y�T����)^�2���g��
��u[M}��(�\����|f�C7we��.�C�����]U�H������VU��wfrt�0��#�,9q������
wW +o VH���1A���2lV��Z�� ���6�����"��+r97W5T�n�2^.Y#��ieW������n��y?M�������&.�J�]%"J��K���%J~�P�][�}tJ����k`��@�� V���l ����i���#��s�2��f)����e��������F�M� \��A�|@m������ll[+�P���-��� �1�����Q_x3���������i1����-}G�V��x���R_�����>����U'��D\���m�[J��Y�|P,����[}p�$1�tX�V�l��.���� �z��F W���m��v��� $��$�u2����s��u��F)�vb��H�h�;��tV*��X���d��%�S������� ���E����[c��dlE���c�L�
f|��n�R|�����0�6d��cV����\@�AjV������� #"����m6��#���j{�U������9���A��6�M�����m ��l���3��3�L��D T����Ydr5����n4����jr<>y����2��u�~��z}f�<o=�;.`�
�q}�|��!o�fPOp����I �y����W �H1_�s,)a�/�Ff���&����Qj{w�� ��sF�� �f��Gby���m���o�u� ��Gqzf��������'E���3�k��n�f�ua38���k� A���|�����v�����w�D�w�q��O�{ �t�h.�!���/~Pnh�p6�(���` ����F&F�-� �z�:��6��e�{������}��m�Lxy�{Z��j�x��h�8�7���m�c����{��q>����x��
��] B$G����`xL�S:��rR�� >6T����?������9k ��qm%�K��
c�$1�����1 _ ���'^\b��� �;.Gx�
����n ��6��F�����tH��E �
�D�D/�� R+R�����.�8z��L����p����}`�J c{��y�H� ��Qj"2����T�p�@��?��0��^������++���������9���� 7J ���;��Qf�m�� �hn����a�������~������q�[8�-zYk�B�����*��`�V�'�@� ���[��5F+W�5�e� ���� ��'0�:'�(_=��@6/rK��h���w6Dr�(:M��7����w���B��W��
�*^�� *������ �ZR�~]��N�xu+��f��w��7��2�� �x �������
&e1�.������i� Z-c��+��
"\��U�!��n�v0�����q7�D$��j:�ue�J�Y����|�18��k��L�����4��`��>��g��m������9�u�H�1��sL�����iKK��
^�������4���0�T��S��j7nY���[~��-c�f��T�N����� �*#V����4F��� ���nD&(���\� �� �����$ �,Ax� <�vaN�l�q9K���<$�-xm��/+{��?�����F����C���?���K������<4z�-�������������f6��9�W�nr��?�j;��RX#�� �2+�d������P����\�;t�)�]��H�3�
]���^����& �;kZ��#�2���F6�6�
�%��$� 'ef�������v&*��C���u@d�h��W-�kE�j�T��{�bXF�a~n�?�����:�����d*��]��?���e��z�� �+�-;�����W;;
=�r�<���
��n�� k
��^f2�{o�L��:�:[�Bg����������2���0���BO��z�Y�5�������^��}���% �t�yo �y<�&������� ��#�7����o�9f.z���7v\{���B�*o���J��*&���RT���_8� N���d+���� .�q�� m��_��_'�u ��M����$��fupV��Zg��z����[�~��(�l����&� �� N�����=��������OrM�H�ED [����[��k��@� ��"�Z ��k�[���T���Z��� �C>��}��q�\��|����7� [�������C��V��Z�y����@! #�V�c
}��Q���F������Z��4`w��z���K�^k��~{����
D�+���/��Q�.������z�v������j��n� ��8|[����A���{d_�� ��}���G��Z�U��k� ���v���v������I����M��'�y 2�?�O��~>$��s�_y>���G�� � ��\�~�U��J�I��T�j�a����zq}��^\&������#I=�D�F�����8����f�]8��w��������q�����W%
�F��D(d�����/�z=O���.���d4e'��2��������W��}�8�V���h��
���
�?[-�-�{�n���*,z�R�
q�t
W_UW���Hf���,� e!'��t�09���\���� ����lv�q|tz�w�������)�~N o������+�~��V������_�_G �j=��a�'�V��� @�WD�L���� �b����Z���$����s|x�j�:����-�r�����z���8Sd�
��Z����.[�J�[���f }�2���Z#g�����|q���d�'�^t(�����u�Sm�;��# ~`M�CiB&����$R F��)�y�\Y�d( ��(�����(�O����jaH�;X;�Y��/*��sq�Dl����R�i�b��^������,��M����n�����a��2��T�v�T.���1#q�U���"��#��M��������A�l����V��\^{���w�"&�F��zX]"+-1m�y��2q��L4BF���.���-��l���� ��� �������K�p�{�1��)6���l��Qg3�
k;�P.�4
�'���K���v�d55�|�����Zw��Qt������}�+�[�E?� Vw��e��MoN�����9� �����B2���zJH�v����M�r�Pg�P����k� �Kr�`��sM�@� \1���C)�o_d�$����Ypnzgfphz��EA�6���92�9~�4��j���!�*Z���Q���a�c�� ���`��o&n�p�=
�6=X������8?-=e�<�#J� ���l�����N
�X;MTIf�[�-��5S��b���_/B���5b|k��%�v��n
�z��f1^��Q�We��v�r�J �&&�U ���Q�xR\b^��� ��R���(z�����"|�Z�'�s t�������$�Fa�p���o�`
X��������L�<�C89��DqM.�>wm��HB6m�K2oF�g�T���i��s_��T�kaL�����FV&��@��.�N ^7N �0la��*������f mKG�e �T�a�� �A� ����A���
a������ei�����%�D�uN����"���vSdq����Z��{l~�����o�7
w��^����E��[!��f� ��n��� B�B���� J��$����-Z2���`�-fJA ��*��MDe�2���`�������s�O>��R�&s���#i�; w��[o��~(��(����
3|�p D������
���F��bj�d��S��d�����2���F�f�5��4�P�t��)�#���kt�E��V;h ������E�#��"�)QH��;7C�e��!�I/ ������S�}�Y�VW�����j\�� �z}4hV*�nc��kD�[��YW���
n��1���D�i���V])��A#�VE�����AR��n+L��8
��r�����z_�[t��j��a�� Pr^�Q�g��WPD\yF3a**PPh� �!\�U�
��7�:U�#u�-�5���fB���p����k4����)���r�u����������W._������ �]�Z����M�6��*�v�k�NK�������_�m��6 ���ca��v�Ct�L�j���|W\��cA�
I�`|��b���V! �U� ujJ�d,J��C6)��TX�+�
�$DU�7�mQ�9� ��
� ��x�p�� "�Qa�
{.���!)N3��i�$��>M-D�{���
!�N�*���AU�,,Iu��NY?��>���Kxo��1x�����e a:JF�.3"N�J����I��
h7f�o��Lc���@;�w��7z�������
{.G��e�O��$�A����l�3h���
?� L�.vI��We3����j���_+�h]
�? h/��a[�]����w�����k�@Y#�F������7G���Go��_ � �1 ��?[��k�Es]hVqk��Y� �o��q����v�#
K z
����0���F�$?�K?������2��~ �v�x�1T6L,zxZ �P��fS,&L�&�>�\�D9���ag�%��x�a�YB�As��,��j� ��K}����Y��.!�w�kx(��8�0a07 ��_��m����������{��5��3t�� ��%�ol���z=���
P��������k���a��v�L����[#[����/�]h8�3l6�v��{��]��ekq�w������i3�s�\�%��� ���V�]W_N�>���^R��)�d���Y�!p�.�(H!������s9Y ����o@����� 4S^��I��f ���1� ����x�&��� 3��� ��)��D#���iB��� z}��=��
�i��i �6�@��g�Ai�� �[��;(/��|5��|��lf�%��^����f��I��\B����j2��c����`�t����V�F*(-��/]�,CW6S:��3�t��}g��� )�S�����|�3c<7�do/�c��]���%��4��:�^�9e��"�*4�/g������������F/g�I2XF������l�)����������q�_r�����E�������Pro�c9�<JIgyI+�F�4�<%D��A&�����4����-���W� ��-����� *����0���U���W��l���W�"�.%���i�LT���f����=�n�����H��o<�� ��� ?�B�C��Co� ����og��l�2���<����$ �����&�oS��
Py�d�I�ou ��� v�m�+��2�k��F��Ri��Z��]��]{���WaI����){>#����m��-�W�^�\�i�[xP��<-
�,�`��|�.����.Fl7�&�i* "%*�u,uD]��|��g�?p�a��� o�L�N� �/S��;�N[yS�������������"}Y��~���W����:������lnc�����B3 �YAm[��� �m��2V�j�Q��
7��<�Vs<�"Y�4:9~�r)�}g�&�����3��U.��x����O�N��6�9,����5k�n�Y�6�V�[+7�5�wv��Z>�s�wt��� �%���� &������
}Y����W �V�� ���Z�����p+� ��� g&�'�� n�v0�����P��'��H��7~%,��>�qo�-F\
�j3G���{���o��S�^�t*��d�Fl�<5���"�=����� g�c��9O�� ���1�Q��g�%�������m��s�?��o>����#���W6O����7u q��i�3���*�M��� 7�6�UWg6�+�0�TU�n��X��X�q�����d2 �&Q�X�� �[b�iK��n�(x�/� s 7`�`��4�L�� �F�"]`��\�`� #��0#F����e�G����n�����9�wsx<"�����(s�b���k���a�n\]�H�� ����}��_2>���� ��i_<�r&Z8 I������%�p�$����&����i�����t�3���-�1�-���?.Pn /*nn$���cL��$|��������� p��"���
�\���.V:O8QF�����G��~�b�d�^}����,M�0b��
�"���0P� �C�o�E ��X�1i,�wS�,L?D�o���*8,g���{�����������R� x�P6��c<Y!#�� 1����&��D���L �} �5�>��Xj��M��% �� ����qG�}O��V �p��2�����> g0" "n���*��vnL�� �q���=���d�B2�-q�{da*�5��u���sr � %�A7*���5�r&ri�Z1#�#�}L�D|�J��k3,��h5���tig��e�#�$:��2��w���6�����U�h# �s!$S��^@�|�Y����~�C:�UU�ar�k���k/e���k�Y�,���J�Z t�-:;�k}���
�2��o�?�F����3��\�������>A������%�.��U]�� �=C~R
��������od������
�aA<���b9��$�>bo*�Gp�L�l�{��� `��AS���x�,Q_�N
�.^n=]�uL���B�Z3�a?�����>q��7��!����I��m�k�����G@�� ��H�}%�}d�v�.��2��:���r����F{ �% �1q(�^�n&/I�E6�p2Os��'�Z�El��n p�. ��x������,�n����O�����/������g%�U������:��V�-���B�&����?6�c�]�h���|"� w�5 ��N �m! �<Z�����x��%� \���ePu�{F����edf9��0-f��j���m�y� H�4T�#T�����U� ���+��� DZ�Z� / + ;L��2�Pr�x���������P<�r�XhF����,�`z�1�������x��7.���<����'��/#��(������b�R�~�������gJ�a���6"�svb���[�H�h��#�j�j����,�w���-�LJ7� ���E��]����}}��?���d��C��Tv!��HG �P�W�'"D��������F ���LOW�H&F7�n��m�������M�
qa���w�h����G'/+������_N�� � �dhA��p���7y6C���|��*�Z��v*�u��
���,h���#� �c��FhK�+I �"�~,���7�J�;~F 4(����/�SJ�>�B��:������Uo6�e�G���.ffq�x��zn�n(�� \���9s]4��?�(���U��r����9@� ��"VC������vO�0�������?���1mC 1d�p�z�F����K��?��'�VA���hC�g�C(���(]�d���c� ( H�T9)�s�����4�����7'�F|4*��*vn�hS��@?\���cb�������������$��!;6�^���.1�����t,�<��D�'R���1O�g �$ �{�X���>;�JujUFM�l>��lo0$��:�r
gye���!a H*m�8B�q���q���
��pb ��� ��UQwPj8[��YL������<&>����%�#k��N�� �L�g� ��_S+� o9E'��i �iT��r�"caC.��c|7�xj q���9�Tp�R�3h�Ln�J��c��=����oe$Rn�������.�:�S����|���� Cs�L3����
��)[�IA�u��SJ~�74`Cxs�I��PVq�;��u��f�~�
��ps�7"*��/�%�V ������L������1��Q��N������|}@f�BD��l�y����'`���x� ��� �\�v����++8��l{���@�������a�_�7�����m��5x���3u;2���d��I2 {7 ��5� ���J ��&\���zG����^���vkY�<� S�<6(A��V�h, �i����C&�������� ���A}�d~L�t�`�k�{M��kz�t�`G/RQ�%��W���&A�":��.S���f�
���.�,� ���x�LtD 5"5L��
@�gx��&����~����Y��t� ��k�4���_M�S���]��������g�d���h�ndQ�D�N@�� 9�� �
7�cG{,gX��}�5I�RZ,V���������kR�U�z��>JM B ��
�O��&������&����3>p����sc�t�����E2AN�hu�g:S�}�4��| Q+�� f �� ]2��s�(�L�_�8Jp0H�8Ss6��05 +��8�����(������;���/<��L�~�� �ER�,��.JbS�%?���pu='+�i�$������W�.6u��?��C�T����*N�U�@�(������t�7��% 5�g!���$|R�� �ZMs��A��.Q�<����j��m���D{o��;:�#��|)�{�������G�d�rr�?8��������&���������T9 �m�D�1�����2���e2��'�w5� �m�0J��#��$'�Ga>5���qqKCK�V����*���oJz��A�{ ���)���}��u�zbQ|Bn�tK�r�������x����_<� Lzh��s������� u<[�;�0?25Y�9
��a����F����
G�j �: >�\:J��Ktlch �����R��lEW��fS�/���� �Qpj�-��+K0tyA� r���
d!��b|~!R�=j��g�p��������+�R��.��9� �^t[�X�� $ 8����'�r�Rg��5an� �Q���2���/�`G*�M�cN s�����Em[��[���Zw���3Z=�0��d����Du��QlG�k&h+6����������S���
�9�W�*p���3U��4i3�5�a�20/��������'l�)R'�
l��"��9���sv���4�%F����9.&� �?�/�����eO���(�����5%P��a>K�������x�S��D�rW:��W���s��R�&�K� ��������b�b����Vh���{Q� m[��\��G�� �b�Q@�>36~0�]g
)��477��k�qeF� 2r�c�W�����"��
F�5/�C�����'����v��^��:����� �;����_�RK��O ���SOE�r6S�t6�����zNg�wR�1I%cN��O���PYw� C���� �E�3�������tCH��5�� id�6:�T�O���]5���� p�&��3�6_��W���������J����� X ��"�riK���#��d��M%��B ���Sy1�:��y��S ����� ���Z�i����lb��� �$�^k*����n�74��XK�p5� sZ&���e r��v�+�
;�����B5
�,��h�Sp|q
k����P �#Mx�����'�{qT�������!�#-5W�2��d|�7 Di1�@i�"[2pH�4q��e;��>\&����y;���v�����9o���� '�����'����a�f��n��`��%��P2��B��y����,�]fK�� ����1�~N ��#����E�G����X����\����W�E�#��~gGO>r��5��H�������Oa��' ������v��a���Q��=��
Q����K���S�U�]�����v�����j�gi�B��*C������������{��}T�C�{���w�����~��
�SRQ$������k�@x��9�X&�&����9<~��������_�^�$!���+3g �Mx�Sz�!#��=/�,xo�b�(�X��&M�NL]��g�A��.d� �'B2F-n�J���KXv����6�Ki,�Hbs!=ml}M'Y(�M���DIWV&)�����1�"�g�`��t '�=*��l��Sm�S$N��
�F�o@��M&��� ��������������W���d�y2r�v� �)����(U�}��A�J�(|��^�A�����v��L{4`���!1��( )���W�o2�R��s�`P($=O�E����^!"++��7$�r�xjj2 ����ls�Y=[R��[M�lf2�Y�m�p�n0]v��>�c�������&G�� �xm� /&�fX�J�S>M��XGsN���\�a*�#��\|����G4�&�)��y��,�����K���h��-Z��k�u�Y<����j&����F
` ���&�0g:��T��za�=8B ��L(qu�`b����������U���Hl���-5��9������-���N�!���x��p���� t�`8�/��+��{���d9��4�G��Y�w�3�D� t����V���F�����@~�;��hfY�o�D)+� ^���N\�S�F%�iv�|L�����5A)<�R3�f ��<C��%�c�w���3]e�����8MW���p����&����1��l���om-S��4����) 7��@�p��b|�B�$d�M�����FTO����#]g>VP�KV���!��A}�������=V�7@fe��lqt�3l[TN�<
(�_���� �~�����=
�W3��)]Q��>������O���* ���
��b�O�/�c
k����/j ��x�<1����X��Frr�Z�����$�$d G���������������*��������f��X�q���z� L������������-�u��Ck6��3=���
x��4�������QH�f�E/d�g��)��2���gPwOQ��|c���M|���,� �� �~�%�"
X2�-�T���Qa��~wk8��'���4��'3�8���0���C��]�x�� {Zzg9�}��� El@=|�w���:�<AN�����5�l1>'E�j�V3�����I:� fb1�/� �X�-�m��p,O�:�e��� ��x}^�y��V�� ����u�^��m�!K�,2O�7�����s��c�"S����Cds��Uw�=;��[\W�
��RP�/]�]T�]!��J����B��V�����YS=���W7�A� n�\��B����J�[@�7�,������yu84���6��c�������Z��^��V��Mo�l�h��3�� J9p������'>�����m��_���y����i �8�z����2��2�V�]�mJ���Y������/YIF;���!]P�#Ya��?�k��(c�1�<���`w�\��i���I�X�M����j�<�F"+�A��t��+%�g�iu�y $��])w�6�NT#�!&��f�����-W�w}�!iG�7d�(� +��h���h��f4g�r�Z��� �������{V�tk+01 ND� f���A�*������q!����=H��f��8b{"�_��c������vq.]���)�%����2~(�� ��� �<�4Kw+���U���
��q��]�>*�i���)i��t����.��s�)����S���Y�|� f��9D���� ' �)J�%I�����|;�%���r|��������m����=���<R�D��[�������� �����oR�i��[�
� ��� 4�#�2by���YO��VM|�A ���������%+k�`����c�)�qL
��d�
��Z�Ri���^�G!����k� �I7�U�����m�����%��N�F��G������z�E ����NCws;��\/\LCHu|�4��3 9�}r���7�r�
��
��K �]�z�{�M��@�$� ��V�2�Be�����J D����}v 7
��|���2�%����8?!^� �o��� Y4����of�I�z�A�I�+ �hh+�O�����c��sH`�G#�B��$� K��6�u$�w�?��
�78(����L���GYVn��Z��q�c t �� � s��U��2��c.GS��hj�����)b&�8�@���{? �o?:*����A�~O�E8���Y��o�2c8���\�g�H ��%R�j*8 E�����G��c� k��v�z������i �����X��;���rE��G�*��lG��2s��U� >�/�dO9OP�%0�- �~}�i� n�_�;<~y�?��B/a�<x�&���M�1���4�e���� �ZSNv�T��Y��l�}����i�G��
��|� `[����0��gbI�����!�0��=�<3x��9 ��[9��=� ��G%F���� �@A^`NIC~;r.��v ���d ��]sV���# �`��!x0)q�^athc�Wsb+�����k��b����'Z���/�4�����5� �����*�� ���f�@�r |�4�W��dw��Q^ �z$e���p�=/[� y�1E �h�X 9��i��#���e���L�3�VR�%�Y�&�"��*b����la]|�f@������P6 ��C���^�R�R����A�f�yT���|���"� sc�;uIk6j1��L���� �svq8�����x� �����B{EYM �%����P+��M �����-!�������r�Sb���������Zgq\ ����j�^��J�w�����w���Y.��7��I���:���k�����.�!V�*����A���n�3���<�y��C!D�$� z���E���G�U���c�A�/�G<�o�8p��|pF]78k��� o4�F�^n�:
{���=�+E���� �o.�>��v� OON�yyZZ�����M���80>�%�������)�KZ�z/_� � �4���.��eM��8Af��\�/a�$v w��v�[�TU��N�
�q��� ��;�=((�N����d��`p)�b��='�"�QJWs�=v���g�. ���#a����S6s��2��$^��d��� %��_��r*��J�S�������~�\�(|�6�ng4�v;�J2 �����R��9 U\&$*��a(�8��iE��{���2���h��C��p�� �bx��@<�%��<� �R8��e@��t��j�V
�4\Mb2b������GsX����/azs�:*�"S���;"F?f���
��w��q �7
����K�J7�l5�U`E�[ Yf ;vO���I��LE���Z�I��$=�4�� ����� m������ZM:��s���j���UuEd��l�rJ�
ui�h����������#��@p>1n/O�RLW>I��N/�wi�,Z�� �3�k�������3���i�l����>q���mp0;�`.��r>O E�\z� *x�|���s��i�[�xj{g�/$S���`�8\�`�q^�s��P9Z�Z�^��H ��Q�����������t2�������@�Z���Ev�O ������'R��Y t�B�4�')L}�x������
������~���$#���`1�+i�Lf.T"����� ��W�B/8������|}�$� B^������h��� 5�:�aS��2�r��D��,� �6�O�Z�\��,H�U����2 �z������v��� U�m��� w��~Fa��&�>I-�`� ���,�:��?�Zn��O���K���2����������X�t�A�#���l��bW�� na;
@1G��?���c�O� �t0�X^��i��~}�{���2_��3t����J�-O��%)�~�9�l�u�.��Qc�� ��m:$�.I2{D������.���4&�f�;�B
�h��t6E\2S��� 3����-�
i>�/��k�&,u(��q�����7���$ ]h�H ��t�� �{�T�#�;�� �&�d �������=��LI�����}�-�f^���`S�3����1
z�V�Rv��A�\�3L}�v�)S�38�.�\�N,���~�`� �@�]�(F��s�� �6��g ��VS22�N����ER�"�D� �N.�REH��t�[�,yk���4i�����Y Uu~��B��:�,�{c���("Z �J ���P�B2G��!���I#��$�'��yf+`85������������Uz��������������S������l�zz�lY��1[~n���7�/O�ye\�����A��q7z}���H��9��;VN�A-z���pD��T�Y������9<����������z� �����I���V��o"&�N�oqx<�����-���|����D�
j��H�lg4[
+<I~�� ���E"y���] �0v��W�����`�PM�zey���p�/g��� Hm���rJ���.����%MEn�K�� �� �e!����=� k� H��3vE�!�����y ���%��D����Nl��-��������?E���-[&��?� T���{� ����� M��E��m�� �F��"<����q�.����=;�G �.QM�H�g���� s�)&��y�3;u����pl��Ri� �+�n��?���S�x ��z�>���$ &������k��hxU������V��B������Z����S��k��_��V �G���8 V)� �+���/��Q�.1��E�Uk�Zm�GV�f��i�;��m ����6� ��G��g��p���~���uZ5��f�A��l�������>��4�?��Tx���!s���x*,��-�g�Q��d`!��5o�:��{��a�R�%�Vk�%����I+��Dx�y�c��/o_"Do����w���J�x��A1�`�r�F�Z�\���
���
��F�c�D��dr���fm����d�{l��e`���xH���&Q����5
�kn}c� �Q��F���6��A���v������<�>�l����
�']����ve�
�*p�a,e �'�i�!��^#
�N����
����O�y IR�������->����b��w��8wO�Ah2�J��&�5�,oP��.�h>_����`(��V�`J���D�o)e�d���f�����P��X-�{9��� Q��g���
�*r���|o����j���O=�E;��3�gs�W
/�,yGG4������T�~� �-��mR@��k�3T�R��.�Fc�A?��[�5n���V�������e+����L��H���> ����t�����r��*����U�q6Z���O��?���SG�X�6�m�3e�n5:�V���(�x��<�=c x����y����d�p�&��A�' �M?b
A��0;���e1��n��M�r��>������X5F��Lr�i��%VI� &p�����J��N���U*��`���
��uA���g)Hv����%���F�V�j�s|� � ��O�-^
;'KtuF��9[���_������U�6�o^���g���Rfo�4`�8�����[,��l��TZ����y�df�B�(���,`,���������������G���d�$3kZ����U�+����5`�=��+0Y�GH�J�c��g�c�fCF�������o��m�T�s4�rAm�S2��� �.h�du��
�C����8�]�LRC9Z#��Rw��t���#��%�l4��fU���3�� _�/���.�iK��,�� �d�h_�a_tj9r��-�,]�&�dA����]6���Q��w�Y��������D:�BJ��i�o�������KJ��ve {�3�|�q$?\���7�~��pmh-�P��s��Q=R��bv��T � �����k
�Y��x`�}��qjA� �,E������������� @ck��q������?%n!'p��r��%��N�\����!O
I��Nr��6��eM� �F{����=k�YyS�y-Q�o��M������-���]4[,i�2x9��D����� �)�� �����D:=X�n ���E�/�!zF Nb��|�%����� ��%���i���n���W<�) .D;��n� ��hw��nM�����?
��0O��#Y��������]
�T�k1?'���;D���p>���|f���~�������� ����G&�����D���f��v����{��{���j�ra�c�>��3�S�(��1�2���0�\�V�1�Y��&��/�� S�H3�����8��cSMPo �N�� Bh32�u�|Q�-�P�7
����,���|~����vR8��^TF� ar"���x>��!����;U���a��l�����KTQ���r�� �Om�
�w �\
\��d2S�$Wsd V�u�!��Y< b�Q� �oBs'`���8�z���k?��RF�|c����������7���c����~�Q�V�G� �7����D�O;�Wo�{��o�U!��@L^^.�����F�����jU~C+���Y��M�r��'�q����s�Zt�zxe��2m�X�R�e_�FYm2n�^�S���/l�I��0SI���n��1��|����B�A��n� M� -�/}����3�}�_�s"��N <h��F�������b�f*P�F���n�����6g�8�s"paZ���P�����E�2�������W�������=N�����Ls�Z��?��O��+����O8�eO#p��}'�o#����I��������
n�%T�fh�� e��IG! �x���!�qo��r�w2���Tm? ������
Rg4_�#Y$dn���V"^����D��v2������n�V��j�^`?�DG�����4���(�+���A���A@��M��}��' �'�73�`3�|1;��8G.fG/���T�������r�1���v���DYu �Rs
@Z|�4!��O-P5mg�h-�����8���VSlc ��c?��6���p��$�!b�*V�Dc�G���,�����<g��SS "�O%� ��'�B���t����d'��a~�f>h7�r�0'&� �}/S�������
)�u���,� M)���������*���F&0���N���Y�q. �lX!�_1r�rE�AY����%AK6�}n�����c��$%��
8s�
` �?��By+��0 ��
�DQj�(% h��u���[��8 ����|}r�v����+���+���`q� �:<V|iL�(I`�Q�r����� ��-#���N����ula� �?�0�~w �����������+>c�5���Uk�����yo�ey����m\��/� '����������P �5<O;�^��{�����w�� �T`�{����[��mI��P��0��B��O8��+�}�{=eOH��;����1h��v���c�b7�=#�k��0���!RnS���A���8�R=�K������1%�6 ��b�>9uNC����� "��'h�2�`��p���T] �a �q�P��Sb�_2�`�
A�YGM9-�(Z���x���M��Q�j���U��m�� ��Q&97� V�3���ST��|y/��|Zz*�c��b��q�[F/>mr5S���X���u�o2���67tx��=����� �����k�f����n��Ag���Yfe�=�ja+����>~�Z�]���+i�����N����{��E���J%e�
��V1�]�)��c �u�*o��V
,S� =*�)[S9�].�U���f;
g�g��S�����C�0o1���n�aj�k^��9\D��ZBh��
@�X�1[5���i��+D��T* �,� ���Zk�Z�z�^S
�O1:��&a��
�`M�\3���k��U�����P��/��D��Kw� ��xP����> ��R�;>���x���y�����T�^H��#*���������Qi0�D��$�L�ie`��u�ah����@�S�P�]5�|�����1FK�������"X`�H�Tp["���������/O-]wz�r���z�;��8k[��.��#����Rz ���]D/$��'���_���)?UI��]<��H�I�H@}L �"-j�Hm����q%� _@qR��|4 u2;���3�#
g&��dZ 51��X�%E����rH�a��J��jIX�I�����*rp� U`@�j]=�e�Q*x����`����4t�����������t�xY���Q^p�
!{���� 9�i�Q j�R�w2�o3d��sOg��6��@q�����R$ ����;��-V�4����SifIO�����V��o���AbL��q7�s>� �cz�3� 1���E���r����Bz��!�X%�t�-J!���,�3��8�+&?*g�� *{�6�*���f�����
��H��2c2�����L������:S��H� ���*X1�U�6q��L2��&,�����^~�6�r����� �+t����d���O��3�R�M������J�`����]yULs��f���������RY��� ����\������rG�h5h�_�A��l����������|��T�1��q \���^��l��������% L %�E:�J*;K�L��s��� >J����;��S0��O� ==��V��/�&��������A�� �qw�������C�������O(�� �H���w7��[��\�����{�f��k�2*�M���|�5��d��UBfWo
�:Kt`����� n����yE��c-Z�5`�1� �����%�y�]/L��\��'J �D�=�������(]a�=����)�;�2��y�!V+�f S���7�)Nk�92N�
���Y����}���KX� m�� 5��j ���j�0�;��E����F��q2�4!e�JrI|X��R &�c���G���+��h�y�f�
p8���h�|!��j
KD=��@n��H;z{zx������W�����������9!� ���N��iz�:�)a������3�$/�5�kQ��F�� o/�?:�WS����]�K$��S�C�lg%��Y*�����B9W>V���E z���r�� �d(3(���?����
-x@���P�qJ`9F�qk�����P��u9�)�y�_��>� 0=���.���k� W����d3;ChpL�5���C�I�� �1��.����7�B �1,&��a�:;o���0��1c�����(5@_��[���Tr������4��g��A%��<��a��@td��=�&�d���}9���8���a���n�F(��+�*e�u�$R�\�*{���d����3��sVq�l��q{����?)�����?��7id��fa� '��J|��iq� �Z��B
���S� /�� N��~8�'oW�g�bw_��������
;R�6��Z�n��pC�}Bn���C���QT���$��W���O�?
1v2Jp���%W
��w���7��A�G�`3 KE �X-MNa{!�,U4*���i �(��!�9��$���A9���k����E2H���r�#� _�J c�= ��� Ny� s4R%��D��ZDI����8�VN�"������w C�|�9����]�����H��|Vb���\��������4>��k�~z��6�G��C��~C�[;��
_&h�J�jiG��9���>0� �7������� i�A�����8Y,�l���'5;{�o{� ��3���M�= G
���]��\��Z�'D�O���Q���aBr�h�.�<�� x1GG*�${��3��r��FQ�ADE��\+�?� ����(00ff�L��%�������=�Q�\����<��snw ��k����� �� ���9��[�{��;��������l5�������b����vvdG�.��S~� m_��Z��n Mk�OI<?]�������jNOMnrR��?x�7!
�R��n�.a�Z�K��\�>�4l�M6�{k>�l^#��z �O;C�iPg��g:��V�����Lk+I�O:�������N��P�9�K<�t� X 2���������� �o~><y��Pq =nj� �����u�o�x��6�?i� ���H�\,G�d����,$�� v ��=����U��y�8/L>>���q34a������ E������}�x����Ii�^��{B)<.-����JXx�b������kfz � y ��HB�=��k� n�����N�^�/Hg�����P-8W���G {w����Z��l�n������&����M�U���D�������#� �3k�+L��: ���h&6� �Dr��������������U*�V�wV��l��G���?
T�4��J\��8�n4_,k��������
���� �Q9��Q�e�3JM����Pe!�!E}�U��2 jyq� �����*W"�������0I���o��*]�.�EZ"#���&��o S�b/P6!q�����-�������rx��@�����rT��xz=[��pGzz���k��vN��j�4�(cG�| Mf����?8>=B�y�� ���0*�zA5�jTzs�7� �}W���'5�k�� }-p��`'��'.K����j��a�;O� ���)�������.fE:/��t��uk��E ���k{I�Ai�,r�- �I��[���J�l�������=���y�)�� ��;3��-Y '������Z+�*�/�,,S�\��=
|�^C%[������� �J�G};�n���� S ��]��Rm7:E1Q�����)��Y<��{��F��N�������[t��e7�u�I��=����b^_������v��m��Tdw��q
q��1Y����W�'�pJ����z��lM
��U��������h{�cWg�1��9��+�2�eP��.J�t�O�`�De�4���la�������8�.�mi:4~rI�� ���7Go����N�}LAh}wrz��S�?�
������� ���� w�b����B�� Js����YEq&(f��N��n�����������)JpI�������a�f���T�a�FP-����������7��]��"E�P��o��S���~N��)�L�E��w1����||�XD?f^u�F�.6:��7�Z�]�����|u6*��s60%�8c�zxp��@'p�89���k�c��{rn�I�|l Hi�q�AvHyT�-�����[�t���{���s�E<OV=s
>)k�����K�dA�����G�Y
� �4�� ����[��q��a���tuY�W�Np����� �s�����+��p�m�yb:J/
��4�����s�'���/�����
�~�4���<����� �e��'�tsN����?X'&K��/�K� � m�D �Zw��uA���}^P�9�b��NcWb��� jp���� ��1|)�I��X�v�����}���P���2�����~���c��W����(<�-�!y�z��"6O�T�6�6��� ?���v���)�� �T����U��X�[-V;���[�������� ���b����e+���
��&�m����:I~e!iKyh��J�z ���I"4�#�YV��?���d�#L1\|9���r�r�E[Fo�J�3���c���0����N]��i� ��;IJ����� �I:^`���1������S��i=K�K�����A4���6HM7�����T�^ ��|xY��IB 9�D�%�D�I�)�Wo��� BQf�9�}������y�A������������� �'��������� �U���q����|��h~��8o����j������nv��k��TS�f��g.L���Zg�n�z��Y�RiW��n���� ��_l�J�&��������z��@� F�8w���&yj7$��.�v��M)@�� �I ����flS6+P)U�(�6/A�~��-���R� ,�����n��-�~ZP#�S�n>�Z�����Sc������8 �56q�����$f��4��r����g ��������8�x\5�%��$���,��"��=�:����Y
�nA3� y!��~h4�0_a��O?_�H9~��x3zuu1��`��
C��y��X�1�����r ��
�jM�{��M���7�� ��
����}<�)� �q5 ./\)����uVf������2����J�_d`b.y��������L�qj ��Q���[M�o �a���Y��x�����.s��h����f��+�����_�=�M�Sd J��9��0�K���y� ��
`�.�I���1����
FlV��f���!p`sY����}�~B���~/�3�������}�D&&� ,�D�\� �3�����N'�W ���A���muL'�� ��7$c�����G��3�>k�O�^W '���i �"2]�j������D���r���|�t�S���vvP�_ �}�j����7�?#}�����=��9��1�q"���������nW ��qR �������v� ��&d���?�� �qv(��g�����#|�?�Y�v\�s
����6>����<�DI�1�<����*������d�
US�M��,> =2�����p�9w��X�G�rpQ�N`P�� K����k9��e��g&�/x>��Az�p`� �����%<[����hQc�0KWgi��
� �"+�j��
�����g�j E��([Fb����TlX' ���wF��~��Y7��I-\�g��?��3�\��i������)�O�B
x����l�!Z�M���i ���B.�"r�I��R@\��o����� ��ZMA����h@0��4Yr7�N +�DM8� ��� F��l$U�9�������S��_����������%H���q��zu����7������?9=8>��} mp��X���
�xs���&��[x�4"|Y�j��t����j�Lu<�O��<�>Ol*�����A:���\�)>���Hfu�Z C�c�G!����z$��ZP�p� |lf�k $
���H �U��� ���4�%'�B9�-&�����dq��v [��\H�x<�J �f������D^A�1�����"(�QqC� `{jR���5��Bn�w�h��-
��AZ������f }�[
;���h����E~��E,n� ������'�@�-��G �2L��1pYd�&����S���i�`YC� ��(7��Rc��tV&�i��Y ����p�}��4�>&�DI �i��Da|�B�T��������IU��Kt��3�"b*�f0�?# wKq�����a3��p��,uC��J�?���J�� ';�,!J��������9 ��t����/BG���$sp�F.�@���p
�(!M��)0��l���zc����
� $T� �u� HF�x�NK����8`�4e}���7���&�x���Y���k��� S���@ r����
����������H�"�hH]B�0�h( n�������t:,�^�.�*�����?"N�9 sB1j�etl�U�[�.>�K:���<�+�B���
��{�������"�4k�@wY�?� �����Z/��zB{�~���IED- �"� � �g���+�XE�!����J<$Mp��*[��%[�l�Td�������xLA&�l�h� O�|�;�lq?"~����l�H� V�>[+
xwm����F�B���2V�G�1����Z��8UN�aG�f��3^����f����9��&J���� 9�U$ �JT���3�Q������d��
V��z�Y����|���#��9��qy�G�$����\ �������%U�E(e��]&T��� Ng��������d�,� /��f��q�*�#���\2Ia�6�D�� ���W���}�7���C)� ��D�_�����Is��V���FFH�)C)#!�7(rM�R���}z�@xrAATz��9������d&�qfu��j7����9�m��� K8���N�����lm����B�2f�vr�d� ~��F��N�]�9� �U����6�a7��x��&N����� &H�K^1�)��� ����n�]���m� l���o���;�#�9 a�,Mo"�|���Si�[������ Nv%�SwUi��rS����D���KF�%/��cv���r�-�;8� S�hd$�2��S��Z� ���@���=n-b�N��������
��N{�]���m��8�F4��j�@����k�\��L
�4��f����Xw��Q�|EE�_�V�(P8& DX2b��C�����z�IB�� ��.V|� /��+;Zu�
P+�O$��Sa��`1>3x����6*5��
V�PN)�[����As�By3Fc
����8�$>�6_ �� Qzt���|��L�����AO�� n(��K��0��}H&�q���Z�P#u��*:�������O�@�3�P[� ����I���xy� :|�5���2��� q��
���<� �\�#�]�u��F��y=���o���,"�=�?��[R���n�� ���mJkg����E���,l\��j���f��=2L~ {g�� �_�%?� ��Z������_>��� ���SdKw :D: ]t ��X�D�1�t ���%����
��������J �1��2!��*� �X�R�o�i���F����<�r5y�2kHdm��Fz�r��
dx����t ���3�������,�Kk� k4���9: pw���>9�u5I/���8�C� t?�$�0�e�|H��g����~K�%e��9�� �-��� ������;m�� �u������"93�/����(��-�&v�������N��[k����wPI�Y�g�4�[f%��`�p> �5�� ����$�1��u���;:� ���s6W�#��*�,�B���-Q������ d����lZBS�dB���/�Dmz7�iY�>�7��W ��|�VYZ�����5�������f�T9�%H���E��?��RA�����;���v����^ww�������G"�� #�d���p#�8tG���}T�C������"Rd�����9u[dH����J� ���1�n�����u������F�[�vTW���e��|,�H�� -����cLG
��#��?�T��p�0P�_��C���u��r���e�If� ����?���[�����ru�TA���5v�=����� [�t��{b�������[���yirw� �ln9��I��Q��f��uf�=m�4��E�4@;N��#�������N��n{��V\rzk' T`D#���#�G��4A���4�BN���W�s���'���O�'G?�������� ��y��f����_��#�������|l\TL"�g4p��s���r�-���������;�^��m�P �8)��#U�D�4�n~�eZ����{�p�)�8�tx��=^��P����8gu��]��(G�o�������yQ"�A���(S �3�?�o�]+u��9w�f���� �M�Xx�{�g:C�
��7����#/&]� �&��t5���3�= ����z������} *r�� M;$��IR"z) �����!�� gT'������ d���G�����"]qL��D?S��
��GLNvp|;/��S��'��-d=&�b��_>#������Nc����'s�IU�@1^[��j�9+d�:0%&�zB�i�9=�O��G����
�^���p�M%7����KZ���6����,��{�r�E}�����'�+��@H�|i����?�[` &E��@���oQu���d#�~!��tmY�����3Y�q�+�O]AU�s4�8�$�/��S��G�i2&�[W��HK����E;L�K�X�j�K,x�F �EGN��^v2= U_Q�Y�.�yq�G�U��\�����q�@���y�H���Q��C�S-�z� �'�@&��a`|.~���T�/���e�B~�E�
�����!�4�G+�
q�����Q�^� !sWa ����jyn;�q'�n��Z��Mq'~�����z�����Z��pZH�����-�� "���I�g�<!���T�B�R�9�!(��R���x:07 ���/_�;��Q-K=�4���S���>���np��rz��<�vA@���`��q_���p&�sR�5�"N9L ����5��A ����!|���>l�!�W�_7(� ��)�J��!��w�2 �������N�R���@]�SR�
��B�B�c��B��z����e�2�@�@�������>@H�� �P�s�B���
3����8 �"�3
+6�F��=��,^��F j��4�M��q
�&��i�Z` p�d?AQh2�}�P5��O����L��aX@DW����H��9�4��w�*� I�u@����<6���,����h���/�W�-%F����������H�������t����[ �X(3'�����;���-��Hz�)��|��+�j ���a�����k ~q���\�2�9����� U���p��vM��{���
����E�G��v4�/k���
7�{�n;�R�sc�I�7�T�^��P�E �C�V� �9?skI�q�}� �F�? ����N���H}�i� ��nnd��P���o��`�>��E������vV ��\��-��GS`?���us�(���5�)�ja
H����<�2R��.��1�������T���}�'������v3)R� �h�������8��S-E����Y�dz�4�ky��vQ��
� �q<0hDtPH�_Q��I��)G�A�����AV�tY�J���s��a1� ,w�f�M*a�+�5B���y��O���7R�c���$����
c�Nfp���7��N%� �e�{�.Q ��omza
;�uR����m�Gb ����2��gk� "�
����kI`�[�N�4��Cd��-���q�TX�����XOj����`;�q|��J�z���%��~ ���E� O�`/ #g������o v���]����R��z�
����(��iO�����gl���M���������!B���p� aT��s�!�xj���u�����sq� X�
l�'������*�~V��� x�����r����.�l�V^'�c=<�[V��k�zb� �HP��1�8�\���lA�H�-�))r(*]�0J(��,�Z���3*������C���D�l�3�g:����P����� E���L��GiI^�4��ooln��\7�@�yoLw�CKmw���l�^Go���f��
1"VL�`���I�L�A������}�W9��)��6g���P�$ V�;�� iW�T l�-uo�Q���!���n7�����������]8*�-eY�;'��*��%�
������q2���ZV�U9�F��T��r�d_2��+����i9�&������6� '�������Z�q-@���k;g8I�����?���Z���hg�������;!��q�����cR�� /��f[��1h}��/�
����������Povke�G����Z�\WBN���1I ��_��sder s�o&�\����nf�/U}���jS�Q��S�%SG��T��]��&7����*3����'�TgM����{����#�|
��^��#�FjT���,(��F ����Q���soF��K���[`t��
�3�� F��-1:�'�6�{�����F��v�{�0:y��ct�m�� ����%{����w��{w���� �btZO��`t�� N��V��S %Y�����Bz�3N��Z�N7��8�$)-��9�X�1>��Q�
�S�����{���i|Z��������{����e�9���o g�U
�.�!;���G} �%m�M��{��&�d�����3����{�q�a>r�A�|</9 �2��cY���E� �����,��y*��{��``��
f��``���-``��c3 ������m��f�00f����),�``�n ����n��a`�u`# ���00a�500{����m���#N���{��l�y���8�d4�.�J�^�\oUm0� �)��}o�����gL�����������>����U+�;m�0��S�����j�bBC��v.��[� w������^��� F��X�����Sqt����;��������f� k:��B`v��{�����3u~Q�W8}p4Y)���rO�2�x��$+ 7�a|O-���Z[�"^��*�q����y��\W�������h�������7���� �Of�_<��{�U�wV]��bL�)$i �n>M�]k@��J��[�a����=_���v�Og���=�����]k����|Fx�����TvU��D�t{�8����\
� &�wuSS<�+��:�*QA�}TE>U @�=����M��E����b5�+���_SE�q�h�9�c� ��w�J�����"G�'rz������*`�����&l��g��k�L[��J{����;���������I@gFQ��Pt!�JP ��;����4��N�
�8<���Qors�t r��O�� ����m%kD����T��CkUE���Q��q
�]��T 9\|��C��P��y`��M#/'��\����=�;m]"���L�;l�cH:��xZ� �#����k�(���M%�~��#���=��1��<�^��B �/����G���/�BG4)���[_��i�@(l�[�i`�=Y�w���pF��� E���o���l-(8#=,%+���5�Mrx@��]�{0�����,+6�C�2�yC�</N�3���:��^]� �i5���SC�_�����;H�������x5��U&�
��q^��)�y���� ��F��\�j$��"I1�m�� X��Dl�)G&�[�P����/���,��g*|�-� .b�@!�
_ �C������;d����y��������%����8s��~DUj�iH�S�s7dT A�t�0�on����<�%gy�_�������/��?���2���s�|yIR~|~N�~ 'J�kw�p�< f=�r61��^����^��� �%�+ ���+���ny)��Ql|��lO4!E��q(�>�<'o���V��. Cqk��r@'��Q������{As����������|�zX�
����@�T;�z���D����jp��U���� �@~|vcKr���&3��w>���~!�$��C��&� Z ?qS�����vLf ���r��l��K6F�����'p�����3��D�W�R���� ��u�����}l�[/7������g!� nTD�#w]3�$!~���b{3�|��`�l������������ ��K*����� 7z]��>I�Q�W��
����S�]����=�������{ Ah��� ��dT!�Y � N ��iVt��z',)�dd� �5�w��V��P��)l�iB �Dl.|����l���<F
�h��B�;%lJ����w?t s�{���
�T���������9���K���?�c����>��������h�7}�z������2qw$���v=�\� v3O�v�C��n����#YO��D�Z���Kn�/��Is>��T1� ��jFVW_1�6�@��f����Yj
�J����v�h��Pn9l�e�H������9��$���i����}��va��F�Z+�?�L��m{�>�n���2����KUo�R5�v�T�y����r7#YS�O q��0v���0p��� ��/3#~f�F����T �;� �V/���
� �6o��h�k� ���=���6B=�K��A������NK��Ca0�@����,Y^%������q2m(�
�{d����t `L8���d4� �t���&6hJWI4;��T��V���%����q�q��v���,?��g 6��b1r�A���9��~1�������uk��pcUx�P� �X=#�v�|�6�s�������1����lj?0[D ~Db����� ���Go� ��7��Oa�G?�tJ�n��_N�9�������6=�k�"��V�����#������� ���"^ ���q����c.�h �_,��A������NWeG�3�7�;�?��.� ����R���D��O"8�W���1��
E�R�f ���(J*��/f/�x,��� ����Q�*�}u��sM��� ��'�t���� e z�� g�?����� �o��R����>����(�M��������_!�����n$�r�C#E ��h?���`�f6�����?���� )���I�#v[��#�b�����j6��r����k6�� ��%Ge%��tD.\~�'E�#IV��s�`dE ��N�� 9��,P�!\Z!0WA��v59��m2��SL��%9����������8��1�1���k��Ye�}���� �(F
����L����@<���J�F����7lRv&�`�*� P�c��Sl��b��Ql7E��UQ0����/����][\�,��Q��,~w$�V����'�� �i6�o��N�P�X�C�Rt���F������!o�������������~~�G�� � 9t1�&�p���1�] ��%.W��u=����ny���$,O �K������?(a7m�N
6nO9x���������&�����a2�~99� �� ��_�t��oGo��.�a�,�#���������V�&��1- I��##U�i� /+����0�LSR�L�~(�����>W9\�>m_�i���1��Bm��p�s�4KP8���+��E[6Jt���V���< ��� �M.W#�����;���v����F�Y�)������*6*Y���t(��0�q?���!
���^Q����KmW�-������ZR�
!���f�(������r=�'w��" Ou�Pw�[�����(��k��%��;��"V 2A�y1�pX�A� %e$g ���&�@2M�h��j8���CL�]��l�D��m4j-�_���Q��tS���%�g`�d�+Ut�L��L�F��P� ���`q����6���� ML��!eIG�a�#�r��z��~��I�)��K) �5 >-���1��3����������s}D�2|��~�1����z�����|~~'�c�8��� �{�f���rK ��5�r�U�H��;& � ��i,��% ���X����M�l �t;�F����� ���2���>�0��]����r���=��[��� �w��-.LO�N p�5iv�Ya$�}�/��������t�{�m����I�H�/S:�M���Kj/�����[�(4F$�� ����(&Q�FL@$\��cWb���������-�j�W?���p�=��BM�Y~���A�g2������L._�)b��C�PWwu�����+pz��<ML�d�H��{m�8>LD���
���%� ��V����x�mqV��19�m-�2)hsy�d\�p���L��|������H��,� y3��4:N�7C��?EKk*�#b=�5�;����0�J��2F�0��332d
�C��T �i�����7�n���|���i, ��\&QM l�� �����G+v8`1�Rqk�����2P�6
�1�qQ�AJ�a��$U�n��T����-��5<��H���W*�F���L]M��2 *�����1��)��B= ��@ee��I�]f`5� �1KB&�=��"����S��R�-�%u�"�2�����` ��J�� ��� ��b6q��G}���K����
`]���" ��[`M���� �g�\��� lu����F���� ��U��� �D�0����S��j@��R�^�$d���� �<�'#�' �����?�c#�<��\z�g�b] �� �5�6�@ UMu�p�v�[r��c: ��r4��N�F��,���i,. X$��g$�`���O�������;(��E;|��n���&�� L-�h�F���Du���i ��[�
���8�����C��������~����������R��xw:8�� ?��
�L|� ,A]�2f���%4x�7$fU��eJQ@�U����Q :��9�6�_�Nt���~ v���M)� �
A_Z .)�Q ��i2U<�T��YAMU �r�;��
��A �E�������*��s��%�F�cf�M��^ E|� �j��[X����{�7 ����j�*J$��G
��"N#��0���I<���@��������/mXVB0�F��f���
�+Y����� �1f� V��mr]Q_�u`����� ��Q�����G�wv}�� ���t���%~S�N�� �_r���^A����� ���iS��Y����N(e����'�!������b�O��D�f��@�b���=�o��3L���Q�� ������Dj�K��-�N�������-O����_��h�w��)tz<|7 �U���C"�h�� �C�"��c���woY�j�l*
������x�U'|{I�����Bj\u��U�O�;X4�2@�`hr���U��� Z1 �������V�w�8�?[-���h�Ea��V �O���� <y���T�=z� |u������M�������e ��\� L�`~]R��<�# xD�8�z��p<k�@���E�� >T�#.![F�Qu�R ��B�
{Rt��������?�S���������QY_����t�� ���pvI lD����;Q��i�[��}���i�|� �U9��=�����^V�?[ /O�cu���f�t���*#'r�����H��BV��a���N������&4tc�c�D�2�sjL?������v��P�{_�u�� �e1~���a�
����S����O�/b�k��h_��8>�����������$�"5 V���� 0n�� ��<��} �:� ���DK� yOP�({������<����
�,�f���x���pa�������_��Z5<�$I(5����� �|��(��������p�6���h�
-�wa���XMA���qv-Xfe��qk�O��fr�q� ��S�W�}���J|�RW�Y@%�\
�I] ���5*KX�) {x�
��<�XO��� 3 o�j*�;�!� ������5ed �';J����o0��A��
�%9�3ev�<P�������� f
��R2�Q&wq��yn ����m���^�,x�A���9�����)s�GI%
��]nUkm ��3� U�f��t\d #�9��e}b.�g���M�'�*��E�&�E<����s��c���� �%"�L�{�BwgC��Fw�C�t&�����V���N������`F[p���
��JXR�L�6��>�,��l3y��.�� ��<�������:���Q!zsI�Gn�K����XL2��������D4�X
��DUars�?��������w�@�����zG��u_����� �I�����R�����t��0��3��A!\*����:����<�faV�!#}0��~�� R�
j���HZc��i���v��w�Rx&f�6�C_^BtZ�|������7����hS�_��:K��3vY�� ������ Z�
N�Y��e�}koGK��dv^zu��/?6���� � �i�Hb�$zC��]Od ����������h"
�X�#����}p�?�
V�mw6�J� ��Sv��,z
�E �\��v � �HJ 4M�s��� SL���l�������<`5� z�U��@z����?���1c�@gph]����C�V��%�=�� ����H�(Q�r6^��j&���81i������JT�O��"&Y
%@mrm�.����G��� �����0j���M��O�1
��o�*B^A�Ld�'���>�l�qA P�a��~g��n�&Q$����Z����g0y�5P
��$B�2N�.��<wWfA]j2R� < 5��M{��.l���t��� Iv�8���ep�+��]O ���F/�vDa��eDfBf1�'��RnO�W*�{�pE��KvQ"��� ��@`�<�`%�1�����k�Kf���U����u
O���li�����i�z6m�S~�5���duUa*F�..|�}M�h��� ������f� �W� ��2���n�8�:��� W# �OD(jh�W
~�3�fr�:C�_3���a��� �������.���F�h��j�CZ�a���8����?mt���Z�w���;�"���(*������C#��M�9�eX��������Q<*�A���� ^t���R�"�htL�������&`Sn
*�w���m���y���_����.���m�%�Z�Wn���@?xG�� 2��G���=�
��2V� V���Y!7�B���m������8��b�zrA I:��#�b�zO �N8V%m���?�n����9� ��']p�m9������.g"{���3J,<�*�)��F�bKQ9?`�w^�2(?2m�����1����������y��9y���KeH������=�����d�{ &z�3���3�� !����B�~��G��� {��9 w�0�h��
d�]�����GQq $��.�~oh??���:�.����Z����7[6�����N�����_f H��dH��[bbfe���8��
�}b��
0+TV��&� ��Qq9O\��K�f)
"*�jtW-�?t��}X�>:�� ����j!J����PlK�#
k+��[S� �U/�"�f �o���1�������&���\a��O��r[��>+�bP�[��rR�� �`N�Gwj� ���JAi����lQ9n �l.����
9 ���#��>�� ��S���&�~����(�N�o"������� �} �cq��r�A�bA����;!�s���5 l�^�7h�a��:)��"�j`����
'�*?Y�Cq��EX9����i(�(�)�@S�U���Te3�q� �������^��[�������=����j&���(���r -���s)���n� �vZ����M�&��$����{3Z��^ �/d""�x���zDZ[� �8��( ���-�Q��.��(KB�}'���Lq�op,� �m�D�s�����-����k 37� ��y��8�r��.(��d�IEAg�/K 7�� ��0#c�����\���� � Z�h3��t/���{�*n�NK��];���*�� 4+1�a�1��`���;5:Q�� f<!�y������g��-�]�@������R�x���������<��K�M&��d��� �U�$W���`Y}OA)�fL�.���o��AmSZ*��hz6�� ��[Z�T3�`8l������mc"�Nk!�������]�����5�q��|������|{�9�u����Mn�?��m��'�Y<�Jb��h�<��r��� �0���Iw����2���������@3�.��"����`<X�L��9��{ S��1��k��1 \Y�)���1q!���'� �Eu2�R* L��� ��[7� �d�K�`F� y�
�(k���f #��)��
Pe�Lg���Z��x��7�=uS��x=����o��b] ��`qGy��y���S�ZSi>���H�|��4N�6��r�H+���/�s<��y�bh�Mc�P����G�LB��_`� �7�<�X<a�3��Kbw�H\��v� ����������C��U��F�]n5=?c�gZ�K���� �;(Y��W������!�O �[�|��$l�n���Wo�����Y)a��M9��b���� #�/4:�{�
�;�J�� R� ^�?�|Ma.��O pC���]5��=%t����� ��$�(������������� n%��2x ^q/���*�H�Q C� q���^�,/�&|v�\��%���*�I�&S��x��$��Tc��o"��J��P�x_ � ����c�3���}� �� F��� ����h��553���6��Z�+��d���W���I�3Y_G�,Sf�
&A�S��B�����B�p� �n,$L�B��������8C�*�P����p�R"c����������mNv�����d0 �D�l�2��U!�@�L@�SgQ�)����>�. 4J%�8��� �sK����?z���/���~|�������c�[ ^{b���# 4�i(�1�������@�|`R���h I����Y��&�Kt�����q��;��e����:�F���ihu����]��5k� 5)L�� �"Y�����$
�t �^^�������N�i��v�m�/q�5Kk�|����>4��h'si����B��Z������(�78�4@9��n��jV
��L�Zo���l��Yw��tnN����IB�� �
5�!��P �������N�d���R�J��i�v������,�2!� �{^� /)�� |a�yq���l!��[�1���$��5 ��{7R|H��#�w� 4Bp�����yd"Jn&C1/��LS~e��ld�E� �]�����w2^��l"L��97��
����l�>S�h��Hp
$8�]�(��3�K��Cl���Rj�B�Jw}q
5EfR���n�cyg��`W���+�8A������FF�%��/Y1���=�`����g�m����RUw��#��Wm���2����ny3Y�c%Ew����v�!��j9�kA� �/#�aw\�,_����y���#�����5���p���(����:�RL�'�Y2�+�� SVcm�4�
pn��������� D����&a �WL��{��y|�����>���
��r���6����Ps��J�8���Tv�&)O�g
3��Av�q�� �s���}�eEG��1�0�"z����lc��N�+-)k�| �������5*�J�������VT���*�s| ��=�2��
����� %��1@��!����y�����U�,n,c�����cQ�o^�:�o�c�~����� 1��j�r���#�e��M`�n��.9T����[�;��N���U�N�S��� �H���3=8��6���pt��d9��X*
N��]lC�$��_����aOx~��8�4ZhGO}��#|?�I6�Jq����� \���A?;[���d`�KX���e�g"=���M�]
��<�� ����k��v�i~��m�\�f%�����$�C�Cl5���34����X�D�����6�P�Te �?��TSr&9�e����I�t�����S�H p6CYR�\o �Z����av��h��mb�v6G�=���S'}�]�~��y�Q3� } �Z��� #p��G�":���#�� � �%��j�j��|4����N�{V�6*���q�����&��]���@m���z����FGYu��x{<�����������G�=�0�� n��2�B�q�S�nH8�����'1G�H�7�����{�h��/Z����{��7��7 �,v�R�^|{������ Q�3n!�;���;����#Q| ���W�����s��< �C2��:����Y�"W���v)�x�i��������tbJ��#�E���,>6����>Z�N��}V��u��yo�z3�k����>��/c ���2�n���>�,���u��K� ��ssH��Y�-,D` ���?~:<>��Q���l=�-�J2�l���*�a�vX��pIm���L�� ���?B�=�z�2��g�T&I E���ZA�a3 �T8���DRC��������U|�2���2i?�J�!&�|h��\cM����Q�+p�J ffl!*sZ:1Q���*��]��� ��a���1���i&'�\��V��.�{ZS��n��c��� �]K�0�i2Q����{U}<�O����??.��V�x'Y+di�4Y���dv�I���q���������{U����e�!���}� *Z�J��y.���W�Jq�sr+Z��$v5��9I�?&�f����wo���N���;���6�6�������pa��X*��j
OVg )|��R�5���� ��#Uw=eu1����y��.
�^��2#!� �% D�Ha���`�q�/W�t�x��D\��?�d`a*�N�5��Z�n����/�9z ������� ��� %9���9�^�f�'�'e��hp� �4�u� �x�JmQ�����l W � ��� �� F?[,��^g�N �:�t���@W=�v���Sz��$A t�d��$��,=`� 8 �}�mdaB8&/iV������Rt�O����!��% $��"�d_�,^r���l5*GY������d������h����&���� ��b���9�Vg��4��g��"���@�z �TA2Y�2yE�-�':�]��:�����U�n�^�����Ei� b��7.I ���p�:6��,�EJ����T�Ac�t*��h��4G�T�u�Qp�����������V����Y������h�X��!��������;�F OC�0�#4��oY_T2�P�U%���\�� 4�����"����w�_U��So��� ^�������'��Q2�5���P_�A,C�� ����pE��C��@��R��{�}������O~x��� /�EZ��>_�� ��Le=e,�n!:B��� ��N��O�� ����$*U������n9*�>�����:�~���k��v�1��� e�� �(�Y�l�F��O�N�@|�5;>x��a%���W����������P�4�k���2.��tP�}�.�pG�]�vw#�,�����h�� L �Jq��Iu7:8��K�5]�m���=w�Z�^����XyZ2��w�B}���j*�RE5��?3c7�tt�v x
�z0����?5EK���(�jm���
�
��; �T/u�����l������~|+� ��d�uz�Zz�����y)�v��w�~�*�����KNf��Y<��{��F'�(� =5-<���-�A�sO�7�ImmO���b�����V��Lm��` �B ��L�k��s����O^��r��S4m���~�*� ����w(��:|)��� ���w�����Y������T:P�������0�<�\Q��tw�h���=���s�����$�ac�^������`"p�|���� )�`���V�e���W?���=(������ ]����*�?A1 ��e��N�>N5O��5 �C^O�:��:��x�*����^�����SFQ� ��sV �bF� �l�/��f|^A�m�=]]��>lo6�e�q����KA���p�F����K#��4������ ����?MI��p'�'v���u��Q+`�hC��3X������ ���^0m-�a���������<Y�;��� m ���IJ�0����876i���%v!����4��SnK h�Fk����E�G�O^f�t1x�|���s��� *���z����;���k7;I��i4{�j,w�M5��rS��>y����%�>N���e�n�,�)�Nt��r����g�~�>0.&����?Q�&l`'�C1�@��v��@� �.�NW���(�Dth��>� �p1w�%� ��� y O&�^�Y��rk���j�J�q����F����]��������;Nc��C�$9���Dl����=�u\p�P��AM��<�6����8��'�g�C�%x�X&6u��gf9�"L0��t' �<~"��L�1� �M*NbF1'�7�#V/ R�)������Fp)d]��q��
|6���Y�a�'� � �"7���\�T(C'��Sv�(\bq@wM+��t�)� n]�9X�>���,�7��� V���w���^����0���9@���[��L�XJ� n�= }m������G�|-� �`(L�+t�K)��<
t7���C��-�d��K3^�)O�Y2�]iW����� � q����]� G�%����}��S� �%���0�%yy���AO��x���as�q�8�ej"'u� j���Ei��������4��vtf��) |����m�3k~�����fn�w �����v��4rD����
4��3%_Fv8Y�
������I5L�Z�6�����L�����4�9<�)���Y�@x�=�D���)~
��-�����^�t ����6?9g��J�j ������8A��J �������r�[O��4_ q�i%��>_��N R �����^MYn��f���;� \��� ���b<B�:�]�yo��K����T�z�r� "?��8R:������y&��������k��Ku��~���� 7����S I��+TVY(�Z�JLU��L6��� ���x{��m���|�<�k��#3�6����hW�%�K����m�A����/�������>e!���w`��*�3K����Y� �=Ku!��d�����L �U�
�^�R��� R
��1�f:�~�Q��X���d*��g�^�1�t�d�&��
��4������%Wd�O�S����4��|���&M������+��&�5����G���5l�I�z7��U�<� ����
.���
����C�j�SnU�� �q�5���_(�$�8#����G��u�']7�-0�S���� 7-$��x} UM���������8���R�>�MBtQoe�m��V]#�����7���dp|g����Jp��5%��I�n��d��}��\/d��{���p���Q(�]s[�Q� ���:�����������_�� .�����;55p �!/��"m6hf�������E�b��K��l�t����n�1���J%nw�g��f~����/U�����J�a��F�Qn�j
`������N&G��U9���n�GvoT:L���7C�!��������Mz }C��� ��J�MC�y�x+�%7����(�� ���x����Z:� S�V�g��������+���:���# �����. 5s�a�}�f y�S!�9l�[`�=��8�u w K��/g�Z<�*�����6���K��� &S:s��J�J�l�@m!e��� 6qc�:Uo����r���@?�^�L��s�����{������{9IJ�$a�F���%z��C�D��~����uAt�����S�n1# ��J�E>>���3���3 <M�d�\.�$���=������N��l�
B&�=I���Z�@^�X�j#�Y�]U���V �`L ��w�'�
���
83(A������������.�Yl��5
$N�#AC3���O���HV���1�d�% �vwK�����/���N.�\Q EH+z��)5�4&�,��
'��3��K�W u�����?� ����x}��W�� �5�7�[nz��n�Wnu�M5��6Lq�goPN/H�f0 #�%
Pt�4�&T��rO�\^�o�7q �ab��� �q��l��&p�9������� �$N�j�_�#]��2&� |9�_������w������ � jd���p���0Q���� D ����N�����F��[ �p#�Nr[l�1����z�R������c�U�� wE��� <�a��A�m�V����z��o�S��2���Tz8La"���
����/����?�
����W�r�1nIq���=;����qs����% UF�b�f��=>�G ~De���e�P���$����V ? ��a�
��J�@zwM�NL���GGo_ �G������r6�����i����m�]Z�{��]�;����=w������WcY�|i6�Z�t/;��AI����Y����H^���2���T+��X/NIQ�{�L�!JCt��s����
���bL�Pg =�KIbl]�W����q f !LM�S���)D)���{i���&9�\#6�"��AE�m�-�
�l�!� [�D ��p����2#
���"��
]�CM�?�I��^$rO�6!�5l��1�7������+��_"����%r�l ��fD;��z.��9|������:Q��L����R����jWk| �s"\a����q�~#r���,��KN�30 ]z�q ���(O?�)����?����������s�,�)�n��(���(�� =����dRv��C*.X��:�?�Z<���La�u}|�% �\��X�57�J�N�������b���
9=wP(��9sM����� =��"�Cr�g7�H`����
�6���e|���h����Rq}Y��)x�
r�3Vx\� >�Y3�U�z�qU���F�R��Z��j��V�=�2E���Q:P�0X�q�����z��m �'�
��Sq��2�����/G^}���l��q�����h 8��*�~rB`KyN��C��/��Y�]�a�����x��o��e�f������[ +`�l�b:���`@x����t�' �B�c_I��w�F9�� ���s����������o?%1BV��L��j}���L*��o�F�J\�� ?���F�/��Z�^�����Tkm��/�o�;���""BW>E[_�O�]b���V��f���4:�j��nw ��8���uo��~��}���'������ ��k�V���9~� ��������${����bS9��k����������������v9~�V.O��nsX�4z�f��\k���Y�H�oM�q����4��J)MTUc�U$!��vuP =������`|%U�cL�K�+T*�DC�
]�'���a2��`f�Y�!I��
��g��Z��'3�'���^{D��b�&1"%L����C�2Th�\��h/0��c�FTj��n�M���z�xJ�yoKc ^���7c)�d�{��!���$�� bt9{��k���A�b1M B%���Tf�t-LG=H����P�3�KW���W�uE ~��a�Z ���
�C�[BP�dqi�I0J��#���Ck������"S��k!�9�����o��Z�St���� �����Q��j���� �3�j� *��z ��>$�M-���s����xp�(�7@��c5aw� �E��=�q9�F���H��C���f�����g���o��uLV�n'P!.$��IJY������������QIz������`�Cug 'Mu�-����0����UJ��\J ~;P��dp1 ����
\,@��P)A ��U�b��ET��P�����p�*�� fI��|q��%���^� ��$iu���J�w5 C5d���K���� ��1���%s ��5\R �1)
7��xI���*�r�G����T 8�gT�`�a�CFw�!/, �M��x<4�f��7{��}t���`p{W(R?��P-h 0��H��p�Wl �� ���&��3����������a�>���1U���O� 8�.1 " )�� �3��3�c`�?��=�1�!�
jbB�� 8546xw6c&5%�h����(�MA}��A�{X��R|% ��+�,�Q"��N�.��Q���S���d��Dm�;��Pe���������+�e�9O,�h��Gd��f��p���u [{ ��I6�� s�=�y7�Z�i �.�s~�����dy� ��FMA�� ��h�������� d'��EL��Q����?�Zw�1or��4Q�o�j����O�����N���W[�Z����w�J������j�����G��g��p���~6���p�
������I~^�GV�� 6��E�Z ��}�K��z��L��� ��:�lC�E��a|}�O^����p��JV���WRg6BqW2h����H�T��8C��
r�/��KC��a2[a����I]�A�_$��"�l�����7��2nL�g��O@�?_qv����g�%zh����L����>�KT���~�wh����5M~#\��yb>o��N�����
���u ���6E[n�4%*�D��
�\B�,�h����pWj�P��:F�|�e��m>���|���h�"Z�|�M�+:]�����K�xe��q�q�l�h ?���AP��C���x����|�&|������%A����&��U��� �Y/� �i� ����>���XR���}��%SeNX���z������������?_��|�����������?_��|����%����
� � 
Re: [HACKERS] WIP: Aggregation push-down
On Fri, Dec 22, 2017 at 10:43 AM, Antonin Houska <ah@cybertec.at> wrote:
Michael Paquier <michael.paquier@gmail.com> wrote:
On Sat, Nov 4, 2017 at 12:33 AM, Antonin Houska <ah@cybertec.at> wrote:
I'm not about to add any other features now. Implementation of the missing
parts (see the TODO comments in the code) is the next step. But what I'd
appreciate most is a feedback on the design. Thanks.I am getting a conflict after applying patch 5 but this did not get
any reviews so moved to next CF with waiting on author as status.Attached is the next version.
I've been a bit confused for a while about what this patch is trying
to do, so I spent some time today looking at it to try to figure it
out. There's a lot I don't understand yet, but it seems like the
general idea is to build, potentially for each relation in the join
tree, not only the regular list of paths but also a list of "grouped"
paths. If the pre-grouped path wins, then we can get a final path
that looks like Finalize Aggregate -> Some Joins -> Partial Aggregate
-> Maybe Some More Joins -> Base Table Scan. In some cases the patch
seems to replace that uppermost Finalize Aggregate with a Result node.
translate_expression_to_rels() looks unsafe. Equivalence members are
known to be equal under the semantics of the relevant operator class,
but that doesn't mean that one can be freely substituted for another.
For example:
rhaas=# create table one (a numeric);
CREATE TABLE
rhaas=# create table two (a numeric);
CREATE TABLE
rhaas=# insert into one values ('0');
INSERT 0 1
rhaas=# insert into two values ('0.00');
INSERT 0 1
rhaas=# select one.a, count(*) from one, two where one.a = two.a group by 1;
a | count
---+-------
0 | 1
(1 row)
rhaas=# select two.a, count(*) from one, two where one.a = two.a group by 1;
a | count
------+-------
0.00 | 1
(1 row)
There are, admittedly, a large number of data types for which such a
substitution would work just fine. numeric will not, citext will not,
but many others are fine. Regrettably, we have no framework in the
system for identifying equality operators which actually test identity
versus some looser notion of equality. Cross-type operators are a
problem, too; if we have foo.x = bar.y in the query, one might be int4
and the other int8, for example, but they can still belong to the same
equivalence class.
Concretely, in your test query "SELECT p.i, avg(c1.v) FROM
agg_pushdown_parent AS p JOIN agg_pushdown_child1 AS c1 ON c1.parent =
p.i GROUP BY p.i" you assume that it's OK to do a Partial
HashAggregate over c1.parent rather than p.i. This will be false if,
say, c1.parent is of type citext and p.i is of type text; this will
get grouped together that shouldn't. It will also be false if the
grouping expression is something like GROUP BY length(p.i::text),
because one value could be '0'::numeric and the other '0.00'::numeric.
I can't think of a reason why it would be false if the grouping
expressions are both simple Vars of the same underlying data type, but
I'm a little nervous that I might be wrong even about that case.
Maybe you've handled all of this somehow, but it's not obvious to me
that it has been considered.
Another thing I noticed is that the GroupedPathInfo looks a bit like a
stripped-down RelOptInfo, in that for example it has both a pathlist
and a partial_pathlist. I'm inclined to think that we should build new
RelOptInfos instead. As you have it, there are an awful lot of things
that seem to know about the grouped/ungrouped distinction, many of
which are quite low-level functions like add_path(),
add_paths_to_append_rel(), try_nestloop_path(), etc. I think that
some of this would go away if you had separate RelOptInfos instead of
GroupedPathInfo.
Some compiler noise:
allpaths.c:2794:11: error: variable 'partial_target' is used
uninitialized whenever 'if' condition is false
[-Werror,-Wsometimes-uninitialized]
else if (rel->gpi != NULL && rel->gpi->target != NULL)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
allpaths.c:2798:56: note: uninitialized use occurs here
create_gather_path(root, rel, cheapest_partial_path,
partial_target,
^~~~~~~~~~~~~~
allpaths.c:2794:7: note: remove the 'if' if its condition is always true
else if (rel->gpi != NULL && rel->gpi->target != NULL)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
allpaths.c:2794:11: error: variable 'partial_target' is used
uninitialized whenever '&&' condition is false
[-Werror,-Wsometimes-uninitialized]
else if (rel->gpi != NULL && rel->gpi->target != NULL)
^~~~~~~~~~~~~~~~
allpaths.c:2798:56: note: uninitialized use occurs here
create_gather_path(root, rel, cheapest_partial_path,
partial_target,
^~~~~~~~~~~~~~
allpaths.c:2794:11: note: remove the '&&' if its condition is always true
else if (rel->gpi != NULL && rel->gpi->target != NULL)
^~~~~~~~~~~~~~~~~~~
allpaths.c:2773:28: note: initialize the variable 'partial_target' to
silence this warning
PathTarget *partial_target;
^
= NULL
Core dump running the regression tests:
* frame #0: 0x00007fffe633ad42 libsystem_kernel.dylib`__pthread_kill + 10
frame #1: 0x00007fffe6428457 libsystem_pthread.dylib`pthread_kill + 90
frame #2: 0x00007fffe62a0420 libsystem_c.dylib`abort + 129
frame #3: 0x000000010e3ae90e
postgres`ExceptionalCondition(conditionName=<unavailable>,
errorType=<unavailable>, fileName=<unavailable>,
lineNumber=<unavailable>) at assert.c:54 [opt]
frame #4: 0x000000010e17c23b
postgres`check_list_invariants(list=<unavailable>) at list.c:39 [opt]
frame #5: 0x000000010e17cdff postgres`list_free [inlined]
list_free_private(list=0x00007fe6a483be40, deep='\0') at list.c:1107
[opt]
frame #6: 0x000000010e17cdfa
postgres`list_free(list=0x00007fe6a483be40) at list.c:1135 [opt]
frame #7: 0x000000010e1b365f
postgres`generate_bitmap_or_paths(root=0x00007fe6a68131c8,
rel=0x00007fe6a6815310, clauses=<unavailable>,
other_clauses=<unavailable>, agg_info=0x0000000000000000) at
indxpath.c:1580 [opt]
frame #8: 0x000000010e1b305c
postgres`create_index_paths(root=0x00007fe6a68131c8,
rel=<unavailable>, agg_info=0x0000000000000000) at indxpath.c:334
[opt]
frame #9: 0x000000010e1a7cfa postgres`set_rel_pathlist [inlined]
set_plain_rel_pathlist at allpaths.c:745 [opt]
frame #10: 0x000000010e1a7bc1
postgres`set_rel_pathlist(root=0x00007fe6a68131c8,
rel=0x00007fe6a6815310, rti=1, rte=0x00007fe6a6803270) at
allpaths.c:454 [opt]
frame #11: 0x000000010e1a4910 postgres`make_one_rel at allpaths.c:312 [opt]
frame #12: 0x000000010e1a48cc
postgres`make_one_rel(root=0x00007fe6a68131c8,
joinlist=0x00007fe6a6815810) at allpaths.c:182 [opt]
frame #13: 0x000000010e1cb806
postgres`query_planner(root=0x00007fe6a68131c8, tlist=<unavailable>,
qp_callback=<unavailable>, qp_extra=0x00007fff51ca24d8) at
planmain.c:268 [opt]
frame #14: 0x000000010e1ce7b4
postgres`grouping_planner(root=<unavailable>, inheritance_update='\0',
tuple_fraction=<unavailable>) at planner.c:1789 [opt]
frame #15: 0x000000010e1ccb12
postgres`subquery_planner(glob=<unavailable>,
parse=0x00007fe6a6803158, parent_root=<unavailable>,
hasRecursion=<unavailable>, tuple_fraction=0) at planner.c:901 [opt]
frame #16: 0x000000010e1cba3e
postgres`standard_planner(parse=0x00007fe6a6803158, cursorOptions=256,
boundParams=0x0000000000000000) at planner.c:364 [opt]
frame #17: 0x000000010e291b3b
postgres`pg_plan_query(querytree=0x00007fe6a6803158,
cursorOptions=256, boundParams=0x0000000000000000) at postgres.c:807
[opt]
frame #18: 0x000000010e291c6e
postgres`pg_plan_queries(querytrees=<unavailable>, cursorOptions=256,
boundParams=0x0000000000000000) at postgres.c:873 [opt]
frame #19: 0x000000010e296335
postgres`exec_simple_query(query_string="SELECT p1.oid,
p1.typname\nFROM pg_type as p1\nWHERE p1.typnamespace = 0 OR\n
(p1.typlen <= 0 AND p1.typlen != -1 AND p1.typlen != -2) OR\n
(p1.typtype not in ('b', 'c', 'd', 'e', 'p', 'r')) OR\n NOT
p1.typisdefined OR\n (p1.typalign not in ('c', 's', 'i', 'd')) OR\n
(p1.typstorage not in ('p', 'x', 'e', 'm'));") at postgres.c:1048
[opt]
frame #20: 0x000000010e295099
postgres`PostgresMain(argc=<unavailable>, argv=<unavailable>,
dbname="regression", username=<unavailable>) at postgres.c:0 [opt]
frame #21: 0x000000010e20a910 postgres`PostmasterMain [inlined]
BackendRun at postmaster.c:4412 [opt]
frame #22: 0x000000010e20a741 postgres`PostmasterMain [inlined]
BackendStartup at postmaster.c:4084 [opt]
frame #23: 0x000000010e20a3bd postgres`PostmasterMain at
postmaster.c:1757 [opt]
frame #24: 0x000000010e2098ff
postgres`PostmasterMain(argc=1516050552, argv=<unavailable>) at
postmaster.c:1365 [opt]
frame #25: 0x000000010e177537 postgres`main(argc=<unavailable>,
argv=<unavailable>) at main.c:228 [opt]
frame #26: 0x00007fffe620c235 libdyld.dylib`start + 1
(lldb) p debug_query_string
(const char *) $0 = 0x00007fe6a401e518 "SELECT p1.oid,
p1.typname\nFROM pg_type as p1\nWHERE p1.typnamespace = 0 OR\n
(p1.typlen <= 0 AND p1.typlen != -1 AND p1.typlen != -2) OR\n
(p1.typtype not in ('b', 'c', 'd', 'e', 'p', 'r')) OR\n NOT
p1.typisdefined OR\n (p1.typalign not in ('c', 's', 'i', 'd')) OR\n
(p1.typstorage not in ('p', 'x', 'e', 'm'));"
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] WIP: Aggregation push-down
Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Dec 22, 2017 at 10:43 AM, Antonin Houska <ah@cybertec.at> wrote:
Michael Paquier <michael.paquier@gmail.com> wrote:
On Sat, Nov 4, 2017 at 12:33 AM, Antonin Houska <ah@cybertec.at> wrote:
I'm not about to add any other features now. Implementation of the missing
parts (see the TODO comments in the code) is the next step. But what I'd
appreciate most is a feedback on the design. Thanks.I am getting a conflict after applying patch 5 but this did not get
any reviews so moved to next CF with waiting on author as status.Attached is the next version.
I've been a bit confused for a while about what this patch is trying
to do, so I spent some time today looking at it to try to figure it
out.
Thanks.
There's a lot I don't understand yet, but it seems like the general idea is
to build, potentially for each relation in the join tree, not only the
regular list of paths but also a list of "grouped" paths. If the
pre-grouped path wins, then we can get a final path that looks like Finalize
Aggregate -> Some Joins -> Partial Aggregate -> Maybe Some More Joins ->
Base Table Scan.
Yes. The regression test output shows some more plans.
In some cases the patch seems to replace that uppermost Finalize Aggregate
with a Result node.
This is a feature implemented in 09_avoid_agg_finalization.diff. You can
ignore it for now, it's of lower priority than the preceding parts and needs
more work to be complete.
translate_expression_to_rels() looks unsafe. Equivalence members are
known to be equal under the semantics of the relevant operator class,
but that doesn't mean that one can be freely substituted for another.
For example:rhaas=# create table one (a numeric);
CREATE TABLE
rhaas=# create table two (a numeric);
CREATE TABLE
rhaas=# insert into one values ('0');
INSERT 0 1
rhaas=# insert into two values ('0.00');
INSERT 0 1
rhaas=# select one.a, count(*) from one, two where one.a = two.a group by 1;
a | count
---+-------
0 | 1
(1 row)rhaas=# select two.a, count(*) from one, two where one.a = two.a group by 1;
a | count
------+-------
0.00 | 1
(1 row)There are, admittedly, a large number of data types for which such a
substitution would work just fine. numeric will not, citext will not,
but many others are fine. Regrettably, we have no framework in the
system for identifying equality operators which actually test identity
versus some looser notion of equality. Cross-type operators are a
problem, too; if we have foo.x = bar.y in the query, one might be int4
and the other int8, for example, but they can still belong to the same
equivalence class.Concretely, in your test query "SELECT p.i, avg(c1.v) FROM
agg_pushdown_parent AS p JOIN agg_pushdown_child1 AS c1 ON c1.parent =
p.i GROUP BY p.i" you assume that it's OK to do a Partial
HashAggregate over c1.parent rather than p.i. This will be false if,
say, c1.parent is of type citext and p.i is of type text; this will
get grouped together that shouldn't.
I've really missed this problem, thanks for bringing it up! Your
considerations made me think of the specifics of the types like numeric or
citext. Although your example does not demonstrate what I consider the core
issue, I believe we eventually think of the same.
Consider this example query (using the tables you defined upthread):
SELECT one.a
FROM one, two
WHERE one.a = two.a AND scale(two.a) > 1
GROUP BY 1
I we first try to aggregate "two", then evaluate WHERE clause and then
finalize the aggregation, the partial aggregation of "two" can put various
binary values of the "a" column (0, 0.0, etc.) into the same group so the
scale of the output (of the partial agregation) will be undefined. Thus the
aggregation push-down will change the input of the WHERE clause.
So one problem is that the grouping expression can be inappropriate for
partial aggregation even if there's no type change during the
translation. What I consider typical for this case is that the equality
operator used to identify groups (SortGroupClause.eqop) can return true even
if binary (stored) values of the inputs differ. The partial aggregation could
only take place if we had a special equality operator which distinguishes the
*binary* values (I don't know yet how to store this operator the catalog ---
in pg_opclass recors for the hash opclasses?)..
On the other hand, if the grouping expression is not a plain Var and there's
no type change during the translation and the expression output type is not of
the "identity-unsafe type" (numeric, citext, etc.), input vars of the
"identity-unsafe type"should not prevent us from using the expression for
partial aggregation: in such a case the grouping keys are computed in the same
way they would be w/o the partial aggregation.
The other problem is which type changes are legal. We should not allow any
type-specific information (such as scale or precision of the numeric type) to
bet lost or ambiguous. In this example
It will also be false if the grouping expression is something like GROUP BY
length(p.i::text), because one value could be '0'::numeric and the other
'0.00'::numeric.
you show that not only numeric -> int is a problem but also int ->
numeric. This is the tricky part.
One of my ideas is to check whether the source and destination types are
binary coercible (i.e. pg_cast(castfunc) = 0) but this might be a misuse of
the binary coercibility. Another idea is to allow only such changes that the
destination type is in the same operator class as the source, and explicitly
enumerate the "safe opclasses". But that would mean that user cannot define
new opclasses within which the translation is possible --- not sure this is a
serious issue.
Perhaps the most consistent solution is that the translation is not performed
if any input variable of the grouping expression should change its data type.
Another thing I noticed is that the GroupedPathInfo looks a bit like a
stripped-down RelOptInfo, in that for example it has both a pathlist
and a partial_pathlist. I'm inclined to think that we should build new
RelOptInfos instead. As you have it, there are an awful lot of things
that seem to know about the grouped/ungrouped distinction, many of
which are quite low-level functions like add_path(),
add_paths_to_append_rel(), try_nestloop_path(), etc. I think that
some of this would go away if you had separate RelOptInfos instead of
GroupedPathInfo.
This was proposed by Ashutosh Bapat upthread. I actually did this in the early
(not published) prototype of the patch and I considered it too invasive for
standard_join_search() and subroutines. IIRC an array like simple_rel_array
had to be introduced for the grouped relation in which the meaning of NULL was
twofold: either the relation is not a base relation or it is base relation but
no grouped relation exists for it. Also I thought there would be too much
duplicate data if the grouped relation had a separate RelOptInfo. Now that I'm
done with one approach I can consider the original approach aggain.
Some compiler noise:
Core dump running the regression tests:
I'll fix these, thanks.
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com
Re: [HACKERS] WIP: Aggregation push-down
On Fri, Jan 26, 2018 at 8:04 AM, Antonin Houska <ah@cybertec.at> wrote:
So one problem is that the grouping expression can be inappropriate for
partial aggregation even if there's no type change during the
translation. What I consider typical for this case is that the equality
operator used to identify groups (SortGroupClause.eqop) can return true even
if binary (stored) values of the inputs differ. The partial aggregation could
only take place if we had a special equality operator which distinguishes the
*binary* values (I don't know yet how to store this operator the catalog ---
in pg_opclass recors for the hash opclasses?)..
We don't have an operator that tests for binary equality, but it's
certainly testable from C; see datumIsEqual. I'm not sure how much
this really helps, though. I think it would be safe to build an
initial set of groups based on datumIsEqual comparisons and then
arrange to later merge some of the groups. But that's not something
we ever do in the executor today, so it might require quite a lot of
hacking. Also, it seems like it might really suck in some cases. For
instance, consider something like SELECT scale(one.a), sum(two.a) FROM
one, two WHERE one.a = two.a GROUP BY 1. Doing a Partial Aggregate on
two.a using datumIsEqual semantics, joining to a, and then doing a
Finalize Aggregate looks legal, but the Partial Aggregate may produce
a tremendous number of groups compared to the Finalize Aggregate. In
other words, this technique wouldn't merge any groups that shouldn't
be merged, but it might fail to merge groups that really need to be
merged to get good performance.
One of my ideas is to check whether the source and destination types are
binary coercible (i.e. pg_cast(castfunc) = 0) but this might be a misuse of
the binary coercibility.
Yeah, binary coercibility has nothing to do with this; that tells you
whether the two types are the same on disk, not whether they have the
same equality semantics. For instance, I think text and citext are
binary coercible, but their equality semantics are not the same.
Another idea is to allow only such changes that the
destination type is in the same operator class as the source, and explicitly
enumerate the "safe opclasses". But that would mean that user cannot define
new opclasses within which the translation is possible --- not sure this is a
serious issue.
Enumerating specific opclasses in the source code is a non-starter --
Tom Lane would roll over in his grave if he weren't still alive. What
we could perhaps consider doing is adding some mechanism for an
opclass or opfamily to say whether its equality semantics happen to be
exactly datumIsEqual() semantics. That's a little grotty because it
leaves data types for which that isn't true out in the cold, but on
the other hand it seems like it would be useful as a way of optimizing
a bunch of things other than this. Maybe it could also include a way
to specify that the comparator happens to have the semantics as C's
built-in < and > operators, which seems like a case that might also
lend itself to some optimizations.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] WIP: Aggregation push-down
Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Jan 26, 2018 at 8:04 AM, Antonin Houska <ah@cybertec.at> wrote:
So one problem is that the grouping expression can be inappropriate for
partial aggregation even if there's no type change during the
translation. What I consider typical for this case is that the equality
operator used to identify groups (SortGroupClause.eqop) can return true even
if binary (stored) values of the inputs differ. The partial aggregation could
only take place if we had a special equality operator which distinguishes the
*binary* values (I don't know yet how to store this operator the catalog ---
in pg_opclass recors for the hash opclasses?)..We don't have an operator that tests for binary equality, but it's
certainly testable from C; see datumIsEqual. I'm not sure how much
this really helps, though. I think it would be safe to build an
initial set of groups based on datumIsEqual comparisons and then
arrange to later merge some of the groups. But that's not something
we ever do in the executor today, so it might require quite a lot of
hacking. Also, it seems like it might really suck in some cases. For
instance, consider something like SELECT scale(one.a), sum(two.a) FROM
one, two WHERE one.a = two.a GROUP BY 1. Doing a Partial Aggregate on
two.a using datumIsEqual semantics, joining to a, and then doing a
Finalize Aggregate looks legal, but the Partial Aggregate may produce
a tremendous number of groups compared to the Finalize Aggregate. In
other words, this technique wouldn't merge any groups that shouldn't
be merged, but it might fail to merge groups that really need to be
merged to get good performance.
I don't insist on doing Partial Aggregate in any case. If we wanted to group
by the binary value, we'd probably have to enhance statistics accordingly. The
important thing is to recognize the special case like this. Rejection of the
Partial Aggregate would be the default response.
Another idea is to allow only such changes that the
destination type is in the same operator class as the source, and explicitly
enumerate the "safe opclasses". But that would mean that user cannot define
new opclasses within which the translation is possible --- not sure this is a
serious issue.Enumerating specific opclasses in the source code is a non-starter --
Tom Lane would roll over in his grave if he weren't still alive. What
we could perhaps consider doing is adding some mechanism for an
opclass or opfamily to say whether its equality semantics happen to be
exactly datumIsEqual() semantics. That's a little grotty because it
leaves data types for which that isn't true out in the cold, but on
the other hand it seems like it would be useful as a way of optimizing
a bunch of things other than this. Maybe it could also include a way
to specify that the comparator happens to have the semantics as C's
built-in < and > operators, which seems like a case that might also
lend itself to some optimizations.
I think of a variant of this: implement an universal function that tests the
binary values for equality (besides the actual arguments, caller would have to
pass info like typlen, typalign, typbyval for each argument, and cache these
for repeated calls) and set pg_proc(oprcode) to 0 wherever this function is
sufficient. Thus the problematic cases like numeric, citext, etc. would be
detected by their non-zero oprcode.
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com
Re: [HACKERS] WIP: Aggregation push-down
On 01/29/18 03:32, Antonin Houska wrote:
Robert Haas <robertmhaas@gmail.com> wrote:
only take place if we had a special equality operator which distinguishes the
*binary* values (I don't know yet how to store this operator the catalog ---
...
We don't have an operator that tests for binary equality, but it's
certainly testable from C; see datumIsEqual.
Disclaimer: I haven't been following the whole thread closely. But could
there be some way to exploit the composite-type *= operator?
https://www.postgresql.org/docs/current/static/functions-comparisons.html#COMPOSITE-TYPE-COMPARISON
-Chap
Re: [HACKERS] WIP: Aggregation push-down
On Mon, Jan 29, 2018 at 3:32 AM, Antonin Houska <ah@cybertec.at> wrote:
I think of a variant of this: implement an universal function that tests the
binary values for equality (besides the actual arguments, caller would have to
pass info like typlen, typalign, typbyval for each argument, and cache these
for repeated calls) and set pg_proc(oprcode) to 0 wherever this function is
sufficient. Thus the problematic cases like numeric, citext, etc. would be
detected by their non-zero oprcode.
I don't think that's a good option, because we don't want int4eq to
have to go through a code path that has branches to support varlenas
and cstrings. Andres is busy trying to speed up expression evaluation
by removing unnecessary code branches; adding new ones would be
undoing that valuable work.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] WIP: Aggregation push-down
Robert Haas <robertmhaas@gmail.com> wrote:
On Mon, Jan 29, 2018 at 3:32 AM, Antonin Houska <ah@cybertec.at> wrote:
I think of a variant of this: implement an universal function that tests the
binary values for equality (besides the actual arguments, caller would have to
pass info like typlen, typalign, typbyval for each argument, and cache these
for repeated calls) and set pg_proc(oprcode) to 0 wherever this function is
sufficient. Thus the problematic cases like numeric, citext, etc. would be
detected by their non-zero oprcode.I don't think that's a good option, because we don't want int4eq to
have to go through a code path that has branches to support varlenas
and cstrings. Andres is busy trying to speed up expression evaluation
by removing unnecessary code branches; adding new ones would be
undoing that valuable work.
I spent some more time thinking about this. What about adding a new strategy
number for hash index operator classes, e.g. HTBinaryEqualStrategyNumber? For
most types both HTEqualStrategyNumber and HTBinaryEqualStrategyNumber strategy
would point to the same operator, but types like numeric would naturally have
them different.
Thus the pushed-down partial aggregation can only use the
HTBinaryEqualStrategyNumber's operator to compare grouping expressions. In the
initial version (until we have useful statistics for the binary values) we can
avoid the aggregation push-down if the grouping expression output type has the
two strategies implemented using different functions because, as you noted
upthread, grouping based on binary equality can result in excessive number of
groups.
One open question is whether the binary equality operator needs a separate
operator class or not. If an opclass cares only about the binary equality, its
hash function(s) can be a lot simpler.
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com
Re: [HACKERS] WIP: Aggregation push-down
On Fri, Feb 23, 2018 at 11:08 AM, Antonin Houska <ah@cybertec.at> wrote:
I spent some more time thinking about this. What about adding a new strategy
number for hash index operator classes, e.g. HTBinaryEqualStrategyNumber? For
most types both HTEqualStrategyNumber and HTBinaryEqualStrategyNumber strategy
would point to the same operator, but types like numeric would naturally have
them different.Thus the pushed-down partial aggregation can only use the
HTBinaryEqualStrategyNumber's operator to compare grouping expressions. In the
initial version (until we have useful statistics for the binary values) we can
avoid the aggregation push-down if the grouping expression output type has the
two strategies implemented using different functions because, as you noted
upthread, grouping based on binary equality can result in excessive number of
groups.One open question is whether the binary equality operator needs a separate
operator class or not. If an opclass cares only about the binary equality, its
hash function(s) can be a lot simpler.
Hmm. How about instead adding another regproc field to pg_type which
stores the OID of a function that tests binary equality for that
datatype? If that happens to be equal to the OID you got from the
opclass, then you're all set.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] WIP: Aggregation push-down
Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Dec 22, 2017 at 10:43 AM, Antonin Houska <ah@cybertec.at> wrote:
Michael Paquier <michael.paquier@gmail.com> wrote:
On Sat, Nov 4, 2017 at 12:33 AM, Antonin Houska <ah@cybertec.at> wrote:
I'm not about to add any other features now. Implementation of the missing
parts (see the TODO comments in the code) is the next step. But what I'd
appreciate most is a feedback on the design. Thanks.I am getting a conflict after applying patch 5 but this did not get
any reviews so moved to next CF with waiting on author as status.Attached is the next version.
translate_expression_to_rels() looks unsafe. Equivalence members are
known to be equal under the semantics of the relevant operator class,
but that doesn't mean that one can be freely substituted for another.
For example:rhaas=# create table one (a numeric);
CREATE TABLE
rhaas=# create table two (a numeric);
CREATE TABLE
rhaas=# insert into one values ('0');
INSERT 0 1
rhaas=# insert into two values ('0.00');
INSERT 0 1
rhaas=# select one.a, count(*) from one, two where one.a = two.a group by 1;
a | count
---+-------
0 | 1
(1 row)rhaas=# select two.a, count(*) from one, two where one.a = two.a group by 1;
a | count
------+-------
0.00 | 1
(1 row)There are, admittedly, a large number of data types for which such a
substitution would work just fine. numeric will not, citext will not,
but many others are fine. Regrettably, we have no framework in the
system for identifying equality operators which actually test identity
versus some looser notion of equality. Cross-type operators are a
problem, too; if we have foo.x = bar.y in the query, one might be int4
and the other int8, for example, but they can still belong to the same
equivalence class.Concretely, in your test query "SELECT p.i, avg(c1.v) FROM
agg_pushdown_parent AS p JOIN agg_pushdown_child1 AS c1 ON c1.parent =
p.i GROUP BY p.i" you assume that it's OK to do a Partial
HashAggregate over c1.parent rather than p.i. This will be false if,
say, c1.parent is of type citext and p.i is of type text; this will
get grouped together that shouldn't. It will also be false if the
grouping expression is something like GROUP BY length(p.i::text),
because one value could be '0'::numeric and the other '0.00'::numeric.
I can't think of a reason why it would be false if the grouping
expressions are both simple Vars of the same underlying data type, but
I'm a little nervous that I might be wrong even about that case.
Maybe you've handled all of this somehow, but it's not obvious to me
that it has been considered.
These problems are still being investigated, see [1]/messages/by-id/CA+Tgmoa5Pp-DBJg=W8Xj8Czf-32PfxPgxwFPkA6qN2w_wPX8bg@mail.gmail.com
Another thing I noticed is that the GroupedPathInfo looks a bit like a
stripped-down RelOptInfo, in that for example it has both a pathlist
and a partial_pathlist. I'm inclined to think that we should build new
RelOptInfos instead. As you have it, there are an awful lot of things
that seem to know about the grouped/ungrouped distinction, many of
which are quite low-level functions like add_path(),
add_paths_to_append_rel(), try_nestloop_path(), etc. I think that
some of this would go away if you had separate RelOptInfos instead of
GroupedPathInfo.
I've reworked the patch so that separate RelOptInfo is used for grouped
relation. The attached patch is only the core part. Neither postgres_fdw
changes nor the part that tries to avoid the final aggregation is included
here. I'll add these when the patch does not seem to need another major rework.
[1]: /messages/by-id/CA+Tgmoa5Pp-DBJg=W8Xj8Czf-32PfxPgxwFPkA6qN2w_wPX8bg@mail.gmail.com
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com
Attachments:
agg_pushdown_v6.difftext/x-diffDownload
diff --git a/src/backend/executor/execExpr.c b/src/backend/executor/execExpr.c
new file mode 100644
index db5fcaf..813249a
*** a/src/backend/executor/execExpr.c
--- b/src/backend/executor/execExpr.c
*************** ExecInitExprRec(Expr *node, ExprState *s
*** 794,799 ****
--- 794,840 ----
break;
}
+ case T_GroupedVar:
+
+ /*
+ * If GroupedVar appears in targetlist of Agg node, it can
+ * represent either Aggref or grouping expression.
+ */
+ if (state->parent && (IsA(state->parent, AggState)))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ /*
+ * The only reason to execute GroupedVar is to generate either
+ * aggregate transient state or grouping expression value. So
+ * any contained Aggref must be partial.
+ *
+ * (The purpose of propagating GroupedVars to upper plan nodes
+ * is just to transfer the value, no execution takes place
+ * there.)
+ */
+ if (IsA(gvar->gvexpr, Aggref))
+
+ ExecInitExprRec((Expr *) gvar->agg_partial, state,
+ resv, resnull);
+ else
+ ExecInitExprRec((Expr *) gvar->gvexpr, state,
+ resv, resnull);
+ break;
+ }
+ else
+ {
+ /*
+ * set_plan_refs should have replaced GroupedVar in the
+ * targetlist with an ordinary Var.
+ *
+ * XXX Should we error out here? There's at least one legal
+ * case here which we'd have to check: a Result plan with no
+ * outer plan which represents an empty Append plan.
+ */
+ break;
+ }
+
case T_GroupingFunc:
{
GroupingFunc *grp_node = (GroupingFunc *) node;
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
new file mode 100644
index 266a3ef..dca0653
*** a/src/backend/nodes/copyfuncs.c
--- b/src/backend/nodes/copyfuncs.c
*************** _copyAggref(const Aggref *from)
*** 1367,1372 ****
--- 1367,1373 ----
COPY_SCALAR_FIELD(aggcollid);
COPY_SCALAR_FIELD(inputcollid);
COPY_SCALAR_FIELD(aggtranstype);
+ COPY_SCALAR_FIELD(aggcombinefn);
COPY_NODE_FIELD(aggargtypes);
COPY_NODE_FIELD(aggdirectargs);
COPY_NODE_FIELD(args);
*************** _copyPlaceHolderVar(const PlaceHolderVar
*** 2219,2224 ****
--- 2220,2241 ----
}
/*
+ * _copyGroupedVar
+ */
+ static GroupedVar *
+ _copyGroupedVar(const GroupedVar *from)
+ {
+ GroupedVar *newnode = makeNode(GroupedVar);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gvid);
+
+ return newnode;
+ }
+
+ /*
* _copySpecialJoinInfo
*/
static SpecialJoinInfo *
*************** _copyPlaceHolderInfo(const PlaceHolderIn
*** 2292,2297 ****
--- 2309,2329 ----
return newnode;
}
+ static GroupedVarInfo *
+ _copyGroupedVarInfo(const GroupedVarInfo *from)
+ {
+ GroupedVarInfo *newnode = makeNode(GroupedVarInfo);
+
+ COPY_SCALAR_FIELD(gvid);
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gv_eval_at);
+ COPY_SCALAR_FIELD(gv_width);
+
+ return newnode;
+ }
+
/* ****************************************************************
* parsenodes.h copy functions
* ****************************************************************
*************** copyObjectImpl(const void *from)
*** 5034,5039 ****
--- 5066,5074 ----
case T_PlaceHolderVar:
retval = _copyPlaceHolderVar(from);
break;
+ case T_GroupedVar:
+ retval = _copyGroupedVar(from);
+ break;
case T_SpecialJoinInfo:
retval = _copySpecialJoinInfo(from);
break;
*************** copyObjectImpl(const void *from)
*** 5046,5051 ****
--- 5081,5089 ----
case T_PlaceHolderInfo:
retval = _copyPlaceHolderInfo(from);
break;
+ case T_GroupedVarInfo:
+ retval = _copyGroupedVarInfo(from);
+ break;
/*
* VALUE NODES
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
new file mode 100644
index bbffc87..fb26311
*** a/src/backend/nodes/equalfuncs.c
--- b/src/backend/nodes/equalfuncs.c
*************** _equalPlaceHolderVar(const PlaceHolderVa
*** 873,878 ****
--- 873,886 ----
}
static bool
+ _equalGroupedVar(const GroupedVar *a, const GroupedVar *b)
+ {
+ COMPARE_SCALAR_FIELD(gvid);
+
+ return true;
+ }
+
+ static bool
_equalSpecialJoinInfo(const SpecialJoinInfo *a, const SpecialJoinInfo *b)
{
COMPARE_BITMAPSET_FIELD(min_lefthand);
*************** equal(const void *a, const void *b)
*** 3179,3184 ****
--- 3187,3195 ----
case T_PlaceHolderVar:
retval = _equalPlaceHolderVar(a, b);
break;
+ case T_GroupedVar:
+ retval = _equalGroupedVar(a, b);
+ break;
case T_SpecialJoinInfo:
retval = _equalSpecialJoinInfo(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
new file mode 100644
index 6c76c41..d34b26b
*** a/src/backend/nodes/nodeFuncs.c
--- b/src/backend/nodes/nodeFuncs.c
*************** exprType(const Node *expr)
*** 259,264 ****
--- 259,270 ----
case T_PlaceHolderVar:
type = exprType((Node *) ((const PlaceHolderVar *) expr)->phexpr);
break;
+ case T_GroupedVar:
+ if (IsA(((const GroupedVar *) expr)->gvexpr, Aggref))
+ type = exprType((Node *) ((const GroupedVar *) expr)->agg_partial);
+ else
+ type = exprType((Node *) ((const GroupedVar *) expr)->gvexpr);
+ break;
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(expr));
type = InvalidOid; /* keep compiler quiet */
*************** exprTypmod(const Node *expr)
*** 492,497 ****
--- 498,508 ----
return ((const SetToDefault *) expr)->typeMod;
case T_PlaceHolderVar:
return exprTypmod((Node *) ((const PlaceHolderVar *) expr)->phexpr);
+ case T_GroupedVar:
+ if (IsA(((const GroupedVar *) expr)->gvexpr, Aggref))
+ return exprTypmod((Node *) ((const GroupedVar *) expr)->agg_partial);
+ else
+ return exprTypmod((Node *) ((const GroupedVar *) expr)->gvexpr);
default:
break;
}
*************** exprCollation(const Node *expr)
*** 903,908 ****
--- 914,925 ----
case T_PlaceHolderVar:
coll = exprCollation((Node *) ((const PlaceHolderVar *) expr)->phexpr);
break;
+ case T_GroupedVar:
+ if (IsA(((const GroupedVar *) expr)->gvexpr, Aggref))
+ coll = exprCollation((Node *) ((const GroupedVar *) expr)->agg_partial);
+ else
+ coll = exprCollation((Node *) ((const GroupedVar *) expr)->gvexpr);
+ break;
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(expr));
coll = InvalidOid; /* keep compiler quiet */
*************** expression_tree_walker(Node *node,
*** 2176,2181 ****
--- 2193,2200 ----
break;
case T_PlaceHolderVar:
return walker(((PlaceHolderVar *) node)->phexpr, context);
+ case T_GroupedVar:
+ return walker(((GroupedVar *) node)->gvexpr, context);
case T_InferenceElem:
return walker(((InferenceElem *) node)->expr, context);
case T_AppendRelInfo:
*************** expression_tree_mutator(Node *node,
*** 2968,2973 ****
--- 2987,3002 ----
return (Node *) newnode;
}
break;
+ case T_GroupedVar:
+ {
+ GroupedVar *gv = (GroupedVar *) node;
+ GroupedVar *newnode;
+
+ FLATCOPY(newnode, gv, GroupedVar);
+ MUTATE(newnode->gvexpr, gv->gvexpr, Expr *);
+ MUTATE(newnode->agg_partial, gv->agg_partial, Aggref *);
+ return (Node *) newnode;
+ }
case T_InferenceElem:
{
InferenceElem *inferenceelemdexpr = (InferenceElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
new file mode 100644
index 011d2a3..37d2fb0
*** a/src/backend/nodes/outfuncs.c
--- b/src/backend/nodes/outfuncs.c
*************** _outAggref(StringInfo str, const Aggref
*** 1142,1147 ****
--- 1142,1148 ----
WRITE_OID_FIELD(aggcollid);
WRITE_OID_FIELD(inputcollid);
WRITE_OID_FIELD(aggtranstype);
+ WRITE_OID_FIELD(aggcombinefn);
WRITE_NODE_FIELD(aggargtypes);
WRITE_NODE_FIELD(aggdirectargs);
WRITE_NODE_FIELD(args);
*************** _outPlannerInfo(StringInfo str, const Pl
*** 2218,2223 ****
--- 2219,2225 ----
WRITE_BITMAPSET_FIELD(all_baserels);
WRITE_BITMAPSET_FIELD(nullable_baserels);
WRITE_NODE_FIELD(join_rel_list);
+ WRITE_NODE_FIELD(join_grouped_rel_list);
WRITE_INT_FIELD(join_cur_level);
WRITE_NODE_FIELD(init_plans);
WRITE_NODE_FIELD(cte_plan_ids);
*************** _outPlannerInfo(StringInfo str, const Pl
*** 2232,2237 ****
--- 2234,2240 ----
WRITE_NODE_FIELD(pcinfo_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
*************** _outPlannerInfo(StringInfo str, const Pl
*** 2239,2244 ****
--- 2242,2248 ----
WRITE_NODE_FIELD(distinct_pathkeys);
WRITE_NODE_FIELD(sort_pathkeys);
WRITE_NODE_FIELD(processed_tlist);
+ WRITE_INT_FIELD(max_sortgroupref);
WRITE_NODE_FIELD(minmax_aggs);
WRITE_FLOAT_FIELD(total_table_pages, "%.0f");
WRITE_FLOAT_FIELD(tuple_fraction, "%.4f");
*************** _outRelOptInfo(StringInfo str, const Rel
*** 2278,2283 ****
--- 2282,2288 ----
WRITE_NODE_FIELD(cheapest_parameterized_paths);
WRITE_BITMAPSET_FIELD(direct_lateral_relids);
WRITE_BITMAPSET_FIELD(lateral_relids);
+ WRITE_NODE_FIELD(agg_info);
WRITE_UINT_FIELD(relid);
WRITE_OID_FIELD(reltablespace);
WRITE_ENUM_FIELD(rtekind, RTEKind);
*************** _outParamPathInfo(StringInfo str, const
*** 2454,2459 ****
--- 2459,2476 ----
}
static void
+ _outRelAggInfo(StringInfo str, const RelAggInfo *node)
+ {
+ WRITE_NODE_TYPE("RELAGGINFO");
+
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(input);
+ WRITE_NODE_FIELD(group_clauses);
+ WRITE_NODE_FIELD(group_exprs);
+ WRITE_NODE_FIELD(agg_exprs);
+ }
+
+ static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
WRITE_NODE_TYPE("RESTRICTINFO");
*************** _outPlaceHolderVar(StringInfo str, const
*** 2497,2502 ****
--- 2514,2530 ----
}
static void
+ _outGroupedVar(StringInfo str, const GroupedVar *node)
+ {
+ WRITE_NODE_TYPE("GROUPEDVAR");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_UINT_FIELD(gvid);
+ }
+
+ static void
_outSpecialJoinInfo(StringInfo str, const SpecialJoinInfo *node)
{
WRITE_NODE_TYPE("SPECIALJOININFO");
*************** _outPlaceHolderInfo(StringInfo str, cons
*** 2551,2556 ****
--- 2579,2597 ----
}
static void
+ _outGroupedVarInfo(StringInfo str, const GroupedVarInfo *node)
+ {
+ WRITE_NODE_TYPE("GROUPEDVARINFO");
+
+ WRITE_UINT_FIELD(gvid);
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_BITMAPSET_FIELD(gv_eval_at);
+ WRITE_INT_FIELD(gv_width);
+ }
+
+ static void
_outMinMaxAggInfo(StringInfo str, const MinMaxAggInfo *node)
{
WRITE_NODE_TYPE("MINMAXAGGINFO");
*************** outNode(StringInfo str, const void *obj)
*** 4060,4071 ****
--- 4101,4118 ----
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_RelAggInfo:
+ _outRelAggInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
case T_PlaceHolderVar:
_outPlaceHolderVar(str, obj);
break;
+ case T_GroupedVar:
+ _outGroupedVar(str, obj);
+ break;
case T_SpecialJoinInfo:
_outSpecialJoinInfo(str, obj);
break;
*************** outNode(StringInfo str, const void *obj)
*** 4078,4083 ****
--- 4125,4133 ----
case T_PlaceHolderInfo:
_outPlaceHolderInfo(str, obj);
break;
+ case T_GroupedVarInfo:
+ _outGroupedVarInfo(str, obj);
+ break;
case T_MinMaxAggInfo:
_outMinMaxAggInfo(str, obj);
break;
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
new file mode 100644
index 068db35..f5dceb0
*** a/src/backend/nodes/readfuncs.c
--- b/src/backend/nodes/readfuncs.c
*************** _readVar(void)
*** 534,539 ****
--- 534,555 ----
}
/*
+ * _readGroupedVar
+ */
+ static GroupedVar *
+ _readGroupedVar(void)
+ {
+ READ_LOCALS(GroupedVar);
+
+ READ_NODE_FIELD(gvexpr);
+ READ_NODE_FIELD(agg_partial);
+ READ_UINT_FIELD(sortgroupref);
+ READ_UINT_FIELD(gvid);
+
+ READ_DONE();
+ }
+
+ /*
* _readConst
*/
static Const *
*************** _readAggref(void)
*** 589,594 ****
--- 605,611 ----
READ_OID_FIELD(aggcollid);
READ_OID_FIELD(inputcollid);
READ_OID_FIELD(aggtranstype);
+ READ_OID_FIELD(aggcombinefn);
READ_NODE_FIELD(aggargtypes);
READ_NODE_FIELD(aggdirectargs);
READ_NODE_FIELD(args);
*************** parseNodeString(void)
*** 2483,2488 ****
--- 2500,2507 ----
return_value = _readTableFunc();
else if (MATCH("VAR", 3))
return_value = _readVar();
+ else if (MATCH("GROUPEDVAR", 10))
+ return_value = _readGroupedVar();
else if (MATCH("CONST", 5))
return_value = _readConst();
else if (MATCH("PARAM", 5))
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
new file mode 100644
index 0be2a73..ecff708
*** a/src/backend/optimizer/geqo/geqo_eval.c
--- b/src/backend/optimizer/geqo/geqo_eval.c
*************** merge_clump(PlannerInfo *root, List *clu
*** 249,254 ****
--- 249,255 ----
if (force ||
desirable_join(root, old_clump->joinrel, new_clump->joinrel))
{
+ JoinSearchResult *joinrels;
RelOptInfo *joinrel;
/*
*************** merge_clump(PlannerInfo *root, List *clu
*** 257,265 ****
* root->join_rel_list yet, and so the paths constructed for it
* will only include the ones we want.
*/
! joinrel = make_join_rel(root,
! old_clump->joinrel,
! new_clump->joinrel);
/* Keep searching if join order is not valid */
if (joinrel)
--- 258,267 ----
* root->join_rel_list yet, and so the paths constructed for it
* will only include the ones we want.
*/
! joinrels = make_join_rel(root,
! old_clump->joinrel,
! new_clump->joinrel);
! joinrel = joinrels->plain;
/* Keep searching if join order is not valid */
if (joinrel)
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
new file mode 100644
index 1c792a0..be61d30
*** a/src/backend/optimizer/path/allpaths.c
--- b/src/backend/optimizer/path/allpaths.c
*************** typedef struct pushdown_safety_info
*** 57,62 ****
--- 57,63 ----
/* These parameters are set by GUC */
bool enable_geqo = false; /* just in case GUC doesn't set it */
+ bool enable_agg_pushdown;
int geqo_threshold;
int min_parallel_table_scan_size;
int min_parallel_index_scan_size;
*************** set_rel_pathlist_hook_type set_rel_pathl
*** 68,76 ****
join_search_hook_type join_search_hook = NULL;
! static void set_base_rel_consider_startup(PlannerInfo *root);
! static void set_base_rel_sizes(PlannerInfo *root);
! static void set_base_rel_pathlists(PlannerInfo *root);
static void set_rel_size(PlannerInfo *root, RelOptInfo *rel,
Index rti, RangeTblEntry *rte);
static void set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
--- 69,77 ----
join_search_hook_type join_search_hook = NULL;
! static void set_base_rel_consider_startup(PlannerInfo *root, bool grouped);
! static void set_base_rel_sizes(PlannerInfo *root, bool grouped);
! static void set_base_rel_pathlists(PlannerInfo *root, bool grouped);
static void set_rel_size(PlannerInfo *root, RelOptInfo *rel,
Index rti, RangeTblEntry *rte);
static void set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
*************** static void set_namedtuplestore_pathlist
*** 117,123 ****
RangeTblEntry *rte);
static void set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
! static RelOptInfo *make_rel_from_joinlist(PlannerInfo *root, List *joinlist);
static bool subquery_is_pushdown_safe(Query *subquery, Query *topquery,
pushdown_safety_info *safetyInfo);
static bool recurse_pushdown_safe(Node *setOp, Query *topquery,
--- 118,125 ----
RangeTblEntry *rte);
static void set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
! static JoinSearchResult *make_rel_from_joinlist(PlannerInfo *root,
! List *joinlist);
static bool subquery_is_pushdown_safe(Query *subquery, Query *topquery,
pushdown_safety_info *safetyInfo);
static bool recurse_pushdown_safe(Node *setOp, Query *topquery,
*************** static void add_paths_to_append_rel(Plan
*** 141,152 ****
/*
* make_one_rel
* Finds all possible access paths for executing a query, returning a
! * single rel that represents the join of all base rels in the query.
*/
! RelOptInfo *
make_one_rel(PlannerInfo *root, List *joinlist)
{
! RelOptInfo *rel;
Index rti;
/*
--- 143,155 ----
/*
* make_one_rel
* Finds all possible access paths for executing a query, returning a
! * single rel that represents the join of all base rels in the query. If
! * possible, also return a join that contains partial aggregate(s).
*/
! JoinSearchResult *
make_one_rel(PlannerInfo *root, List *joinlist)
{
! JoinSearchResult *rels;
Index rti;
/*
*************** make_one_rel(PlannerInfo *root, List *jo
*** 170,196 ****
root->all_baserels = bms_add_member(root->all_baserels, brel->relid);
}
! /* Mark base rels as to whether we care about fast-start plans */
! set_base_rel_consider_startup(root);
/*
! * Compute size estimates and consider_parallel flags for each base rel,
! * then generate access paths.
*/
! set_base_rel_sizes(root);
! set_base_rel_pathlists(root);
/*
* Generate access paths for the entire join tree.
*/
! rel = make_rel_from_joinlist(root, joinlist);
/*
* The result should join all and only the query's base rels.
*/
! Assert(bms_equal(rel->relids, root->all_baserels));
! return rel;
}
/*
--- 173,215 ----
root->all_baserels = bms_add_member(root->all_baserels, brel->relid);
}
! /*
! * Mark base rels as to whether we care about fast-start plans. XXX We
! * deliberately do not mark grouped rels --- see the comment on
! * consider_startup in build_simple_rel().
! */
! set_base_rel_consider_startup(root, false);
/*
! * As for grouped relations, paths differ substantially by the
! * AggStrategy. Paths that use AGG_HASHED should not be parameterized
! * (because creation of hashtable would have to be repeated for different
! * parameters) but paths using AGG_SORTED can be. The latter seem to
! * justify calling the function for grouped relations too.
*/
! set_base_rel_consider_startup(root, true);
!
! /*
! * Compute size estimates and consider_parallel flags for each plain and
! * each grouped base rel, then generate access paths.
! */
! set_base_rel_sizes(root, false);
! set_base_rel_pathlists(root, false);
!
! set_base_rel_sizes(root, true);
! set_base_rel_pathlists(root, true);
/*
* Generate access paths for the entire join tree.
*/
! rels = make_rel_from_joinlist(root, joinlist);
/*
* The result should join all and only the query's base rels.
*/
! Assert(rels->plain && bms_equal(rels->plain->relids, root->all_baserels));
! return rels;
}
/*
*************** make_one_rel(PlannerInfo *root, List *jo
*** 204,210 ****
* be better to move it here.
*/
static void
! set_base_rel_consider_startup(PlannerInfo *root)
{
/*
* Since parameterized paths can only be used on the inside of a nestloop
--- 223,229 ----
* be better to move it here.
*/
static void
! set_base_rel_consider_startup(PlannerInfo *root, bool grouped)
{
/*
* Since parameterized paths can only be used on the inside of a nestloop
*************** set_base_rel_consider_startup(PlannerInf
*** 229,237 ****
if ((sjinfo->jointype == JOIN_SEMI || sjinfo->jointype == JOIN_ANTI) &&
bms_get_singleton_member(sjinfo->syn_righthand, &varno))
{
! RelOptInfo *rel = find_base_rel(root, varno);
! rel->consider_param_startup = true;
}
}
}
--- 248,262 ----
if ((sjinfo->jointype == JOIN_SEMI || sjinfo->jointype == JOIN_ANTI) &&
bms_get_singleton_member(sjinfo->syn_righthand, &varno))
{
! RelOptInfo *rel = !grouped ? find_base_rel(root, varno) :
! find_grouped_base_rel(root, varno);
! if (rel != NULL)
! rel->consider_param_startup = true;
! #ifdef USE_ASSERT_CHECKING
! else
! Assert(grouped);
! #endif /* USE_ASSERT_CHECKING */
}
}
}
*************** set_base_rel_consider_startup(PlannerInf
*** 247,262 ****
* generate paths.
*/
static void
! set_base_rel_sizes(PlannerInfo *root)
{
Index rti;
for (rti = 1; rti < root->simple_rel_array_size; rti++)
{
! RelOptInfo *rel = root->simple_rel_array[rti];
RangeTblEntry *rte;
! /* there may be empty slots corresponding to non-baserel RTEs */
if (rel == NULL)
continue;
--- 272,294 ----
* generate paths.
*/
static void
! set_base_rel_sizes(PlannerInfo *root, bool grouped)
{
Index rti;
+ RelOptInfo **rel_array;
+
+ rel_array = !grouped ? root->simple_rel_array :
+ root->simple_grouped_rel_array;
for (rti = 1; rti < root->simple_rel_array_size; rti++)
{
! RelOptInfo *rel = rel_array[rti];
RangeTblEntry *rte;
! /*
! * There may be empty slots corresponding to non-baserel RTEs, or to
! * baserel which cannot be grouped.
! */
if (rel == NULL)
continue;
*************** set_base_rel_sizes(PlannerInfo *root)
*** 290,304 ****
* Each useful path is attached to its relation's 'pathlist' field.
*/
static void
! set_base_rel_pathlists(PlannerInfo *root)
{
Index rti;
for (rti = 1; rti < root->simple_rel_array_size; rti++)
{
! RelOptInfo *rel = root->simple_rel_array[rti];
! /* there may be empty slots corresponding to non-baserel RTEs */
if (rel == NULL)
continue;
--- 322,343 ----
* Each useful path is attached to its relation's 'pathlist' field.
*/
static void
! set_base_rel_pathlists(PlannerInfo *root, bool grouped)
{
Index rti;
+ RelOptInfo **rel_array;
+
+ rel_array = !grouped ? root->simple_rel_array :
+ root->simple_grouped_rel_array;
for (rti = 1; rti < root->simple_rel_array_size; rti++)
{
! RelOptInfo *rel = rel_array[rti];
! /*
! * There may be empty slots corresponding to non-baserel RTEs, or to
! * baserel which cannot be grouped.
! */
if (rel == NULL)
continue;
*************** static void
*** 320,325 ****
--- 359,372 ----
set_rel_size(PlannerInfo *root, RelOptInfo *rel,
Index rti, RangeTblEntry *rte)
{
+ bool grouped = rel->agg_info != NULL;
+
+ /*
+ * build_simple_rel() should not have created rels that do not match this
+ * condition.
+ */
+ Assert(!grouped || rte->rtekind == RTE_RELATION);
+
if (rel->reloptkind == RELOPT_BASEREL &&
relation_excluded_by_constraints(root, rel, rte))
{
*************** set_rel_size(PlannerInfo *root, RelOptIn
*** 349,354 ****
--- 396,403 ----
if (rte->relkind == RELKIND_FOREIGN_TABLE)
{
/* Foreign table */
+ /* Not supported yet, see build_simple_rel(). */
+ Assert(!grouped);
set_foreign_size(root, rel, rte);
}
else if (rte->relkind == RELKIND_PARTITIONED_TABLE)
*************** set_rel_size(PlannerInfo *root, RelOptIn
*** 362,367 ****
--- 411,418 ----
else if (rte->tablesample != NULL)
{
/* Sampled relation */
+ /* Not supported yet, see build_simple_rel(). */
+ Assert(!grouped);
set_tablesample_rel_size(root, rel, rte);
}
else
*************** static void
*** 423,428 ****
--- 474,487 ----
set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
Index rti, RangeTblEntry *rte)
{
+ bool grouped = rel->agg_info != NULL;
+
+ /*
+ * build_simple_rel() should not have created rels that do not match this
+ * condition.
+ */
+ Assert(!grouped || rte->rtekind == RTE_RELATION);
+
if (IS_DUMMY_REL(rel))
{
/* We already proved the relation empty, so nothing more to do */
*************** set_rel_pathlist(PlannerInfo *root, RelO
*** 440,450 ****
--- 499,513 ----
if (rte->relkind == RELKIND_FOREIGN_TABLE)
{
/* Foreign table */
+ /* Not supported yet, see build_simple_rel(). */
+ Assert(!grouped);
set_foreign_pathlist(root, rel, rte);
}
else if (rte->tablesample != NULL)
{
/* Sampled relation */
+ /* Not supported yet, see build_simple_rel(). */
+ Assert(!grouped);
set_tablesample_rel_pathlist(root, rel, rte);
}
else
*************** static void
*** 689,694 ****
--- 752,759 ----
set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
{
Relids required_outer;
+ Path *seq_path;
+ bool grouped = rel->agg_info != NULL;
/*
* We don't support pushing join clauses into the quals of a seqscan, but
*************** set_plain_rel_pathlist(PlannerInfo *root
*** 697,714 ****
*/
required_outer = rel->lateral_relids;
! /* Consider sequential scan */
! add_path(rel, create_seqscan_path(root, rel, required_outer, 0));
! /* If appropriate, consider parallel sequential scan */
if (rel->consider_parallel && required_outer == NULL)
create_plain_partial_paths(root, rel);
! /* Consider index scans */
create_index_paths(root, rel);
/* Consider TID scans */
! create_tidscan_paths(root, rel);
}
/*
--- 762,796 ----
*/
required_outer = rel->lateral_relids;
! /* Consider sequential scan, both plain and grouped. */
! seq_path = create_seqscan_path(root, rel, required_outer, 0);
! /*
! * It's probably not good idea to repeat hashed aggregation with different
! * parameters, so check if there are no parameters.
! */
! if (!grouped)
! add_path(rel, seq_path);
! else if (required_outer == NULL)
! {
! /*
! * Only AGG_HASHED is suitable here as it does not expect the input
! * set to be sorted.
! */
! create_grouped_path(root, rel, seq_path, false, false, AGG_HASHED);
! }
!
! /* If appropriate, consider parallel sequential scan (plain or grouped) */
if (rel->consider_parallel && required_outer == NULL)
create_plain_partial_paths(root, rel);
! /*
! * Consider index scans.
! */
create_index_paths(root, rel);
/* Consider TID scans */
! create_tidscan_paths(root, rel, grouped);
}
/*
*************** static void
*** 719,734 ****
create_plain_partial_paths(PlannerInfo *root, RelOptInfo *rel)
{
int parallel_workers;
! parallel_workers = compute_parallel_worker(rel, rel->pages, -1,
! max_parallel_workers_per_gather);
/* If any limit was set to zero, the user doesn't want a parallel scan. */
if (parallel_workers <= 0)
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! add_partial_path(rel, create_seqscan_path(root, rel, NULL, parallel_workers));
}
/*
--- 801,907 ----
create_plain_partial_paths(PlannerInfo *root, RelOptInfo *rel)
{
int parallel_workers;
+ Path *path;
+ bool grouped = rel->agg_info != NULL;
! parallel_workers = compute_parallel_worker(rel, rel->pages, -1, max_parallel_workers_per_gather);
/* If any limit was set to zero, the user doesn't want a parallel scan. */
if (parallel_workers <= 0)
return;
/* Add an unordered partial path based on a parallel sequential scan. */
! path = create_seqscan_path(root, rel, NULL, parallel_workers);
!
! if (!grouped)
! add_partial_path(rel, path);
! else
! {
! /*
! * Do partial aggregation at base relation level if the relation is
! * eligible for it. Only AGG_HASHED is suitable here as it does not
! * expect the input set to be sorted.
! */
! create_grouped_path(root, rel, path, false, true, AGG_HASHED);
! }
! }
!
! /*
! * Apply partial aggregation to a subpath and add the AggPath to the pathlist.
! *
! * "precheck" tells whether the aggregation path should first be checked using
! * add_path_precheck() / add_partial_path_precheck().
! *
! * If "partial" is true, the aggregation path is considered partial in terms
! * of parallel execution.
! *
! * The return value tells whether the path was added to the pathlist.
! */
! bool
! create_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
! bool precheck, bool partial, AggStrategy aggstrategy)
! {
! Path *agg_path;
! RelAggInfo *agg_info = rel->agg_info;
!
! Assert(agg_info != NULL);
!
! /*
! * If the AggPath should be partial, the subpath must be too, and
! * therefore the subpath is essentially parallel_safe.
! */
! Assert(subpath->parallel_safe || !partial);
!
! /*
! * Repeated creation of hash table does not sound like a good idea. Caller
! * should avoid asking us to do so.
! */
! Assert(subpath->param_info == NULL || aggstrategy != AGG_HASHED);
!
! /*
! * Note that "partial" in the following function names refers to 2-stage
! * aggregation, not to parallel processing.
! */
! if (aggstrategy == AGG_HASHED)
! agg_path = (Path *) create_partial_agg_hashed_path(root, subpath,
! subpath->rows);
! else if (aggstrategy == AGG_SORTED)
! agg_path = (Path *) create_partial_agg_sorted_path(root, subpath,
! true,
! subpath->rows);
! else
! elog(ERROR, "unexpected strategy %d", aggstrategy);
!
! /* Add the grouped path to the list of grouped base paths. */
! if (agg_path != NULL)
! {
! if (precheck)
! {
! List *pathkeys;
!
! /* AGG_HASH is not supposed to generate sorted output. */
! pathkeys = aggstrategy == AGG_SORTED ? subpath->pathkeys : NIL;
!
! if (!partial &&
! !add_path_precheck(rel, agg_path->startup_cost,
! agg_path->total_cost, pathkeys, NULL))
! return false;
!
! if (partial &&
! !add_partial_path_precheck(rel, agg_path->total_cost,
! pathkeys))
! return false;
! }
!
! if (!partial)
! add_path(rel, (Path *) agg_path);
! else
! add_partial_path(rel, (Path *) agg_path);
!
! return true;
! }
!
! return false;
}
/*
*************** set_append_rel_size(PlannerInfo *root, R
*** 869,874 ****
--- 1042,1048 ----
double *parent_attrsizes;
int nattrs;
ListCell *l;
+ bool grouped = rel->agg_info != NULL;
/* Guard against stack overflow due to overly deep inheritance tree. */
check_stack_depth();
*************** set_append_rel_size(PlannerInfo *root, R
*** 919,925 ****
* The child rel's RelOptInfo was already created during
* add_base_rels_to_query.
*/
! childrel = find_base_rel(root, childRTindex);
Assert(childrel->reloptkind == RELOPT_OTHER_MEMBER_REL);
if (rel->part_scheme)
--- 1093,1100 ----
* The child rel's RelOptInfo was already created during
* add_base_rels_to_query.
*/
! childrel = !grouped ? find_base_rel(root, childRTindex) :
! find_grouped_base_rel(root, childRTindex);
Assert(childrel->reloptkind == RELOPT_OTHER_MEMBER_REL);
if (rel->part_scheme)
*************** set_append_rel_size(PlannerInfo *root, R
*** 929,938 ****
/*
* We need attr_needed data for building targetlist of a join
* relation representing join between matching partitions for
! * partitionwise join. A given attribute of a child will be
! * needed in the same highest joinrel where the corresponding
! * attribute of parent is needed. Hence it suffices to use the
! * same Relids set for parent and child.
*/
for (attno = rel->min_attr; attno <= rel->max_attr; attno++)
{
--- 1104,1113 ----
/*
* We need attr_needed data for building targetlist of a join
* relation representing join between matching partitions for
! * partitionwise join. A given attribute of a child will be needed
! * in the same highest joinrel where the corresponding attribute
! * of parent is needed. Hence it suffices to use the same Relids
! * set for parent and child.
*/
for (attno = rel->min_attr; attno <= rel->max_attr; attno++)
{
*************** set_append_rel_size(PlannerInfo *root, R
*** 982,991 ****
* PlaceHolderVars.) XXX we do not bother to update the cost or width
* fields of childrel->reltarget; not clear if that would be useful.
*/
! childrel->reltarget->exprs = (List *)
! adjust_appendrel_attrs(root,
! (Node *) rel->reltarget->exprs,
! 1, &appinfo);
/*
* We have to make child entries in the EquivalenceClass data
--- 1157,1190 ----
* PlaceHolderVars.) XXX we do not bother to update the cost or width
* fields of childrel->reltarget; not clear if that would be useful.
*/
! if (grouped)
! {
! /*
! * Special attention is needed in the grouped case.
! *
! * build_simple_rel() didn't create empty target because it's
! * better to start with copying one from the parent rel.
! */
! Assert(childrel->reltarget == NULL && childrel->agg_info == NULL);
! Assert(rel->reltarget != NULL && rel->agg_info != NULL);
!
! /*
! * Translate the targets and grouping expressions so they match
! * this child.
! */
! childrel->agg_info = translate_rel_agg_info(root, rel->agg_info,
! &appinfo, 1);
!
! /*
! * The relation paths will generate input for partial aggregation.
! */
! childrel->reltarget = childrel->agg_info->input;
! }
! else
! childrel->reltarget->exprs = (List *)
! adjust_appendrel_attrs(root,
! (Node *) rel->reltarget->exprs,
! 1, &appinfo);
/*
* We have to make child entries in the EquivalenceClass data
*************** set_append_rel_size(PlannerInfo *root, R
*** 1140,1145 ****
--- 1339,1364 ----
1, &appinfo);
/*
+ * We have to make child entries in the EquivalenceClass data
+ * structures as well. This is needed either if the parent
+ * participates in some eclass joins (because we will want to consider
+ * inner-indexscan joins on the individual children) or if the parent
+ * has useful pathkeys (because we should try to build MergeAppend
+ * paths that produce those sort orderings).
+ */
+ if (rel->has_eclass_joins || has_useful_pathkeys(root, rel))
+ add_child_rel_equivalences(root, appinfo, rel, childrel);
+ childrel->has_eclass_joins = rel->has_eclass_joins;
+
+ /*
+ * Note: we could compute appropriate attr_needed data for the child's
+ * variables, by transforming the parent's attr_needed through the
+ * translated_vars mapping. However, currently there's no need
+ * because attr_needed is only examined for base relations not
+ * otherrels. So we just leave the child's attr_needed empty.
+ */
+
+ /*
* If parallelism is allowable for this query in general, see whether
* it's allowable for this childrel in particular. But if we've
* already decided the appendrel is not parallel-safe as a whole,
*************** set_append_rel_pathlist(PlannerInfo *roo
*** 1263,1268 ****
--- 1482,1488 ----
int parentRTindex = rti;
List *live_childrels = NIL;
ListCell *l;
+ bool grouped = rel->agg_info != NULL;
/*
* Generate access paths for each member relation, and remember the
*************** set_append_rel_pathlist(PlannerInfo *roo
*** 1282,1288 ****
/* Re-locate the child RTE and RelOptInfo */
childRTindex = appinfo->child_relid;
childRTE = root->simple_rte_array[childRTindex];
! childrel = root->simple_rel_array[childRTindex];
/*
* If set_append_rel_size() decided the parent appendrel was
--- 1502,1510 ----
/* Re-locate the child RTE and RelOptInfo */
childRTindex = appinfo->child_relid;
childRTE = root->simple_rte_array[childRTindex];
! childrel = !grouped ?
! find_base_rel(root, childRTindex) :
! find_grouped_base_rel(root, childRTindex);
/*
* If set_append_rel_size() decided the parent appendrel was
*************** generate_mergeappend_paths(PlannerInfo *
*** 1732,1737 ****
--- 1954,1960 ----
List *partitioned_rels)
{
ListCell *lcp;
+ PathTarget *target = NULL;
foreach(lcp, all_child_pathkeys)
{
*************** generate_mergeappend_paths(PlannerInfo *
*** 1740,1762 ****
List *total_subpaths = NIL;
bool startup_neq_total = false;
ListCell *lcr;
/* Select the child paths for this ordering... */
foreach(lcr, live_childrels)
{
RelOptInfo *childrel = (RelOptInfo *) lfirst(lcr);
Path *cheapest_startup,
*cheapest_total;
/* Locate the right paths, if they are available. */
cheapest_startup =
! get_cheapest_path_for_pathkeys(childrel->pathlist,
pathkeys,
NULL,
STARTUP_COST,
false);
cheapest_total =
! get_cheapest_path_for_pathkeys(childrel->pathlist,
pathkeys,
NULL,
TOTAL_COST,
--- 1963,1987 ----
List *total_subpaths = NIL;
bool startup_neq_total = false;
ListCell *lcr;
+ Path *path;
/* Select the child paths for this ordering... */
foreach(lcr, live_childrels)
{
RelOptInfo *childrel = (RelOptInfo *) lfirst(lcr);
+ List *pathlist = childrel->pathlist;
Path *cheapest_startup,
*cheapest_total;
/* Locate the right paths, if they are available. */
cheapest_startup =
! get_cheapest_path_for_pathkeys(pathlist,
pathkeys,
NULL,
STARTUP_COST,
false);
cheapest_total =
! get_cheapest_path_for_pathkeys(pathlist,
pathkeys,
NULL,
TOTAL_COST,
*************** generate_mergeappend_paths(PlannerInfo *
*** 1789,1807 ****
}
/* ... and build the MergeAppend paths */
! add_path(rel, (Path *) create_merge_append_path(root,
! rel,
! startup_subpaths,
! pathkeys,
! NULL,
! partitioned_rels));
if (startup_neq_total)
! add_path(rel, (Path *) create_merge_append_path(root,
! rel,
! total_subpaths,
! pathkeys,
! NULL,
! partitioned_rels));
}
}
--- 2014,2041 ----
}
/* ... and build the MergeAppend paths */
! path = (Path *) create_merge_append_path(root,
! rel,
! target,
! startup_subpaths,
! pathkeys,
! NULL,
! partitioned_rels);
!
! add_path(rel, path);
!
if (startup_neq_total)
! {
! path = (Path *) create_merge_append_path(root,
! rel,
! target,
! total_subpaths,
! pathkeys,
! NULL,
! partitioned_rels);
! add_path(rel, path);
! }
!
}
}
*************** generate_gather_paths(PlannerInfo *root,
*** 2508,2518 ****
* See comments for deconstruct_jointree() for definition of the joinlist
* data structure.
*/
! static RelOptInfo *
make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
{
int levels_needed;
! List *initial_rels;
ListCell *jl;
/*
--- 2742,2753 ----
* See comments for deconstruct_jointree() for definition of the joinlist
* data structure.
*/
! static JoinSearchResult *
make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
{
int levels_needed;
! List *initial_rels,
! *initial_rels_grouped;
ListCell *jl;
/*
*************** make_rel_from_joinlist(PlannerInfo *root
*** 2531,2568 ****
* sub-joinlists.
*/
initial_rels = NIL;
foreach(jl, joinlist)
{
Node *jlnode = (Node *) lfirst(jl);
! RelOptInfo *thisrel;
if (IsA(jlnode, RangeTblRef))
{
int varno = ((RangeTblRef *) jlnode)->rtindex;
thisrel = find_base_rel(root, varno);
}
else if (IsA(jlnode, List))
{
/* Recurse to handle subproblem */
! thisrel = make_rel_from_joinlist(root, (List *) jlnode);
}
else
{
elog(ERROR, "unrecognized joinlist node type: %d",
(int) nodeTag(jlnode));
thisrel = NULL; /* keep compiler quiet */
}
initial_rels = lappend(initial_rels, thisrel);
}
if (levels_needed == 1)
{
/*
* Single joinlist node, so we're done.
*/
! return (RelOptInfo *) linitial(initial_rels);
}
else
{
--- 2766,2817 ----
* sub-joinlists.
*/
initial_rels = NIL;
+ initial_rels_grouped = NIL;
foreach(jl, joinlist)
{
Node *jlnode = (Node *) lfirst(jl);
! RelOptInfo *thisrel,
! *thisrel_grouped;
if (IsA(jlnode, RangeTblRef))
{
int varno = ((RangeTblRef *) jlnode)->rtindex;
thisrel = find_base_rel(root, varno);
+ thisrel_grouped = find_grouped_base_rel(root, varno);
}
else if (IsA(jlnode, List))
{
+ JoinSearchResult *rels;
+
/* Recurse to handle subproblem */
! rels = make_rel_from_joinlist(root, (List *) jlnode);
! thisrel = rels->plain;
! thisrel_grouped = rels->grouped;
}
else
{
elog(ERROR, "unrecognized joinlist node type: %d",
(int) nodeTag(jlnode));
thisrel = NULL; /* keep compiler quiet */
+ thisrel_grouped = NULL;
}
initial_rels = lappend(initial_rels, thisrel);
+ initial_rels_grouped = lappend(initial_rels_grouped, thisrel_grouped);
}
if (levels_needed == 1)
{
+ JoinSearchResult *result;
+
/*
* Single joinlist node, so we're done.
*/
! result = (JoinSearchResult *) palloc(sizeof(JoinSearchResult));
! result->plain = (RelOptInfo *) linitial(initial_rels);
! result->grouped = (RelOptInfo *) linitial(initial_rels_grouped);
! return result;
}
else
{
*************** make_rel_from_joinlist(PlannerInfo *root
*** 2576,2586 ****
root->initial_rels = initial_rels;
if (join_search_hook)
! return (*join_search_hook) (root, levels_needed, initial_rels);
else if (enable_geqo && levels_needed >= geqo_threshold)
! return geqo(root, levels_needed, initial_rels);
else
! return standard_join_search(root, levels_needed, initial_rels);
}
}
--- 2825,2849 ----
root->initial_rels = initial_rels;
if (join_search_hook)
! return (*join_search_hook) (root, levels_needed,
! initial_rels,
! initial_rels_grouped);
else if (enable_geqo && levels_needed >= geqo_threshold)
! {
! JoinSearchResult *result;
!
! /*
! * TODO Teach GEQO about grouped relations. Don't forget that
! * pathlist can be NIL before set_cheapest() gets called.
! */
! result = (JoinSearchResult *) palloc0(sizeof(JoinSearchResult));
! result->plain = geqo(root, levels_needed, initial_rels);
! return result;
! }
else
! return standard_join_search(root, levels_needed,
! initial_rels,
! initial_rels_grouped);
}
}
*************** make_rel_from_joinlist(PlannerInfo *root
*** 2596,2601 ****
--- 2859,2868 ----
* jointree item. These are the components to be joined together.
* Note that levels_needed == list_length(initial_rels).
*
+ * 'initial_rels_grouped' is a list where i-th position is either RelOptInfo
+ * representing i-th item of 'initial_rels' grouped or NULL if there's no
+ * such grouped relation.
+ *
* Returns the final level of join relations, i.e., the relation that is
* the result of joining all the original relations together.
* At least one implementation path must be provided for this relation and
*************** make_rel_from_joinlist(PlannerInfo *root
*** 2613,2629 ****
* than one join-order search, you'll probably need to save and restore the
* original states of those data structures. See geqo_eval() for an example.
*/
! RelOptInfo *
! standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
{
int lev;
! RelOptInfo *rel;
/*
* This function cannot be invoked recursively within any one planning
! * problem, so join_rel_level[] can't be in use already.
*/
Assert(root->join_rel_level == NULL);
/*
* We employ a simple "dynamic programming" algorithm: we first find all
--- 2880,2901 ----
* than one join-order search, you'll probably need to save and restore the
* original states of those data structures. See geqo_eval() for an example.
*/
! JoinSearchResult *
! standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels,
! List *initial_rels_grouped)
{
int lev;
! Size levels_size;
! List *top_list;
! JoinSearchResult *result;
/*
* This function cannot be invoked recursively within any one planning
! * problem, so join_rel_level[] / join_grouped_rel_level[] can't be in use
! * already.
*/
Assert(root->join_rel_level == NULL);
+ Assert(root->join_grouped_rel_level == NULL);
/*
* We employ a simple "dynamic programming" algorithm: we first find all
*************** standard_join_search(PlannerInfo *root,
*** 2636,2671 ****
* set root->join_rel_level[1] to represent all the single-jointree-item
* relations.
*/
! root->join_rel_level = (List **) palloc0((levels_needed + 1) * sizeof(List *));
!
root->join_rel_level[1] = initial_rels;
for (lev = 2; lev <= levels_needed; lev++)
{
ListCell *lc;
/*
* Determine all possible pairs of relations to be joined at this
* level, and build paths for making each one from every available
* pair of lower-level relations.
*/
join_search_one_level(root, lev);
/*
! * Run generate_partitionwise_join_paths() and
! * generate_gather_paths() for each just-processed joinrel. We could
! * not do this earlier because both regular and partial paths can get
! * added to a particular joinrel at multiple times within
! * join_search_one_level.
*
* After that, we're done creating paths for the joinrel, so run
* set_cheapest().
*/
! foreach(lc, root->join_rel_level[lev])
{
! rel = (RelOptInfo *) lfirst(lc);
! /* Create paths for partitionwise joins. */
generate_partitionwise_join_paths(root, rel);
/* Create GatherPaths for any useful partial paths for rel */
--- 2908,2952 ----
* set root->join_rel_level[1] to represent all the single-jointree-item
* relations.
*/
! levels_size = (levels_needed + 1) * sizeof(List *);
! root->join_rel_level = (List **) palloc0(levels_size);
root->join_rel_level[1] = initial_rels;
+ root->join_grouped_rel_level = (List **) palloc0(levels_size);
+ root->join_grouped_rel_level[1] = initial_rels_grouped;
for (lev = 2; lev <= levels_needed; lev++)
{
+ List *levels;
ListCell *lc;
/*
* Determine all possible pairs of relations to be joined at this
* level, and build paths for making each one from every available
* pair of lower-level relations.
+ *
+ * This step includes creation of grouped relations.
*/
join_search_one_level(root, lev);
/*
! * Run generate_partitionwise_join_paths() and generate_gather_paths()
! * for each just-processed joinrel. We could not do this earlier
! * because both regular and partial paths can get added to a
! * particular joinrel at multiple times within join_search_one_level.
*
* After that, we're done creating paths for the joinrel, so run
* set_cheapest().
+ *
+ * This processing makes no difference betweend plain and grouped
+ * rels, so process them in the same loop.
*/
! levels = list_concat(list_copy(root->join_rel_level[lev]),
! root->join_grouped_rel_level[lev]);
! foreach(lc, levels)
{
! RelOptInfo *rel = lfirst_node(RelOptInfo, lc);
! /* Create paths for partition-wise joins. */
generate_partitionwise_join_paths(root, rel);
/* Create GatherPaths for any useful partial paths for rel */
*************** standard_join_search(PlannerInfo *root,
*** 2681,2697 ****
}
/*
! * We should have a single rel at the final level.
*/
! if (root->join_rel_level[levels_needed] == NIL)
elog(ERROR, "failed to build any %d-way joins", levels_needed);
! Assert(list_length(root->join_rel_level[levels_needed]) == 1);
! rel = (RelOptInfo *) linitial(root->join_rel_level[levels_needed]);
root->join_rel_level = NULL;
! return rel;
}
/*****************************************************************************
--- 2962,2989 ----
}
/*
! * We should have a single plain rel at the final level.
*/
! if ((top_list = root->join_rel_level[levels_needed]) == NIL)
elog(ERROR, "failed to build any %d-way joins", levels_needed);
! Assert(list_length(top_list) == 1);
! result = (JoinSearchResult *) palloc0(sizeof(JoinSearchResult));
! result->plain = linitial_node(RelOptInfo, top_list);
!
! /*
! * Grouped relation might have been created too.
! */
! if ((top_list = root->join_grouped_rel_level[levels_needed]) != NIL)
! {
! Assert(list_length(top_list) == 1);
! result->grouped = linitial_node(RelOptInfo, top_list);
! }
root->join_rel_level = NULL;
+ root->join_grouped_rel_level = NULL;
! return result;
}
/*****************************************************************************
*************** create_partial_bitmap_paths(PlannerInfo
*** 3311,3316 ****
--- 3603,3609 ----
{
int parallel_workers;
double pages_fetched;
+ Path *bmhpath;
/* Compute heap pages for bitmap heap scan */
pages_fetched = compute_bitmap_pages(root, rel, bitmapqual, 1.0,
*************** create_partial_bitmap_paths(PlannerInfo
*** 3322,3329 ****
if (parallel_workers <= 0)
return;
! add_partial_path(rel, (Path *) create_bitmap_heap_path(root, rel,
! bitmapqual, rel->lateral_relids, 1.0, parallel_workers));
}
/*
--- 3615,3635 ----
if (parallel_workers <= 0)
return;
! bmhpath = (Path *) create_bitmap_heap_path(root, rel, bitmapqual,
! rel->lateral_relids, 1.0,
! parallel_workers);
!
! if (rel->agg_info == NULL)
! add_partial_path(rel, bmhpath);
! else
! {
! /*
! * Only AGG_HASHED is suitable here as it does not expect the input
! * set to be sorted.
! */
! create_grouped_path(root, rel, (Path *) bmhpath, false, true,
! AGG_HASHED);
! }
}
/*
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
new file mode 100644
index d8db0b2..14146bd
*** a/src/backend/optimizer/path/costsize.c
--- b/src/backend/optimizer/path/costsize.c
***************
*** 91,96 ****
--- 91,97 ----
#include "optimizer/plancat.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+ #include "optimizer/var.h"
#include "parser/parsetree.h"
#include "utils/lsyscache.h"
#include "utils/selfuncs.h"
*************** cost_bitmap_tree_node(Path *path, Cost *
*** 1065,1070 ****
--- 1066,1082 ----
*cost = path->total_cost;
*selec = ((BitmapOrPath *) path)->bitmapselectivity;
}
+ else if (IsA(path, AggPath))
+ {
+ /*
+ * If partial aggregation was already applied, use only the input
+ * path.
+ *
+ * TODO Take the aggregation into account, both cost and its effect on
+ * selectivity (i.e. how it reduces the number of rows).
+ */
+ cost_bitmap_tree_node(((AggPath *) path)->subpath, cost, selec);
+ }
else
{
elog(ERROR, "unrecognized node type: %d", nodeTag(path));
*************** cost_group(Path *path, PlannerInfo *root
*** 2287,2292 ****
--- 2299,2330 ----
path->total_cost = total_cost;
}
+ static void
+ estimate_join_rows(PlannerInfo *root, Path *path, RelAggInfo *agg_info)
+ {
+ bool grouped = agg_info != NULL;
+
+ if (path->param_info)
+ {
+ double nrows;
+
+ path->rows = path->param_info->ppi_rows;
+ if (grouped)
+ {
+ nrows = estimate_num_groups(root, agg_info->group_exprs,
+ path->rows, NULL);
+ path->rows = clamp_row_est(nrows);
+ }
+ }
+ else
+ {
+ if (!grouped)
+ path->rows = path->parent->rows;
+ else
+ path->rows = agg_info->rows;
+ }
+ }
+
/*
* initial_cost_nestloop
* Preliminary estimate of the cost of a nestloop join path.
*************** final_cost_nestloop(PlannerInfo *root, N
*** 2408,2417 ****
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
! if (path->path.param_info)
! path->path.rows = path->path.param_info->ppi_rows;
! else
! path->path.rows = path->path.parent->rows;
/* For partial paths, scale row estimate. */
if (path->path.parallel_workers > 0)
--- 2446,2452 ----
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
! estimate_join_rows(root, (Path *) path, path->path.parent->agg_info);
/* For partial paths, scale row estimate. */
if (path->path.parallel_workers > 0)
*************** final_cost_mergejoin(PlannerInfo *root,
*** 2854,2863 ****
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
! if (path->jpath.path.param_info)
! path->jpath.path.rows = path->jpath.path.param_info->ppi_rows;
! else
! path->jpath.path.rows = path->jpath.path.parent->rows;
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
--- 2889,2896 ----
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
! estimate_join_rows(root, (Path *) path,
! path->jpath.path.parent->agg_info);
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
*************** final_cost_hashjoin(PlannerInfo *root, H
*** 3277,3286 ****
ListCell *hcl;
/* Mark the path with the correct row estimate */
! if (path->jpath.path.param_info)
! path->jpath.path.rows = path->jpath.path.param_info->ppi_rows;
! else
! path->jpath.path.rows = path->jpath.path.parent->rows;
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
--- 3310,3317 ----
ListCell *hcl;
/* Mark the path with the correct row estimate */
! estimate_join_rows(root, (Path *) path,
! path->jpath.path.parent->agg_info);
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
*************** cost_qual_eval_walker(Node *node, cost_q
*** 3803,3810 ****
* estimated execution cost given by pg_proc.procost (remember to multiply
* this by cpu_operator_cost).
*
! * Vars and Consts are charged zero, and so are boolean operators (AND,
! * OR, NOT). Simplistic, but a lot better than no model at all.
*
* Should we try to account for the possibility of short-circuit
* evaluation of AND/OR? Probably *not*, because that would make the
--- 3834,3842 ----
* estimated execution cost given by pg_proc.procost (remember to multiply
* this by cpu_operator_cost).
*
! * Vars, GroupedVars and Consts are charged zero, and so are boolean
! * operators (AND, OR, NOT). Simplistic, but a lot better than no model at
! * all.
*
* Should we try to account for the possibility of short-circuit
* evaluation of AND/OR? Probably *not*, because that would make the
*************** approx_tuple_count(PlannerInfo *root, Jo
*** 4283,4293 ****
--- 4315,4327 ----
* restriction clauses).
* width: the estimated average output tuple width in bytes.
* baserestrictcost: estimated cost of evaluating baserestrictinfo clauses.
+ * grouped: will partial aggregation be applied to each path?
*/
void
set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel)
{
double nrows;
+ bool grouped = rel->agg_info != NULL;
/* Should only be applied to base relations */
Assert(rel->relid > 0);
*************** set_baserel_size_estimates(PlannerInfo *
*** 4298,4309 ****
0,
JOIN_INNER,
NULL);
-
rel->rows = clamp_row_est(nrows);
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
! set_rel_width(root, rel);
}
/*
--- 4332,4362 ----
0,
JOIN_INNER,
NULL);
rel->rows = clamp_row_est(nrows);
+ /*
+ * Grouping essentially changes the number of rows.
+ */
+ if (grouped)
+ {
+ nrows = estimate_num_groups(root,
+ rel->agg_info->group_exprs, nrows,
+ NULL);
+ rel->agg_info->rows = clamp_row_est(nrows);
+ }
+
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
! /*
! * The grouped target should have the cost and width set immediately on
! * creation, see create_rel_agg_info().
! */
! if (!grouped)
! set_rel_width(root, rel);
! #ifdef USE_ASSERT_CHECKING
! else
! Assert(rel->reltarget->width > 0);
! #endif
}
/*
*************** set_pathtarget_cost_width(PlannerInfo *r
*** 5285,5290 ****
--- 5338,5358 ----
Assert(item_width > 0);
tuple_width += item_width;
}
+ else if (IsA(node, GroupedVar))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+ GroupedVarInfo *gvinfo;
+
+ gvinfo = find_grouped_var_info(root, gvar);
+ tuple_width += gvinfo->gv_width;
+
+ /*
+ * Only AggPath can evaluate GroupedVar, whether it's an aggregate
+ * or generic grouping expression. In the other cases the
+ * GroupedVar we see here only bubbled up from a lower AggPath, so
+ * it does not add any cost to the path that owns this target.
+ */
+ }
else
{
/*
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
new file mode 100644
index 70a925c..6c0a33f
*** a/src/backend/optimizer/path/equivclass.c
--- b/src/backend/optimizer/path/equivclass.c
*************** static bool reconsider_outer_join_clause
*** 65,70 ****
--- 65,83 ----
static bool reconsider_full_join_clause(PlannerInfo *root,
RestrictInfo *rinfo);
+ typedef struct translate_expr_context
+ {
+ Var **keys; /* Dictionary keys. */
+ Var **values; /* Dictionary values */
+ int nitems; /* Number of dictionary items. */
+ Relids *gv_eval_at_p; /* See GroupedVarInfo. */
+ Index relid; /* Translate into this relid. */
+ } translate_expr_context;
+
+ static Node *translate_expression_to_rels_mutator(Node *node,
+ translate_expr_context *context);
+ static int var_dictionary_comparator(const void *a, const void *b);
+
/*
* process_equivalence
*************** is_redundant_derived_clause(RestrictInfo
*** 2510,2512 ****
--- 2523,2844 ----
return false;
}
+
+ /*
+ * translate_expression_to_rels
+ * If the appropriate equivalence classes exist, replace vars in
+ * gvi->gvexpr with vars whose varno is equal to relid.
+ */
+ GroupedVarInfo *
+ translate_expression_to_rels(PlannerInfo *root, GroupedVarInfo *gvi,
+ Index relid)
+ {
+ List *vars;
+ ListCell *l1;
+ int i,
+ j;
+ int nkeys,
+ nkeys_resolved;
+ Var **keys,
+ **values,
+ **keys_tmp;
+ Var *key,
+ *key_prev;
+ translate_expr_context context;
+ GroupedVarInfo *result;
+
+ /* Can't do anything w/o equivalence classes. */
+ if (root->eq_classes == NIL)
+ return NULL;
+
+ /*
+ * Before actually trying to modify the expression tree, find out if all
+ * vars can be translated.
+ */
+ vars = pull_var_clause((Node *) gvi->gvexpr, PVC_RECURSE_AGGREGATES);
+
+ /* No vars to translate? */
+ if (vars == NIL)
+ return NULL;
+
+ /*
+ * Search for individual replacement vars as well as the actual expression
+ * translation will be more efficient if we use a dictionary with the keys
+ * (i.e. the "source vars") unique and sorted.
+ */
+ nkeys = list_length(vars);
+ keys = (Var **) palloc(nkeys * sizeof(Var *));
+ i = 0;
+ foreach(l1, vars)
+ {
+ key = lfirst_node(Var, l1);
+ keys[i++] = key;
+ }
+
+ /*
+ * Sort the keys by varno. varattno decides where varnos are equal.
+ */
+ if (nkeys > 1)
+ pg_qsort(keys, nkeys, sizeof(Var *), var_dictionary_comparator);
+
+ /*
+ * Pick unique values and get rid of the vars that need no translation.
+ */
+ keys_tmp = (Var **) palloc(nkeys * sizeof(Var *));
+ key_prev = NULL;
+ j = 0;
+ for (i = 0; i < nkeys; i++)
+ {
+ key = keys[i];
+
+ if ((key_prev == NULL || (key->varno != key_prev->varno &&
+ key->varattno != key_prev->varattno)) &&
+ key->varno != relid)
+ keys_tmp[j++] = key;
+
+ key_prev = key;
+ }
+ pfree(keys);
+ keys = keys_tmp;
+ nkeys = j;
+
+ /*
+ * Is there actually nothing to be translated?
+ */
+ if (nkeys == 0)
+ {
+ pfree(keys);
+ return NULL;
+ }
+
+ nkeys_resolved = 0;
+
+ /*
+ * Find the replacement vars.
+ */
+ values = (Var **) palloc0(nkeys * sizeof(Var *));
+ foreach(l1, root->eq_classes)
+ {
+ EquivalenceClass *ec = lfirst_node(EquivalenceClass, l1);
+ Relids ec_var_relids;
+ Var **ec_vars;
+ int ec_nvars;
+ ListCell *l2;
+
+ /* TODO Re-check if any other EC kind should be ignored. */
+ if (ec->ec_has_volatile || ec->ec_below_outer_join || ec->ec_broken)
+ continue;
+
+ /* Single-element EC can hardly help in translations. */
+ if (list_length(ec->ec_members) == 1)
+ continue;
+
+ /*
+ * Collect all vars of this EC and their varnos.
+ *
+ * ec->ec_relids does not help because we're only interested in a
+ * subset of EC members.
+ */
+ ec_vars = (Var **) palloc(list_length(ec->ec_members) * sizeof(Var *));
+ ec_nvars = 0;
+ ec_var_relids = NULL;
+ foreach(l2, ec->ec_members)
+ {
+ EquivalenceMember *em = lfirst_node(EquivalenceMember, l2);
+ Var *ec_var;
+
+ if (!IsA(em->em_expr, Var))
+ continue;
+
+ ec_var = castNode(Var, em->em_expr);
+ ec_vars[ec_nvars++] = ec_var;
+ ec_var_relids = bms_add_member(ec_var_relids, ec_var->varno);
+ }
+
+ /*
+ * At least two vars are needed so that the EC is usable for
+ * translation.
+ */
+ if (ec_nvars <= 1)
+ {
+ pfree(ec_vars);
+ bms_free(ec_var_relids);
+ continue;
+ }
+
+ /*
+ * Now check where this EC can help.
+ */
+ for (i = 0; i < nkeys; i++)
+ {
+ Relids ec_rest;
+ bool relid_ok,
+ key_found;
+ Var *key = keys[i];
+ Var *value = values[i];
+
+ /* Skip this item if it's already resolved. */
+ if (value != NULL)
+ continue;
+
+ /*
+ * Can't translate if the EC does not mention key->varno.
+ */
+ if (!bms_is_member(key->varno, ec_var_relids))
+ continue;
+
+ /*
+ * Besides key, at least one EC member must belong to the relation
+ * we're translating our expression to.
+ */
+ ec_rest = bms_copy(ec_var_relids);
+ ec_rest = bms_del_member(ec_rest, key->varno);
+ relid_ok = bms_is_member(relid, ec_rest);
+ bms_free(ec_rest);
+ if (!relid_ok)
+ continue;
+
+ /*
+ * The preliminary checks passed, so try to find the exact vars.
+ */
+ key_found = false;
+ for (j = 0; j < ec_nvars; j++)
+ {
+ Var *ec_var = ec_vars[j];
+
+ if (!key_found && key->varno == ec_var->varno &&
+ key->varattno == ec_var->varattno)
+ key_found = true;
+
+ /*
+ *
+ * Is this Var useful for our dictionary?
+ *
+ * XXX Shouldn't ec_var be copied?
+ */
+ if (value == NULL && ec_var->varno == relid)
+ value = ec_var;
+
+ if (key_found && value != NULL)
+ break;
+ }
+
+ if (key_found && value != NULL)
+ {
+ values[i] = value;
+ nkeys_resolved++;
+
+ if (nkeys_resolved == nkeys)
+ break;
+ }
+ }
+
+ pfree(ec_vars);
+ bms_free(ec_var_relids);
+
+ /* Don't need to check the remaining ECs? */
+ if (nkeys_resolved == nkeys)
+ break;
+ }
+
+ /* Couldn't compose usable dictionary? */
+ if (nkeys_resolved < nkeys)
+ {
+ pfree(keys);
+ pfree(values);
+ return NULL;
+ }
+
+ result = makeNode(GroupedVarInfo);
+ memcpy(result, gvi, sizeof(GroupedVarInfo));
+
+ /*
+ * translate_expression_to_rels_mutator updates gv_eval_at.
+ */
+ result->gv_eval_at = bms_copy(result->gv_eval_at);
+
+ /* The dictionary is ready, so perform the translation. */
+ context.keys = keys;
+ context.values = values;
+ context.nitems = nkeys;
+ context.gv_eval_at_p = &result->gv_eval_at;
+ context.relid = relid;
+ result->gvexpr = (Expr *)
+ translate_expression_to_rels_mutator((Node *) gvi->gvexpr, &context);
+
+ pfree(keys);
+ pfree(values);
+ return result;
+ }
+
+ static Node *
+ translate_expression_to_rels_mutator(Node *node,
+ translate_expr_context *context)
+ {
+ if (node == NULL)
+ return NULL;
+
+ if (IsA(node, Var))
+ {
+ Var *var = castNode(Var, node);
+ Var **key_p;
+ Var *value;
+ int index;
+
+ /*
+ * Simply return the existing variable if already belongs to the
+ * relation we're adjusting the expression to.
+ */
+ if (var->varno == context->relid)
+ return (Node *) var;
+
+ key_p = bsearch(&var, context->keys, context->nitems, sizeof(Var *),
+ var_dictionary_comparator);
+
+ /* We shouldn't have omitted any var from the dictionary. */
+ Assert(key_p != NULL);
+
+ index = key_p - context->keys;
+ Assert(index >= 0 && index < context->nitems);
+ value = context->values[index];
+
+ /* All values should be present in the dictionary. */
+ Assert(value != NULL);
+
+ /* Update gv_eval_at accordingly. */
+ bms_del_member(*context->gv_eval_at_p, var->varno);
+ *context->gv_eval_at_p = bms_add_member(*context->gv_eval_at_p,
+ value->varno);
+
+ return (Node *) value;
+ }
+
+ return expression_tree_mutator(node, translate_expression_to_rels_mutator,
+ (void *) context);
+ }
+
+ static int
+ var_dictionary_comparator(const void *a, const void *b)
+ {
+ Var **var1_p,
+ **var2_p;
+ Var *var1,
+ *var2;
+
+ var1_p = (Var **) a;
+ var1 = castNode(Var, *var1_p);
+ var2_p = (Var **) b;
+ var2 = castNode(Var, *var2_p);
+
+ if (var1->varno < var2->varno)
+ return -1;
+ else if (var1->varno > var2->varno)
+ return 1;
+
+ if (var1->varattno < var2->varattno)
+ return -1;
+ else if (var1->varattno > var2->varattno)
+ return 1;
+
+ return 0;
+ }
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
new file mode 100644
index 7fc7080..1e05faf
*** a/src/backend/optimizer/path/indxpath.c
--- b/src/backend/optimizer/path/indxpath.c
***************
*** 32,37 ****
--- 32,38 ----
#include "optimizer/predtest.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "utils/builtins.h"
#include "utils/bytea.h"
*************** typedef struct
*** 78,84 ****
int indexcol; /* index column we want to match to */
} ec_member_matches_arg;
-
static void consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index,
IndexClauseSet *rclauseset,
--- 79,84 ----
*************** static Const *string_to_const(const char
*** 227,232 ****
--- 227,234 ----
* index quals ... but for now, it doesn't seem worth troubling over.
* In particular, comments below about "unparameterized" paths should be read
* as meaning "unparameterized so far as the indexquals are concerned".
+ *
+ * If agg_info is passed, grouped paths are generated too.
*/
void
create_index_paths(PlannerInfo *root, RelOptInfo *rel)
*************** create_index_paths(PlannerInfo *root, Re
*** 274,281 ****
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
! get_index_paths(root, rel, index, &rclauseset,
! &bitindexpaths);
/*
* Identify the join clauses that can match the index. For the moment
--- 276,282 ----
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
! get_index_paths(root, rel, index, &rclauseset, &bitindexpaths);
/*
* Identify the join clauses that can match the index. For the moment
*************** create_index_paths(PlannerInfo *root, Re
*** 307,318 ****
&bitjoinpaths);
}
/*
* Generate BitmapOrPaths for any suitable OR-clauses present in the
* restriction list. Add these to bitindexpaths.
*/
! indexpaths = generate_bitmap_or_paths(root, rel,
! rel->baserestrictinfo, NIL);
bitindexpaths = list_concat(bitindexpaths, indexpaths);
/*
--- 308,328 ----
&bitjoinpaths);
}
+
+ /*
+ * Bitmap paths are currently not aggregated: AggPath does not accept the
+ * TID bitmap as input, and even if it did, it'd seem weird to aggregate
+ * the individual paths and then AND them together.
+ */
+ if (rel->agg_info != NULL)
+ return;
+
/*
* Generate BitmapOrPaths for any suitable OR-clauses present in the
* restriction list. Add these to bitindexpaths.
*/
! indexpaths = generate_bitmap_or_paths(root, rel, rel->baserestrictinfo,
! NIL);
bitindexpaths = list_concat(bitindexpaths, indexpaths);
/*
*************** bms_equal_any(Relids relids, List *relid
*** 717,723 ****
return false;
}
-
/*
* get_index_paths
* Given an index and a set of index clauses for it, construct IndexPaths.
--- 727,732 ----
*************** get_index_paths(PlannerInfo *root, RelOp
*** 748,753 ****
--- 757,766 ----
* clauses only if the index AM supports them natively, and skip any such
* clauses for index columns after the first (so that we produce ordered
* paths if possible).
+ *
+ * These paths are good candidates for AGG_SORTED, so pass the output
+ * lists for this strategy. AGG_HASHED should be applied to paths with no
+ * pathkeys.
*/
indexpaths = build_index_paths(root, rel,
index, clauses,
*************** get_index_paths(PlannerInfo *root, RelOp
*** 760,765 ****
--- 773,781 ----
* If we skipped any lower-order ScalarArrayOpExprs on an index with an AM
* that supports them, then try again including those clauses. This will
* produce paths with more selectivity but no ordering.
+ *
+ * As for the grouping paths, only AGG_HASHED is considered due to the
+ * missing ordering.
*/
if (skip_lower_saop)
{
*************** get_index_paths(PlannerInfo *root, RelOp
*** 801,806 ****
--- 817,825 ----
* If there were ScalarArrayOpExpr clauses that the index can't handle
* natively, generate bitmap scan paths relying on executor-managed
* ScalarArrayOpExpr.
+ *
+ * As for grouping, only AGG_HASHED is possible here. Again, because
+ * there's no ordering.
*/
if (skip_nonnative_saop)
{
*************** get_index_paths(PlannerInfo *root, RelOp
*** 847,859 ****
* NULL, we do not ignore non-first ScalarArrayOpExpr clauses, but they will
* result in considering the scan's output to be unordered.
*
* 'rel' is the index's heap relation
* 'index' is the index for which we want to generate paths
* 'clauses' is the collection of indexable clauses (RestrictInfo nodes)
* 'useful_predicate' indicates whether the index has a useful predicate
* 'scantype' indicates whether we need plain or bitmap scan support
* 'skip_nonnative_saop' indicates whether to accept SAOP if index AM doesn't
! * 'skip_lower_saop' indicates whether to accept non-first-column SAOP
*/
static List *
build_index_paths(PlannerInfo *root, RelOptInfo *rel,
--- 866,883 ----
* NULL, we do not ignore non-first ScalarArrayOpExpr clauses, but they will
* result in considering the scan's output to be unordered.
*
+ * If 'agg_info' is passed, 'agg_sorted' and / or 'agg_hashed' must be passed
+ * too. In that case AGG_SORTED and / or AGG_HASHED aggregation is applied to
+ * the index path (as long as the index path is appropriate) and the resulting
+ * grouped path is stored here.
+ *
* 'rel' is the index's heap relation
* 'index' is the index for which we want to generate paths
* 'clauses' is the collection of indexable clauses (RestrictInfo nodes)
* 'useful_predicate' indicates whether the index has a useful predicate
* 'scantype' indicates whether we need plain or bitmap scan support
* 'skip_nonnative_saop' indicates whether to accept SAOP if index AM doesn't
! * 'skip_lower_saop' indicates whether to accept non-first-column SAOP.
*/
static List *
build_index_paths(PlannerInfo *root, RelOptInfo *rel,
*************** build_index_paths(PlannerInfo *root, Rel
*** 878,883 ****
--- 902,913 ----
bool index_is_ordered;
bool index_only_scan;
int indexcol;
+ bool grouped;
+ bool can_agg_sorted,
+ can_agg_hashed;
+ AggPath *agg_path;
+
+ grouped = rel->agg_info != NULL;
/*
* Check that index supports the desired scan type(s)
*************** build_index_paths(PlannerInfo *root, Rel
*** 1031,1037 ****
--- 1061,1072 ----
* in the current clauses, OR the index ordering is potentially useful for
* later merging or final output ordering, OR the index has a useful
* predicate, OR an index-only scan is possible.
+ *
+ * This is where grouped path start to be considered.
*/
+ can_agg_sorted = true;
+ can_agg_hashed = true;
+
if (index_clauses != NIL || useful_pathkeys != NIL || useful_predicate ||
index_only_scan)
{
*************** build_index_paths(PlannerInfo *root, Rel
*** 1048,1054 ****
outer_relids,
loop_count,
false);
! result = lappend(result, ipath);
/*
* If appropriate, consider parallel index scan. We don't allow
--- 1083,1145 ----
outer_relids,
loop_count,
false);
!
! if (!grouped)
! result = lappend(result, ipath);
! else
! {
! /*
! * Try to create the grouped paths if caller is interested in
! * them.
! */
! if (useful_pathkeys != NIL)
! {
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! true,
! ipath->path.rows);
!
! if (agg_path != NULL)
! result = lappend(result, agg_path);
! else
! {
! /*
! * If ipath could not be used as a source for AGG_SORTED
! * partial aggregation, it probably does not have the
! * appropriate pathkeys. Avoid trying to apply AGG_SORTED
! * to the next index paths because those will have the
! * same pathkeys.
! */
! can_agg_sorted = false;
! }
! }
! else
! can_agg_sorted = false;
!
! /*
! * Hashed aggregation should not be parameterized: the cost of
! * repeated creation of the hashtable (for different parameter
! * values) is probably not worth.
! */
! if (outer_relids != NULL)
! {
! agg_path = create_partial_agg_hashed_path(root,
! (Path *) ipath,
! ipath->path.rows);
!
! if (agg_path != NULL)
! result = lappend(result, agg_path);
! else
! {
! /*
! * If ipath could not be used as a source for AGG_HASHED,
! * we should not expect any other path of the same index
! * to succeed. Avoid wasting the effort next time.
! */
! can_agg_hashed = false;
! }
! }
! }
/*
* If appropriate, consider parallel index scan. We don't allow
*************** build_index_paths(PlannerInfo *root, Rel
*** 1077,1083 ****
* parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath);
else
pfree(ipath);
}
--- 1168,1213 ----
* parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! {
! if (!grouped)
! add_partial_path(rel, (Path *) ipath);
! else
! {
! if (useful_pathkeys != NIL && can_agg_sorted)
! {
! /*
! * No need to check the pathkeys again.
! */
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! false,
! ipath->path.rows);
!
! /*
! * If create_agg_sorted_path succeeded once, it should
! * always do.
! */
! Assert(agg_path != NULL);
!
! add_partial_path(rel, (Path *) agg_path);
! }
!
! if (can_agg_hashed && outer_relids == NULL)
! {
! agg_path = create_partial_agg_hashed_path(root,
! (Path *) ipath,
! ipath->path.rows);
!
! /*
! * If create_agg_hashed_path succeeded once, it should
! * always do.
! */
! Assert(agg_path != NULL);
!
! add_partial_path(rel, (Path *) agg_path);
! }
! }
! }
else
pfree(ipath);
}
*************** build_index_paths(PlannerInfo *root, Rel
*** 1105,1111 ****
outer_relids,
loop_count,
false);
! result = lappend(result, ipath);
/* If appropriate, consider parallel index scan */
if (index->amcanparallel &&
--- 1235,1266 ----
outer_relids,
loop_count,
false);
!
! if (!grouped)
! result = lappend(result, ipath);
! else
! {
! /*
! * As the input set ordering does not matter to AGG_HASHED,
! * only AGG_SORTED makes sense here. (The AGG_HASHED path we'd
! * create here should already exist.)
! *
! * The existing value of can_agg_sorted is not up-to-date for
! * the new pathkeys.
! */
! can_agg_sorted = true;
!
! /* pathkeys are new, so check them. */
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! true,
! ipath->path.rows);
!
! if (agg_path != NULL)
! result = lappend(result, agg_path);
! else
! can_agg_sorted = false;
! }
/* If appropriate, consider parallel index scan */
if (index->amcanparallel &&
*************** build_index_paths(PlannerInfo *root, Rel
*** 1129,1135 ****
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! add_partial_path(rel, (Path *) ipath);
else
pfree(ipath);
}
--- 1284,1309 ----
* using parallel workers, just free it.
*/
if (ipath->path.parallel_workers > 0)
! {
! if (!grouped)
! add_partial_path(rel, (Path *) ipath);
! else
! {
! if (can_agg_sorted)
! {
! /*
! * The non-partial path above should have been
! * created, so no need to check pathkeys.
! */
! agg_path = create_partial_agg_sorted_path(root,
! (Path *) ipath,
! false,
! ipath->path.rows);
! Assert(agg_path != NULL);
! add_partial_path(rel, (Path *) agg_path);
! }
! }
! }
else
pfree(ipath);
}
*************** build_index_paths(PlannerInfo *root, Rel
*** 1164,1169 ****
--- 1338,1344 ----
* 'rel' is the relation for which we want to generate index paths
* 'clauses' is the current list of clauses (RestrictInfo nodes)
* 'other_clauses' is the list of additional upper-level clauses
+ * 'agg_info' indicates that grouped paths should be added to 'agg_hashed'.
*/
static List *
build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
*************** build_paths_for_OR(PlannerInfo *root, Re
*** 1237,1243 ****
match_clauses_to_index(index, other_clauses, &clauseset);
/*
! * Construct paths if possible.
*/
indexpaths = build_index_paths(root, rel,
index, &clauseset,
--- 1412,1419 ----
match_clauses_to_index(index, other_clauses, &clauseset);
/*
! * Construct paths if possible. Forbid partial aggregation even if the
! * relation is grouped --- it'll be applied to the bitmap heap path.
*/
indexpaths = build_index_paths(root, rel,
index, &clauseset,
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
new file mode 100644
index 688f440..0cc6268
*** a/src/backend/optimizer/path/joinpath.c
--- b/src/backend/optimizer/path/joinpath.c
*************** static void try_partial_mergejoin_path(P
*** 48,76 ****
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra);
static void sort_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra);
static void match_unsorted_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra);
static void consider_parallel_nestloop(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra);
static void consider_parallel_mergejoin(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total);
static void hash_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra);
static List *select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
--- 48,82 ----
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate);
static void sort_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra,
! bool do_aggregate);
static void match_unsorted_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra,
! bool do_aggregate);
static void consider_parallel_nestloop(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate);
static void consider_parallel_mergejoin(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total,
! bool do_aggregate);
static void hash_inner_and_outer(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
! JoinType jointype, JoinPathExtraData *extra,
! bool do_aggregate);
static List *select_mergejoin_clauses(PlannerInfo *root,
RelOptInfo *joinrel,
RelOptInfo *outerrel,
*************** static void generate_mergejoin_paths(Pla
*** 87,93 ****
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial);
/*
--- 93,100 ----
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial,
! bool do_aggregate);
/*
*************** static void generate_mergejoin_paths(Pla
*** 112,117 ****
--- 119,127 ----
* however. Path cost estimation code may need to recognize that it's
* dealing with such a case --- the combination of nominal jointype INNER
* with sjinfo->jointype == JOIN_SEMI indicates that.
+ *
+ * agg_info is passed iff partial aggregation should be applied to the join
+ * path.
*/
void
add_paths_to_joinrel(PlannerInfo *root,
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 120,126 ****
RelOptInfo *innerrel,
JoinType jointype,
SpecialJoinInfo *sjinfo,
! List *restrictlist)
{
JoinPathExtraData extra;
bool mergejoin_allowed = true;
--- 130,137 ----
RelOptInfo *innerrel,
JoinType jointype,
SpecialJoinInfo *sjinfo,
! List *restrictlist,
! bool do_aggregate)
{
JoinPathExtraData extra;
bool mergejoin_allowed = true;
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 265,271 ****
*/
if (mergejoin_allowed)
sort_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra);
/*
* 2. Consider paths where the outer relation need not be explicitly
--- 276,282 ----
*/
if (mergejoin_allowed)
sort_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, do_aggregate);
/*
* 2. Consider paths where the outer relation need not be explicitly
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 276,282 ****
*/
if (mergejoin_allowed)
match_unsorted_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra);
#ifdef NOT_USED
--- 287,293 ----
*/
if (mergejoin_allowed)
match_unsorted_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, do_aggregate);
#ifdef NOT_USED
*************** add_paths_to_joinrel(PlannerInfo *root,
*** 303,309 ****
*/
if (enable_hashjoin || jointype == JOIN_FULL)
hash_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra);
/*
* 5. If inner and outer relations are foreign tables (or joins) belonging
--- 314,320 ----
*/
if (enable_hashjoin || jointype == JOIN_FULL)
hash_inner_and_outer(root, joinrel, outerrel, innerrel,
! jointype, &extra, do_aggregate);
/*
* 5. If inner and outer relations are foreign tables (or joins) belonging
*************** try_nestloop_path(PlannerInfo *root,
*** 364,370 ****
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
Relids required_outer;
JoinCostWorkspace workspace;
--- 375,382 ----
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
*************** try_nestloop_path(PlannerInfo *root,
*** 374,379 ****
--- 386,392 ----
Relids outerrelids;
Relids inner_paramrels = PATH_REQ_OUTER(inner_path);
Relids outer_paramrels = PATH_REQ_OUTER(outer_path);
+ bool success = false;
/*
* Paths are parameterized by top-level parents, so run parameterization
*************** try_nestloop_path(PlannerInfo *root,
*** 420,429 ****
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer))
{
/*
* If the inner path is parameterized, it is parameterized by the
* topmost parent of the outer rel, not the outer rel itself. Fix
--- 433,463 ----
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! /*
! * If the join output should be (partially) aggregated, the precheck
! * includes the aggregation and is postponed to create_grouped_path().
! */
! if ((!do_aggregate &&
! add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer)) ||
! do_aggregate)
{
+ Path *path;
+ PathTarget *target;
+
+ /*
+ * If the join output is subject to partial aggregation, the path must
+ * have the appropriate target.
+ */
+ if (!do_aggregate)
+ target = joinrel->reltarget;
+ else
+ {
+ Assert(joinrel->agg_info != NULL);
+ target = joinrel->agg_info->input;
+ }
+
/*
* If the inner path is parameterized, it is parameterized by the
* topmost parent of the outer rel, not the outer rel itself. Fix
*************** try_nestloop_path(PlannerInfo *root,
*** 445,465 ****
}
}
! add_path(joinrel, (Path *)
! create_nestloop_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer));
}
! else
{
! /* Waste no memory when we reject a path here */
bms_free(required_outer);
}
}
--- 479,534 ----
}
}
! path = (Path *) create_nestloop_path(root,
! joinrel,
! target,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer);
! if (!do_aggregate)
! {
! add_path(joinrel, path);
! success = true;
! }
! else
! {
! /*
! * Try both AGG_HASHED and AGG_SORTED partial aggregation.
! *
! * AGG_HASHED should not be parameterized because we don't want to
! * create the hashtable again for each set of parameters.
! */
! if (required_outer == NULL)
! success = create_grouped_path(root,
! joinrel,
! path,
! true,
! false,
! AGG_HASHED);
!
! /*
! * Don't try AGG_SORTED if create_grouped_path() would reject it
! * anyway.
! */
! if (pathkeys != NIL)
! success = success ||
! create_grouped_path(root,
! joinrel,
! path,
! true,
! false,
! AGG_SORTED);
! }
}
!
! if (!success)
{
! /* Waste no memory when we reject path(s) here */
bms_free(required_outer);
}
}
*************** try_partial_nestloop_path(PlannerInfo *r
*** 476,484 ****
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* If the inner path is parameterized, the parameterization must be fully
--- 545,556 ----
Path *inner_path,
List *pathkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *path;
+ PathTarget *target;
/*
* If the inner path is parameterized, the parameterization must be fully
*************** try_partial_nestloop_path(PlannerInfo *r
*** 513,519 ****
*/
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys))
return;
/*
--- 585,597 ----
*/
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
!
! /*
! * If the join output should be (partially) aggregated, the precheck
! * includes the aggregation and is postponed to create_grouped_path().
! */
! if (!do_aggregate &&
! !add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys))
return;
/*
*************** try_partial_nestloop_path(PlannerInfo *r
*** 532,549 ****
return;
}
/* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_nestloop_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL));
}
/*
--- 610,663 ----
return;
}
+ /*
+ * If the join output is subject to partial aggregation, the path must
+ * have the appropriate target.
+ */
+ if (!do_aggregate)
+ target = joinrel->reltarget;
+ else
+ {
+ Assert(joinrel->agg_info != NULL);
+ target = joinrel->agg_info->input;
+ }
+
/* Might be good enough to be worth trying, so let's try it. */
! path = (Path *) create_nestloop_path(root,
! joinrel,
! target,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL);
!
! if (!do_aggregate)
! add_partial_path(joinrel, path);
! else
! {
! create_grouped_path(root,
! joinrel,
! path,
! true,
! true,
! AGG_HASHED);
!
! /*
! * Don't try AGG_SORTED if create_grouped_path() would reject it
! * anyway.
! */
! if (pathkeys != NIL)
! create_grouped_path(root,
! joinrel,
! path,
! true,
! true,
! AGG_SORTED);
! }
}
/*
*************** try_mergejoin_path(PlannerInfo *root,
*** 562,571 ****
List *innersortkeys,
JoinType jointype,
JoinPathExtraData *extra,
! bool is_partial)
{
Relids required_outer;
JoinCostWorkspace workspace;
if (is_partial)
{
--- 676,687 ----
List *innersortkeys,
JoinType jointype,
JoinPathExtraData *extra,
! bool is_partial,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
+ bool success = false;
if (is_partial)
{
*************** try_mergejoin_path(PlannerInfo *root,
*** 578,584 ****
outersortkeys,
innersortkeys,
jointype,
! extra);
return;
}
--- 694,701 ----
outersortkeys,
innersortkeys,
jointype,
! extra,
! do_aggregate);
return;
}
*************** try_mergejoin_path(PlannerInfo *root,
*** 615,640 ****
outersortkeys, innersortkeys,
extra);
! if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer))
{
! add_path(joinrel, (Path *)
! create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer,
! mergeclauses,
! outersortkeys,
! innersortkeys));
}
! else
{
/* Waste no memory when we reject a path here */
bms_free(required_outer);
--- 732,799 ----
outersortkeys, innersortkeys,
extra);
! if ((!do_aggregate &&
! add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! pathkeys, required_outer)) ||
! do_aggregate)
{
! Path *path;
! PathTarget *target;
!
! /*
! * If the join output is subject to partial aggregation, the path must
! * have the appropriate target.
! */
! if (!do_aggregate)
! target = joinrel->reltarget;
! else
! {
! Assert(joinrel->agg_info != NULL);
! target = joinrel->agg_info->input;
! }
!
! path = (Path *) create_mergejoin_path(root,
! joinrel,
! target,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! required_outer,
! mergeclauses,
! outersortkeys,
! innersortkeys);
! if (!do_aggregate)
! {
! add_path(joinrel, path);
! success = true;
! }
! else
! {
! if (required_outer == NULL)
! success = create_grouped_path(root,
! joinrel,
! path,
! true,
! false,
! AGG_HASHED);
!
! if (pathkeys != NIL)
! success = success ||
! create_grouped_path(root,
! joinrel,
! path,
! true,
! false,
! AGG_SORTED);
! }
}
!
! if (!success)
{
/* Waste no memory when we reject a path here */
bms_free(required_outer);
*************** try_partial_mergejoin_path(PlannerInfo *
*** 656,664 ****
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinCostWorkspace workspace;
/*
* See comments in try_partial_hashjoin_path().
--- 815,826 ----
List *outersortkeys,
List *innersortkeys,
JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *path;
+ PathTarget *target;
/*
* See comments in try_partial_hashjoin_path().
*************** try_partial_mergejoin_path(PlannerInfo *
*** 691,714 ****
outersortkeys, innersortkeys,
extra);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys))
return;
/* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_mergejoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL,
! mergeclauses,
! outersortkeys,
! innersortkeys));
}
/*
--- 853,909 ----
outersortkeys, innersortkeys,
extra);
! if (!do_aggregate &&
! !add_partial_path_precheck(joinrel, workspace.total_cost, pathkeys))
return;
+ /*
+ * If the join output is subject to partial aggregation, the path must
+ * have the appropriate target.
+ */
+ if (!do_aggregate)
+ target = joinrel->reltarget;
+ else
+ {
+ Assert(joinrel->agg_info != NULL);
+ target = joinrel->agg_info->input;
+ }
+
/* Might be good enough to be worth trying, so let's try it. */
! path = (Path *) create_mergejoin_path(root,
! joinrel,
! target,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! extra->restrictlist,
! pathkeys,
! NULL,
! mergeclauses,
! outersortkeys,
! innersortkeys);
!
! if (!do_aggregate)
! add_partial_path(joinrel, path);
! else
! {
! create_grouped_path(root,
! joinrel,
! path,
! true,
! true,
! AGG_HASHED);
!
! if (pathkeys != NIL)
! create_grouped_path(root,
! joinrel,
! path,
! true,
! true,
! AGG_SORTED);
! }
}
/*
*************** try_hashjoin_path(PlannerInfo *root,
*** 723,732 ****
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra)
{
Relids required_outer;
JoinCostWorkspace workspace;
/*
* Check to see if proposed path is still parameterized, and reject if the
--- 918,930 ----
Path *inner_path,
List *hashclauses,
JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate)
{
Relids required_outer;
JoinCostWorkspace workspace;
+ Path *path = NULL;
+ bool success = false;
/*
* Check to see if proposed path is still parameterized, and reject if the
*************** try_hashjoin_path(PlannerInfo *root,
*** 743,772 ****
}
/*
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra, false);
! if (add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! NIL, required_outer))
{
! add_path(joinrel, (Path *)
! create_hashjoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! false, /* parallel_hash */
! extra->restrictlist,
! required_outer,
! hashclauses));
}
! else
{
/* Waste no memory when we reject a path here */
bms_free(required_outer);
--- 941,1020 ----
}
/*
+ * Parameterized execution of grouped path would mean repeated hashing of
+ * the output of the hashjoin output, so forget about AGG_HASHED if there
+ * are any parameters. And AGG_SORTED makes no sense because the hash join
+ * output is not sorted.
+ */
+ if (required_outer && joinrel->agg_info)
+ return;
+
+ /*
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra, false);
! /*
! * If the join output should be (partially) aggregated, the precheck
! * includes the aggregation and is postponed to create_grouped_path().
! */
! if ((!do_aggregate &&
! add_path_precheck(joinrel,
! workspace.startup_cost, workspace.total_cost,
! NIL, required_outer)) ||
! do_aggregate)
{
! PathTarget *target;
!
! /*
! * If the join output is subject to partial aggregation, the path must
! * have the appropriate target.
! */
! if (!do_aggregate)
! target = joinrel->reltarget;
! else
! {
! Assert(joinrel->agg_info != NULL);
! target = joinrel->agg_info->input;
! }
!
! path = (Path *) create_hashjoin_path(root,
! joinrel,
! target,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! false, /* parallel_hash */
! extra->restrictlist,
! required_outer,
! hashclauses);
!
! if (!do_aggregate)
! {
! add_path(joinrel, path);
! success = true;
! }
! else
! {
!
! /*
! * As the hashjoin path is not sorted, only try AGG_HASHED.
! */
! if (create_grouped_path(root,
! joinrel,
! path,
! true,
! false,
! AGG_HASHED))
! success = true;
! }
}
!
! if (!success)
{
/* Waste no memory when we reject a path here */
bms_free(required_outer);
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 790,798 ****
List *hashclauses,
JoinType jointype,
JoinPathExtraData *extra,
! bool parallel_hash)
{
JoinCostWorkspace workspace;
/*
* If the inner path is parameterized, the parameterization must be fully
--- 1038,1049 ----
List *hashclauses,
JoinType jointype,
JoinPathExtraData *extra,
! bool parallel_hash,
! bool do_aggregate)
{
JoinCostWorkspace workspace;
+ Path *path;
+ PathTarget *target;
/*
* If the inner path is parameterized, the parameterization must be fully
*************** try_partial_hashjoin_path(PlannerInfo *r
*** 815,836 ****
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra, true);
! if (!add_partial_path_precheck(joinrel, workspace.total_cost, NIL))
return;
! /* Might be good enough to be worth trying, so let's try it. */
! add_partial_path(joinrel, (Path *)
! create_hashjoin_path(root,
! joinrel,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! parallel_hash,
! extra->restrictlist,
! NULL,
! hashclauses));
}
/*
--- 1066,1118 ----
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra, true);
!
! /*
! * If the join output should be (partially) aggregated, the precheck
! * includes the aggregation and is postponed to create_grouped_path().
! */
! if (!do_aggregate &&
! !add_partial_path_precheck(joinrel, workspace.total_cost, NIL))
return;
! /*
! * If the join output is subject to partial aggregation, the path must
! * have the appropriate target.
! */
! if (!do_aggregate)
! target = joinrel->reltarget;
! else
! {
! Assert(joinrel->agg_info != NULL);
! target = joinrel->agg_info->input;
! }
!
! path = (Path *) create_hashjoin_path(root,
! joinrel,
! target,
! jointype,
! &workspace,
! extra,
! outer_path,
! inner_path,
! parallel_hash,
! extra->restrictlist,
! NULL,
! hashclauses);
! if (!do_aggregate)
! add_partial_path(joinrel, path);
! else
! {
! /*
! * Only AGG_HASHED is useful, see comments in try_hashjoin_path().
! */
! create_grouped_path(root,
! joinrel,
! path,
! true,
! true,
! AGG_HASHED);
! }
}
/*
*************** clause_sides_match_join(RestrictInfo *ri
*** 874,879 ****
--- 1156,1162 ----
* 'innerrel' is the inner join relation
* 'jointype' is the type of join to do
* 'extra' contains additional input values
+ * 'agg_info' tells if/how to apply partial aggregation to the output.
*/
static void
sort_inner_and_outer(PlannerInfo *root,
*************** sort_inner_and_outer(PlannerInfo *root,
*** 881,887 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
Path *outer_path;
--- 1164,1171 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate)
{
JoinType save_jointype = jointype;
Path *outer_path;
*************** sort_inner_and_outer(PlannerInfo *root,
*** 1043,1049 ****
innerkeys,
jointype,
extra,
! false);
/*
* If we have partial outer and parallel safe inner path then try
--- 1327,1334 ----
innerkeys,
jointype,
extra,
! false,
! do_aggregate);
/*
* If we have partial outer and parallel safe inner path then try
*************** sort_inner_and_outer(PlannerInfo *root,
*** 1059,1065 ****
outerkeys,
innerkeys,
jointype,
! extra);
}
}
--- 1344,1351 ----
outerkeys,
innerkeys,
jointype,
! extra,
! do_aggregate);
}
}
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1087,1093 ****
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial)
{
List *mergeclauses;
List *innersortkeys;
--- 1373,1380 ----
bool useallclauses,
Path *inner_cheapest_total,
List *merge_pathkeys,
! bool is_partial,
! bool do_aggregate)
{
List *mergeclauses;
List *innersortkeys;
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1148,1154 ****
innersortkeys,
jointype,
extra,
! is_partial);
/* Can't do anything else if inner path needs to be unique'd */
if (save_jointype == JOIN_UNIQUE_INNER)
--- 1435,1442 ----
innersortkeys,
jointype,
extra,
! is_partial,
! do_aggregate);
/* Can't do anything else if inner path needs to be unique'd */
if (save_jointype == JOIN_UNIQUE_INNER)
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1245,1251 ****
NIL,
jointype,
extra,
! is_partial);
cheapest_total_inner = innerpath;
}
/* Same on the basis of cheapest startup cost ... */
--- 1533,1540 ----
NIL,
jointype,
extra,
! is_partial,
! do_aggregate);
cheapest_total_inner = innerpath;
}
/* Same on the basis of cheapest startup cost ... */
*************** generate_mergejoin_paths(PlannerInfo *ro
*** 1289,1295 ****
NIL,
jointype,
extra,
! is_partial);
}
cheapest_startup_inner = innerpath;
}
--- 1578,1585 ----
NIL,
jointype,
extra,
! is_partial,
! do_aggregate);
}
cheapest_startup_inner = innerpath;
}
*************** match_unsorted_outer(PlannerInfo *root,
*** 1331,1337 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
bool nestjoinOK;
--- 1621,1628 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate)
{
JoinType save_jointype = jointype;
bool nestjoinOK;
*************** match_unsorted_outer(PlannerInfo *root,
*** 1454,1460 ****
inner_cheapest_total,
merge_pathkeys,
jointype,
! extra);
}
else if (nestjoinOK)
{
--- 1745,1752 ----
inner_cheapest_total,
merge_pathkeys,
jointype,
! extra,
! do_aggregate);
}
else if (nestjoinOK)
{
*************** match_unsorted_outer(PlannerInfo *root,
*** 1476,1482 ****
innerpath,
merge_pathkeys,
jointype,
! extra);
}
/* Also consider materialized form of the cheapest inner path */
--- 1768,1775 ----
innerpath,
merge_pathkeys,
jointype,
! extra,
! do_aggregate);
}
/* Also consider materialized form of the cheapest inner path */
*************** match_unsorted_outer(PlannerInfo *root,
*** 1487,1493 ****
matpath,
merge_pathkeys,
jointype,
! extra);
}
/* Can't do anything else if outer path needs to be unique'd */
--- 1780,1787 ----
matpath,
merge_pathkeys,
jointype,
! extra,
! do_aggregate);
}
/* Can't do anything else if outer path needs to be unique'd */
*************** match_unsorted_outer(PlannerInfo *root,
*** 1502,1508 ****
generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
save_jointype, extra, useallclauses,
inner_cheapest_total, merge_pathkeys,
! false);
}
/*
--- 1796,1802 ----
generate_mergejoin_paths(root, joinrel, innerrel, outerpath,
save_jointype, extra, useallclauses,
inner_cheapest_total, merge_pathkeys,
! false, do_aggregate);
}
/*
*************** match_unsorted_outer(PlannerInfo *root,
*** 1523,1529 ****
{
if (nestjoinOK)
consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra);
/*
* If inner_cheapest_total is NULL or non parallel-safe then find the
--- 1817,1823 ----
{
if (nestjoinOK)
consider_parallel_nestloop(root, joinrel, outerrel, innerrel,
! save_jointype, extra, do_aggregate);
/*
* If inner_cheapest_total is NULL or non parallel-safe then find the
*************** match_unsorted_outer(PlannerInfo *root,
*** 1543,1549 ****
if (inner_cheapest_total)
consider_parallel_mergejoin(root, joinrel, outerrel, innerrel,
save_jointype, extra,
! inner_cheapest_total);
}
}
--- 1837,1844 ----
if (inner_cheapest_total)
consider_parallel_mergejoin(root, joinrel, outerrel, innerrel,
save_jointype, extra,
! inner_cheapest_total,
! do_aggregate);
}
}
*************** consider_parallel_mergejoin(PlannerInfo
*** 1566,1572 ****
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total)
{
ListCell *lc1;
--- 1861,1868 ----
RelOptInfo *innerrel,
JoinType jointype,
JoinPathExtraData *extra,
! Path *inner_cheapest_total,
! bool do_aggregate)
{
ListCell *lc1;
*************** consider_parallel_mergejoin(PlannerInfo
*** 1584,1590 ****
generate_mergejoin_paths(root, joinrel, innerrel, outerpath, jointype,
extra, false, inner_cheapest_total,
! merge_pathkeys, true);
}
}
--- 1880,1886 ----
generate_mergejoin_paths(root, joinrel, innerrel, outerpath, jointype,
extra, false, inner_cheapest_total,
! merge_pathkeys, true, do_aggregate);
}
}
*************** consider_parallel_nestloop(PlannerInfo *
*** 1605,1611 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
ListCell *lc1;
--- 1901,1908 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate)
{
JoinType save_jointype = jointype;
ListCell *lc1;
*************** consider_parallel_nestloop(PlannerInfo *
*** 1655,1661 ****
}
try_partial_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra);
}
}
}
--- 1952,1959 ----
}
try_partial_nestloop_path(root, joinrel, outerpath, innerpath,
! pathkeys, jointype, extra,
! do_aggregate);
}
}
}
*************** consider_parallel_nestloop(PlannerInfo *
*** 1670,1675 ****
--- 1968,1974 ----
* 'innerrel' is the inner join relation
* 'jointype' is the type of join to do
* 'extra' contains additional input values
+ * 'agg_info' tells if/how to apply partial aggregation to the output.
*/
static void
hash_inner_and_outer(PlannerInfo *root,
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1677,1683 ****
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra)
{
JoinType save_jointype = jointype;
bool isouterjoin = IS_OUTER_JOIN(jointype);
--- 1976,1983 ----
RelOptInfo *outerrel,
RelOptInfo *innerrel,
JoinType jointype,
! JoinPathExtraData *extra,
! bool do_aggregate)
{
JoinType save_jointype = jointype;
bool isouterjoin = IS_OUTER_JOIN(jointype);
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1752,1758 ****
cheapest_total_inner,
hashclauses,
jointype,
! extra);
/* no possibility of cheap startup here */
}
else if (jointype == JOIN_UNIQUE_INNER)
--- 2052,2059 ----
cheapest_total_inner,
hashclauses,
jointype,
! extra,
! do_aggregate);
/* no possibility of cheap startup here */
}
else if (jointype == JOIN_UNIQUE_INNER)
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1768,1774 ****
cheapest_total_inner,
hashclauses,
jointype,
! extra);
if (cheapest_startup_outer != NULL &&
cheapest_startup_outer != cheapest_total_outer)
try_hashjoin_path(root,
--- 2069,2076 ----
cheapest_total_inner,
hashclauses,
jointype,
! extra,
! do_aggregate);
if (cheapest_startup_outer != NULL &&
cheapest_startup_outer != cheapest_total_outer)
try_hashjoin_path(root,
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1777,1783 ****
cheapest_total_inner,
hashclauses,
jointype,
! extra);
}
else
{
--- 2079,2086 ----
cheapest_total_inner,
hashclauses,
jointype,
! extra,
! do_aggregate);
}
else
{
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1798,1804 ****
cheapest_total_inner,
hashclauses,
jointype,
! extra);
foreach(lc1, outerrel->cheapest_parameterized_paths)
{
--- 2101,2108 ----
cheapest_total_inner,
hashclauses,
jointype,
! extra,
! do_aggregate);
foreach(lc1, outerrel->cheapest_parameterized_paths)
{
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1832,1838 ****
innerpath,
hashclauses,
jointype,
! extra);
}
}
}
--- 2136,2143 ----
innerpath,
hashclauses,
jointype,
! extra,
! do_aggregate);
}
}
}
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1875,1881 ****
cheapest_partial_outer,
cheapest_partial_inner,
hashclauses, jointype, extra,
! true /* parallel_hash */ );
}
/*
--- 2180,2187 ----
cheapest_partial_outer,
cheapest_partial_inner,
hashclauses, jointype, extra,
! true /* parallel_hash */ ,
! do_aggregate);
}
/*
*************** hash_inner_and_outer(PlannerInfo *root,
*** 1896,1902 ****
cheapest_partial_outer,
cheapest_safe_inner,
hashclauses, jointype, extra,
! false /* parallel_hash */ );
}
}
}
--- 2202,2209 ----
cheapest_partial_outer,
cheapest_safe_inner,
hashclauses, jointype, extra,
! false /* parallel_hash */ ,
! do_aggregate);
}
}
}
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
new file mode 100644
index 3f1c1b3..59273c4
*** a/src/backend/optimizer/path/joinrels.c
--- b/src/backend/optimizer/path/joinrels.c
***************
*** 17,28 ****
--- 17,31 ----
#include "miscadmin.h"
#include "catalog/partition.h"
#include "optimizer/clauses.h"
+ #include "optimizer/cost.h"
#include "optimizer/joininfo.h"
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/prep.h"
+ #include "optimizer/tlist.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+ #include "utils/selfuncs.h"
static void make_rels_by_clause_joins(PlannerInfo *root,
*************** static void make_rels_by_clause_joins(Pl
*** 31,36 ****
--- 34,43 ----
static void make_rels_by_clauseless_joins(PlannerInfo *root,
RelOptInfo *old_rel,
ListCell *other_rels);
+ static void set_grouped_joinrel_target(PlannerInfo *root, RelOptInfo *joinrel,
+ RelOptInfo *rel1, RelOptInfo *rel2,
+ SpecialJoinInfo *sjinfo, List *restrictlist,
+ RelAggInfo *agg_info);
static bool has_join_restriction(PlannerInfo *root, RelOptInfo *rel);
static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool is_dummy_rel(RelOptInfo *rel);
*************** static bool restriction_is_constant_fals
*** 38,48 ****
bool only_pushed_down);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
! SpecialJoinInfo *sjinfo, List *restrictlist);
! static void try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *parent_sjinfo,
! List *parent_restrictlist);
static int match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel,
bool strict_op);
--- 45,57 ----
bool only_pushed_down);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
! SpecialJoinInfo *sjinfo, List *restrictlist,
! bool do_aggregate);
! static void try_partition_wise_join(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *parent_sjinfo,
! List *parent_restrictlist,
! bool do_aggregate);
static int match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel,
bool strict_op);
*************** join_search_one_level(PlannerInfo *root,
*** 69,74 ****
--- 78,92 ----
Assert(joinrels[level] == NIL);
+ /*
+ * Subroutines will eventually call make_join_rel() with both input rels
+ * from the joinrels list, i.e. both non-grouped. In addition to joining
+ * these, make_join_rel() will try to combine each of these with grouped
+ * rel and also apply partial aggregation. All the grouped joins will be
+ * added to root->join_grouped_rel_level[level].
+ */
+ Assert(root->join_grouped_rel_level[level] == NIL);
+
/* Set join_cur_level so that new joinrels are added to proper list */
root->join_cur_level = level;
*************** make_rels_by_clauseless_joins(PlannerInf
*** 321,326 ****
--- 339,392 ----
}
}
+ /*
+ * Set joinrel's reltarget according to agg_info and estimate the number of
+ * rows.
+ */
+ static void
+ set_grouped_joinrel_target(PlannerInfo *root, RelOptInfo *joinrel,
+ RelOptInfo *rel1, RelOptInfo *rel2,
+ SpecialJoinInfo *sjinfo, List *restrictlist,
+ RelAggInfo *agg_info)
+ {
+ Assert(agg_info != NULL);
+
+ /*
+ * build_join_rel() / build_child_join_rel() does not create the target
+ * for grouped relation.
+ */
+ Assert(joinrel->reltarget == NULL);
+ Assert(joinrel->agg_info == NULL);
+
+ /*
+ * The output will actually be grouped, i.e. partially aggregated. No
+ * additional processing needed.
+ */
+ joinrel->reltarget = copy_pathtarget(agg_info->target);
+
+ /*
+ * The rest of agg_info will be needed at aggregation time.
+ */
+ joinrel->agg_info = agg_info;
+
+ /*
+ * Now that we have the target, compute the estimates.
+ */
+ set_joinrel_size_estimates(root, joinrel, rel1, rel2, sjinfo,
+ restrictlist);
+
+ /*
+ * Grouping essentially changes the number of rows.
+ *
+ * XXX We do not distinguish whether two plain rels are joined and the
+ * result is partially aggregated, or the partial aggregation has been
+ * already applied to one of the input rels. Is this worth extra effort,
+ * e.g. maintaining a separate RelOptInfo for each case (one difficulty
+ * that would introduce is construction of AppendPath)?
+ */
+ joinrel->rows = estimate_num_groups(root, joinrel->agg_info->group_exprs,
+ joinrel->rows, NULL);
+ }
/*
* join_is_legal
*************** join_is_legal(PlannerInfo *root, RelOptI
*** 659,670 ****
* (The join rel may already contain paths generated from other
* pairs of rels that add up to the same set of base rels.)
*
! * NB: will return NULL if attempted join is not valid. This can happen
! * when working with outer joins, or with IN or EXISTS clauses that have been
! * turned into joins.
*/
! RelOptInfo *
! make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
--- 725,745 ----
* (The join rel may already contain paths generated from other
* pairs of rels that add up to the same set of base rels.)
*
! * 'agg_info' contains the reltarget of grouped relation and everything we
! * need to perform partial aggregation. If NULL, then the join relation
! * should not be grouped.
! *
! * 'do_aggregate' tells that two non-grouped rels should be grouped and
! * partial aggregation should be applied to all their paths.
! *
! * NB: will return NULL if attempted join is not valid. This can happen when
! * working with outer joins, or with IN or EXISTS clauses that have been
! * turned into joins. NULL is also returned if caller is interested in a
! * grouped relation but there's no useful grouped input relation.
*/
! static RelOptInfo *
! make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
! RelAggInfo *agg_info, bool do_aggregate)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
*************** make_join_rel(PlannerInfo *root, RelOptI
*** 672,681 ****
--- 747,760 ----
SpecialJoinInfo sjinfo_data;
RelOptInfo *joinrel;
List *restrictlist;
+ bool grouped = agg_info != NULL;
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(rel1->relids, rel2->relids));
+ /* do_aggregate implies the output to be grouped. */
+ Assert(!do_aggregate || grouped);
+
/* Construct Relids set that identifies the joinrel. */
joinrelids = bms_union(rel1->relids, rel2->relids);
*************** make_join_rel(PlannerInfo *root, RelOptI
*** 725,731 ****
* goes with this particular joining.
*/
joinrel = build_join_rel(root, joinrelids, rel1, rel2, sjinfo,
! &restrictlist);
/*
* If we've already proven this join is empty, we needn't consider any
--- 804,829 ----
* goes with this particular joining.
*/
joinrel = build_join_rel(root, joinrelids, rel1, rel2, sjinfo,
! &restrictlist, grouped);
!
! /*
! * Make sure the joinrel has reltarget initialized. Caller should supply
! * the target for group relation, so build_join_rel() should have omitted
! * its creation.
! *
! * The target can already be there if we were already called with
! * grouped=true.
! */
! if (grouped && joinrel->reltarget == NULL)
! {
! set_grouped_joinrel_target(root, joinrel, rel1, rel2, sjinfo,
! restrictlist, agg_info);
!
! if (rel1->consider_parallel && rel2->consider_parallel &&
! is_parallel_safe(root, (Node *) restrictlist) &&
! is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
! joinrel->consider_parallel = true;
! }
/*
* If we've already proven this join is empty, we needn't consider any
*************** make_join_rel(PlannerInfo *root, RelOptI
*** 739,745 ****
/* Add paths to the join relation. */
populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
! restrictlist);
bms_free(joinrelids);
--- 837,843 ----
/* Add paths to the join relation. */
populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
! restrictlist, do_aggregate);
bms_free(joinrelids);
*************** make_join_rel(PlannerInfo *root, RelOptI
*** 747,752 ****
--- 845,988 ----
}
/*
+ * Front-end to make_join_rel_common(). Generates plain (non-grouped) join and
+ * then uses all the possible strategies to generate the grouped one.
+ */
+ JoinSearchResult *
+ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+ {
+ Relids joinrelids;
+ RelAggInfo *agg_info;
+ RelOptInfo *rel1_grouped,
+ *rel2_grouped,
+ *joinrel;
+ double nrows_plain;
+ JoinSearchResult *result;
+ bool rel1_grouped_useful,
+ rel2_grouped_useful;
+
+ result = (JoinSearchResult *) palloc0(sizeof(JoinSearchResult));
+
+ /* 1) form the plain join. */
+ result->plain = make_join_rel_common(root, rel1, rel2, NULL, false);
+
+ if (result->plain == NULL)
+ return result;
+
+ nrows_plain = result->plain->rows;
+
+ /*
+ * We're done if there are no grouping expressions nor aggregates.
+ */
+ if (root->grouped_var_list == NIL)
+ return result;
+
+ /*
+ * If the same joinrel was already formed, just with the base rels divided
+ * between rel1 and rel2 in a different way, we might already have the
+ * matching agg_info.
+ */
+ joinrelids = bms_union(rel1->relids, rel2->relids);
+ joinrel = find_join_rel(root, joinrelids, true);
+ if (joinrel != NULL && joinrel->agg_info != NULL)
+ agg_info = joinrel->agg_info;
+ else
+ {
+ double nrows;
+
+ /*
+ * agg_info must be created from scratch.
+ */
+ agg_info = create_rel_agg_info(root, result->plain);
+
+ /*
+ * Grouping essentially changes the number of rows.
+ */
+ if (agg_info != NULL)
+ {
+ nrows = estimate_num_groups(root,
+ agg_info->group_exprs,
+ nrows_plain,
+ NULL);
+ agg_info->rows = clamp_row_est(nrows);
+ }
+ }
+
+ /*
+ * Cannot we build grouped join?
+ */
+ if (agg_info == NULL)
+ return result;
+
+ /*
+ * 2) join two plain rels and aggregate the join paths.
+ */
+ result->grouped = make_join_rel_common(root, rel1, rel2, agg_info, true);
+
+ /*
+ * Retrieve the grouped relations.
+ *
+ * Dummy rel indicates join relation able to generate grouped paths as
+ * such (i.e. it has valid agg_info), but for which the path actually
+ * could not be created (e.g. only AGG_HASHED strategy was possible but
+ * work_mem was not sufficient for hash table).
+ */
+ rel1_grouped = IS_JOIN_REL(rel1) ?
+ find_join_rel(root, rel1->relids, true) :
+ find_grouped_base_rel(root, rel1->relid);
+ rel1_grouped_useful = rel1_grouped != NULL && !IS_DUMMY_REL(rel1_grouped);
+
+ rel2_grouped = IS_JOIN_REL(rel2) ?
+ find_join_rel(root, rel2->relids, true) :
+ find_grouped_base_rel(root, rel2->relid);
+ rel2_grouped_useful = rel2_grouped != NULL && !IS_DUMMY_REL(rel2_grouped);
+
+ /*
+ * Nothing else to do?
+ */
+ if (!rel1_grouped_useful && !rel2_grouped_useful)
+ return result;
+
+ /*
+ * 3) combine plain and grouped relation.
+ *
+ * At maximum one input rel can be grouped. If both were grouped, then
+ * grouping of one side would change the occurrence of the other side's
+ * aggregate transient states on the input of the final aggregation. This
+ * can be handled by adjusting the transient states, but it's not worth
+ * the effort because it's hard to find a use case for this kind of join.
+ *
+ * XXX If the join of two grouped rels is implemented someday, note that
+ * both rels can have aggregates, so it'd be hard to join grouped rel to
+ * non-grouped here: 1) such a "mixed join" would require a special
+ * target, 2) both AGGSPLIT_FINAL_DESERIAL and AGGSPLIT_SIMPLE aggregates
+ * could appear in the target of the final aggregation node, originating
+ * from the grouped and the non-grouped input rel respectively.
+ */
+ if (rel1_grouped_useful && rel2_grouped_useful)
+ return result;
+
+ /*
+ * 4) join grouped relation to plain one. The same target we used for
+ * aggregation above should be applicable to either case here.
+ */
+ if (rel1_grouped_useful)
+ joinrel = make_join_rel_common(root, rel1_grouped, rel2, agg_info,
+ false);
+ else if (rel2_grouped_useful)
+ joinrel = make_join_rel_common(root, rel1, rel2_grouped, agg_info,
+ false);
+
+ /*
+ * We expect make_join_rel_common() to return the same joinrel it did in
+ * the 2) case.
+ */
+ Assert(joinrel && result->grouped);
+
+ return result;
+ }
+
+ /*
* populate_joinrel_with_paths
* Add paths to the given joinrel for given pair of joining relations. The
* SpecialJoinInfo provides details about the join and the restrictlist
*************** make_join_rel(PlannerInfo *root, RelOptI
*** 756,762 ****
static void
populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
! SpecialJoinInfo *sjinfo, List *restrictlist)
{
/*
* Consider paths using each rel as both outer and inner. Depending on
--- 992,999 ----
static void
populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
! SpecialJoinInfo *sjinfo, List *restrictlist,
! bool do_aggregate)
{
/*
* Consider paths using each rel as both outer and inner. Depending on
*************** populate_joinrel_with_paths(PlannerInfo
*** 787,796 ****
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_INNER, sjinfo,
! restrictlist);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_INNER, sjinfo,
! restrictlist);
break;
case JOIN_LEFT:
if (is_dummy_rel(rel1) ||
--- 1024,1033 ----
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_INNER, sjinfo,
! restrictlist, do_aggregate);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_INNER, sjinfo,
! restrictlist, do_aggregate);
break;
case JOIN_LEFT:
if (is_dummy_rel(rel1) ||
*************** populate_joinrel_with_paths(PlannerInfo
*** 804,813 ****
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_LEFT, sjinfo,
! restrictlist);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_RIGHT, sjinfo,
! restrictlist);
break;
case JOIN_FULL:
if ((is_dummy_rel(rel1) && is_dummy_rel(rel2)) ||
--- 1041,1050 ----
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_LEFT, sjinfo,
! restrictlist, do_aggregate);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_RIGHT, sjinfo,
! restrictlist, do_aggregate);
break;
case JOIN_FULL:
if ((is_dummy_rel(rel1) && is_dummy_rel(rel2)) ||
*************** populate_joinrel_with_paths(PlannerInfo
*** 818,827 ****
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_FULL, sjinfo,
! restrictlist);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_FULL, sjinfo,
! restrictlist);
/*
* If there are join quals that aren't mergeable or hashable, we
--- 1055,1064 ----
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_FULL, sjinfo,
! restrictlist, do_aggregate);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_FULL, sjinfo,
! restrictlist, do_aggregate);
/*
* If there are join quals that aren't mergeable or hashable, we
*************** populate_joinrel_with_paths(PlannerInfo
*** 854,860 ****
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_SEMI, sjinfo,
! restrictlist);
}
/*
--- 1091,1097 ----
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_SEMI, sjinfo,
! restrictlist, do_aggregate);
}
/*
*************** populate_joinrel_with_paths(PlannerInfo
*** 877,886 ****
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_UNIQUE_INNER, sjinfo,
! restrictlist);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_UNIQUE_OUTER, sjinfo,
! restrictlist);
}
break;
case JOIN_ANTI:
--- 1114,1123 ----
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_UNIQUE_INNER, sjinfo,
! restrictlist, do_aggregate);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_UNIQUE_OUTER, sjinfo,
! restrictlist, do_aggregate);
}
break;
case JOIN_ANTI:
*************** populate_joinrel_with_paths(PlannerInfo
*** 895,901 ****
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_ANTI, sjinfo,
! restrictlist);
break;
default:
/* other values not expected here */
--- 1132,1138 ----
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_ANTI, sjinfo,
! restrictlist, do_aggregate);
break;
default:
/* other values not expected here */
*************** populate_joinrel_with_paths(PlannerInfo
*** 903,910 ****
break;
}
! /* Apply partitionwise join technique, if possible. */
! try_partitionwise_join(root, rel1, rel2, joinrel, sjinfo, restrictlist);
}
--- 1140,1148 ----
break;
}
! /* Apply partition-wise join technique, if possible. */
! try_partition_wise_join(root, rel1, rel2, joinrel, sjinfo, restrictlist,
! do_aggregate);
}
*************** restriction_is_constant_false(List *rest
*** 1304,1315 ****
* obtained by translating the respective parent join structures.
*/
static void
! try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
RelOptInfo *joinrel, SpecialJoinInfo *parent_sjinfo,
! List *parent_restrictlist)
{
int nparts;
int cnt_parts;
/* Guard against stack overflow due to overly deep partition hierarchy. */
check_stack_depth();
--- 1542,1554 ----
* obtained by translating the respective parent join structures.
*/
static void
! try_partition_wise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
RelOptInfo *joinrel, SpecialJoinInfo *parent_sjinfo,
! List *parent_restrictlist, bool do_aggregate)
{
int nparts;
int cnt_parts;
+ bool grouped = joinrel->agg_info != NULL;
/* Guard against stack overflow due to overly deep partition hierarchy. */
check_stack_depth();
*************** try_partitionwise_join(PlannerInfo *root
*** 1334,1341 ****
joinrel->part_scheme == rel2->part_scheme);
/*
! * Since we allow partitionwise join only when the partition bounds of
! * the joining relations exactly match, the partition bounds of the join
* should match those of the joining relations.
*/
Assert(partition_bounds_equal(joinrel->part_scheme->partnatts,
--- 1573,1580 ----
joinrel->part_scheme == rel2->part_scheme);
/*
! * Since we allow partitionwise join only when the partition bounds of the
! * joining relations exactly match, the partition bounds of the join
* should match those of the joining relations.
*/
Assert(partition_bounds_equal(joinrel->part_scheme->partnatts,
*************** try_partitionwise_join(PlannerInfo *root
*** 1386,1392 ****
(List *) adjust_appendrel_attrs(root,
(Node *) parent_restrictlist,
nappinfos, appinfos);
- pfree(appinfos);
child_joinrel = joinrel->part_rels[cnt_parts];
if (!child_joinrel)
--- 1625,1630 ----
*************** try_partitionwise_join(PlannerInfo *root
*** 1394,1408 ****
child_joinrel = build_child_join_rel(root, child_rel1, child_rel2,
joinrel, child_restrictlist,
child_sjinfo,
! child_sjinfo->jointype);
joinrel->part_rels[cnt_parts] = child_joinrel;
}
Assert(bms_equal(child_joinrel->relids, child_joinrelids));
populate_joinrel_with_paths(root, child_rel1, child_rel2,
child_joinrel, child_sjinfo,
! child_restrictlist);
}
}
--- 1632,1678 ----
child_joinrel = build_child_join_rel(root, child_rel1, child_rel2,
joinrel, child_restrictlist,
child_sjinfo,
! child_sjinfo->jointype,
! grouped);
!
! if (grouped)
! {
! RelAggInfo *child_agg_info;
!
! /*
! * Make sure the child_joinrel has reltarget initialized.
! *
! * Although build_child_join_rel() creates reltarget for each
! * child join from scratch as opposed to translating the
! * parent reltarget (XXX set_append_rel_size() uses the
! * translation --- is this inconsistency justified?), we just
! * translate the parent reltarget here. Per-child call of
! * create_rel_agg_info() would introduce too much duplicate
! * work because it needs the *parent* target as a source and
! * that one is identical for all the child joins
! */
! child_agg_info = translate_rel_agg_info(root,
! joinrel->agg_info,
! appinfos, nappinfos);
!
! /*
! * Make sure the child joinrel has reltarget initialized.
! */
! set_grouped_joinrel_target(root, child_joinrel, rel1, rel2,
! child_sjinfo, child_restrictlist,
! child_agg_info);
! }
!
joinrel->part_rels[cnt_parts] = child_joinrel;
}
+ pfree(appinfos);
Assert(bms_equal(child_joinrel->relids, child_joinrelids));
populate_joinrel_with_paths(root, child_rel1, child_rel2,
child_joinrel, child_sjinfo,
! child_restrictlist,
! do_aggregate);
}
}
diff --git a/src/backend/optimizer/path/tidpath.c b/src/backend/optimizer/path/tidpath.c
new file mode 100644
index 3bb5b8d..bb0f814
*** a/src/backend/optimizer/path/tidpath.c
--- b/src/backend/optimizer/path/tidpath.c
*************** TidQualFromBaseRestrictinfo(RelOptInfo *
*** 250,259 ****
* Candidate paths are added to the rel's pathlist (using add_path).
*/
void
! create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel)
{
Relids required_outer;
List *tidquals;
/*
* We don't support pushing join clauses into the quals of a tidscan, but
--- 250,260 ----
* Candidate paths are added to the rel's pathlist (using add_path).
*/
void
! create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel, bool grouped)
{
Relids required_outer;
List *tidquals;
+ Path *tidpath;
/*
* We don't support pushing join clauses into the quals of a tidscan, but
*************** create_tidscan_paths(PlannerInfo *root,
*** 263,270 ****
required_outer = rel->lateral_relids;
tidquals = TidQualFromBaseRestrictinfo(rel);
! if (tidquals)
! add_path(rel, (Path *) create_tidscan_path(root, rel, tidquals,
! required_outer));
}
--- 264,283 ----
required_outer = rel->lateral_relids;
tidquals = TidQualFromBaseRestrictinfo(rel);
+ if (!tidquals)
+ return;
! tidpath = (Path *) create_tidscan_path(root, rel, tidquals,
! required_outer);
!
! if (!grouped)
! add_path(rel, tidpath);
! else if (required_outer == NULL)
! {
! /*
! * Only AGG_HASHED is suitable here as it does not expect the input
! * set to be sorted.
! */
! create_grouped_path(root, rel, tidpath, false, false, AGG_HASHED);
! }
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
new file mode 100644
index 9ae1bf3..bc405bf
*** a/src/backend/optimizer/plan/createplan.c
--- b/src/backend/optimizer/plan/createplan.c
*************** use_physical_tlist(PlannerInfo *root, Pa
*** 815,820 ****
--- 815,826 ----
return false;
/*
+ * Grouped relation's target list contains GroupedVars.
+ */
+ if (rel->agg_info != NULL)
+ return false;
+
+ /*
* If a bitmap scan's tlist is empty, keep it as-is. This may allow the
* executor to skip heap page fetches, and in any case, the benefit of
* using a physical tlist instead would be minimal.
*************** create_projection_plan(PlannerInfo *root
*** 1593,1600 ****
* creation, but that would add expense to creating Paths we might end up
* not using.)
*/
! if (is_projection_capable_path(best_path->subpath) ||
! tlist_same_exprs(tlist, subplan->targetlist))
{
/* Don't need a separate Result, just assign tlist to subplan */
plan = subplan;
--- 1599,1607 ----
* creation, but that would add expense to creating Paths we might end up
* not using.)
*/
! if (!best_path->force_result &&
! (is_projection_capable_path(best_path->subpath) ||
! tlist_same_exprs(tlist, subplan->targetlist)))
{
/* Don't need a separate Result, just assign tlist to subplan */
plan = subplan;
*************** find_ec_member_for_tle(EquivalenceClass
*** 5827,5832 ****
--- 5834,5854 ----
while (tlexpr && IsA(tlexpr, RelabelType))
tlexpr = ((RelabelType *) tlexpr)->arg;
+ /*
+ * GroupedVar can contain either non-Var grouping expression or aggregate.
+ * The grouping expression might be useful for sorting, however aggregates
+ * shouldn't currently appear among pathkeys.
+ */
+ if (IsA(tlexpr, GroupedVar))
+ {
+ GroupedVar *gvar = castNode(GroupedVar, tlexpr);
+
+ if (!IsA(gvar->gvexpr, Aggref))
+ tlexpr = gvar->gvexpr;
+ else
+ return NULL;
+ }
+
foreach(lc, ec->ec_members)
{
EquivalenceMember *em = (EquivalenceMember *) lfirst(lc);
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
new file mode 100644
index a436b53..5d66785
*** a/src/backend/optimizer/plan/initsplan.c
--- b/src/backend/optimizer/plan/initsplan.c
***************
*** 14,19 ****
--- 14,20 ----
*/
#include "postgres.h"
+ #include "access/sysattr.h"
#include "catalog/pg_type.h"
#include "catalog/pg_class.h"
#include "nodes/nodeFuncs.h"
***************
*** 27,32 ****
--- 28,34 ----
#include "optimizer/planner.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+ #include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "parser/analyze.h"
#include "rewrite/rewriteManip.h"
*************** typedef struct PostponedQual
*** 46,51 ****
--- 48,56 ----
} PostponedQual;
+ static void create_aggregate_grouped_var_infos(PlannerInfo *root);
+ static void create_grouping_expr_grouped_var_infos(PlannerInfo *root);
+ static RelOptInfo *copy_simple_rel(PlannerInfo *root, RelOptInfo *rel);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
*************** static void check_hashjoinable(RestrictI
*** 96,105 ****
* jtnode. Internally, the function recurses through the jointree.
*
* At the end of this process, there should be one baserel RelOptInfo for
! * every non-join RTE that is used in the query. Therefore, this routine
! * is the only place that should call build_simple_rel with reloptkind
! * RELOPT_BASEREL. (Note: build_simple_rel recurses internally to build
! * "other rel" RelOptInfos for the members of any appendrels we find here.)
*/
void
add_base_rels_to_query(PlannerInfo *root, Node *jtnode)
--- 101,109 ----
* jtnode. Internally, the function recurses through the jointree.
*
* At the end of this process, there should be one baserel RelOptInfo for
! * every non-grouped non-join RTE that is used in the query. (Note:
! * build_simple_rel recurses internally to build "other rel" RelOptInfos for
! * the members of any appendrels we find here.)
*/
void
add_base_rels_to_query(PlannerInfo *root, Node *jtnode)
*************** add_vars_to_targetlist(PlannerInfo *root
*** 241,246 ****
--- 245,701 ----
}
}
+ /*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate as well as for
+ * each possible grouping expression and setup RelOptInfo for each base or
+ * 'other' relation that can product grouped paths.
+ *
+ * Note that targets of the 'other' relations are not set here ---
+ * set_append_rel_size() will create them by translating the targets of the
+ * base rel.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+ extern void
+ add_grouped_base_rels_to_query(PlannerInfo *root)
+ {
+ int i;
+ ListCell *lc;
+
+ /*
+ * Isn't user interested in the aggregate push-down feature?
+ */
+ if (!enable_agg_pushdown)
+ return;
+
+ /* No grouping in the query? */
+ if (!root->parse->groupClause)
+ return;
+
+ /*
+ * Grouping sets require multiple different groupings but the base
+ * relation can only generate one.
+ */
+ if (root->parse->groupingSets)
+ return;
+
+ /*
+ * SRF is not allowed in the aggregate argument and we don't even want it
+ * in the GROUP BY clause, so forbid it in general. It needs to be
+ * analyzed if evaluation of a GROUP BY clause containing SRF below the
+ * query targetlist would be correct. Currently it does not seem to be an
+ * important use case.
+ */
+ if (root->parse->hasTargetSRFs)
+ return;
+
+ /*
+ * TODO Consider if this is a real limitation.
+ */
+ if (root->parse->hasWindowFuncs)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_aggregate_grouped_var_infos(root);
+
+ /* Isn't there any aggregate to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Create GroupedVarInfo per grouping expression. */
+ create_grouping_expr_grouped_var_infos(root);
+
+ /*
+ * Are all the aggregates AGGSPLIT_SIMPLE?
+ */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /*
+ * Now that we know that grouping can be pushed down, search for the
+ * maximum sortgroupref. The base relations may need it if extra grouping
+ * expressions get added to them.
+ */
+ Assert(root->max_sortgroupref == 0);
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc);
+
+ if (te->ressortgroupref > root->max_sortgroupref)
+ root->max_sortgroupref = te->ressortgroupref;
+ }
+
+ /* Process the individual base relations. */
+ for (i = 1; i < root->simple_rel_array_size; i++)
+ {
+ RelOptInfo *rel = root->simple_rel_array[i];
+ RangeTblEntry *rte;
+ RelOptInfo *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /* NULL should mean a join relation. */
+ if (rel == NULL)
+ continue;
+
+ /*
+ * Not all RTE kinds are supported when grouping is considered.
+ *
+ * TODO Consider relaxing some of these restrictions.
+ */
+ rte = root->simple_rte_array[rel->relid];
+ if (rte->rtekind != RTE_RELATION ||
+ rte->relkind == RELKIND_FOREIGN_TABLE ||
+ rte->tablesample != NULL)
+ return;
+
+ /*
+ * Grouped "other member rels" should not be created until we know
+ * whether the parent can be grouped.
+ */
+ if (rel->reloptkind != RELOPT_BASEREL)
+ continue;
+
+ /*
+ * Retrieve the information we need for aggregation of the rel
+ * contents.
+ */
+ agg_info = create_rel_agg_info(root, rel);
+ if (agg_info == NULL)
+ continue;
+
+ /*
+ * Create the grouped counterpart of "rel".
+ */
+ rel_grouped = copy_simple_rel(root, rel);
+
+ /*
+ * Assign it the aggregation-specific info.
+ *
+ * The aggregation paths will get their input target from agg_info, so
+ * store it too.
+ */
+ rel_grouped->reltarget = agg_info->target;
+ rel_grouped->agg_info = agg_info;
+ }
+ }
+
+ /*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ */
+ static void
+ create_aggregate_grouped_var_infos(PlannerInfo *root)
+ {
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES);
+
+ /*
+ * Although GroupingFunc is related to root->parse->groupingSets, this
+ * field does not necessarily reflect its presence.
+ */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, GroupingFunc))
+ return;
+ }
+
+ /*
+ * Aggregates within the HAVING clause need to be processed in the same
+ * way as those in the main targetlist.
+ */
+ if (root->parse->havingQual != NULL)
+ {
+ List *having_exprs;
+
+ having_exprs = pull_var_clause((Node *) root->parse->havingQual,
+ PVC_INCLUDE_AGGREGATES);
+ if (having_exprs != NIL)
+ tlist_exprs = list_concat(tlist_exprs, having_exprs);
+ }
+
+ if (tlist_exprs == NIL)
+ return;
+
+ /* tlist_exprs may also contain Vars, but we only need Aggrefs. */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ if (IsA(expr, Var))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct || aggref->aggfilter)
+ {
+ /*
+ * Partial aggregation is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /*
+ * Aggregation push-down does not work w/o aggcombinefn. This field is
+ * not mandatory, so check if this particular aggregate can handle
+ * partial aggregation.
+ */
+ if (!OidIsValid(aggref->aggcombinefn))
+ {
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ gvi = lfirst_node(GroupedVarInfo, lc2);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ gvi = makeNode(GroupedVarInfo);
+ gvi->gvid = list_length(root->grouped_var_list);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+ gvi->agg_partial = copyObject(aggref);
+ mark_partial_aggref(gvi->agg_partial, AGGSPLIT_INITIAL_SERIAL);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ gvi->gv_eval_at = NULL;
+
+ /*
+ * The transient state is what appears in the target.
+ */
+ gvi->gv_width =
+ get_typavgwidth(exprType((Node *) gvi->agg_partial),
+ exprTypmod((Node *) gvi->agg_partial));
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+ }
+
+ /*
+ * Create GroupedVarInfo for each expression usable as grouping key.
+ *
+ * In addition to the expressions of the query targetlist, group_pathkeys is
+ * also considered the source of grouping expressions. That increases the
+ * chance to get the relation output grouped.
+ */
+ static void
+ create_grouping_expr_grouped_var_infos(PlannerInfo *root)
+ {
+ ListCell *l1,
+ *l2;
+ List *exprs = NIL;
+ List *sortgrouprefs = NIL;
+
+ /*
+ * Make sure GroupedVarInfo exists for each expression usable as grouping
+ * key.
+ */
+ foreach(l1, root->parse->groupClause)
+ {
+ SortGroupClause *sgClause;
+ TargetEntry *te;
+ Index sortgroupref;
+
+ sgClause = lfirst_node(SortGroupClause, l1);
+ te = get_sortgroupclause_tle(sgClause, root->processed_tlist);
+ sortgroupref = te->ressortgroupref;
+
+ if (sortgroupref == 0)
+ continue;
+
+ /*
+ * Non-zero sortgroupref does not necessarily imply grouping
+ * expression: data can also be sorted by aggregate.
+ */
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ exprs = lappend(exprs, te->expr);
+ sortgrouprefs = lappend_int(sortgrouprefs, sortgroupref);
+ }
+
+ /*
+ * Construct GroupedVarInfo for each expression.
+ */
+ forboth(l1, exprs, l2, sortgrouprefs)
+ {
+ Expr *expr = (Expr *) lfirst(l1);
+ int sortgroupref = lfirst_int(l2);
+ GroupedVarInfo *gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvid = list_length(root->grouped_var_list);
+ gvi->gvexpr = (Expr *) copyObject(expr);
+ gvi->sortgroupref = sortgroupref;
+
+ /* Find out where the expression should be evaluated. */
+ gvi->gv_eval_at = pull_varnos((Node *) expr);
+
+ /*
+ * As a special case, Var can be present in the non-grouped target, so
+ * set_rel_widths() could take care of the width computation. However
+ * the Var might not be propagated high enough in the join tree, so
+ * set_rel_width() might miss it. Handle it manually.
+ */
+ if (IsA(expr, Var))
+ {
+ Var *var = castNode(Var, expr);
+ RelOptInfo *rel = root->simple_rel_array[var->varno];
+ Oid reloid = rel->relid;
+
+ if (reloid != InvalidOid && var->varattno > 0)
+ {
+ int32 width = get_attavgwidth(reloid, var->varattno);
+
+ if (width > 0)
+ {
+ gvi->gv_width += width;
+ continue;
+ }
+ }
+ }
+
+ /*
+ * In general we should not expect any statistics to exist for an
+ * expression just because it's a grouping expression. So use the type
+ * information to get the width estimate.
+ */
+ gvi->gv_width = get_typavgwidth(exprType((Node *) gvi->gvexpr),
+ exprTypmod((Node *) gvi->gvexpr));
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ /*
+ * Take a flat copy of already initialized RelOptInfo and process child rels
+ * recursively.
+ *
+ * Flat copy ensures that we do not miss any information that the non-grouped
+ * rel already contains. XXX Do we need to copy any Node field?
+ */
+ static RelOptInfo *
+ copy_simple_rel(PlannerInfo *root, RelOptInfo *rel)
+ {
+ Index relid = rel->relid;
+ RangeTblEntry *rte;
+ ListCell *l;
+ List *indexlist = NIL;
+ RelOptInfo *result;
+
+ result = makeNode(RelOptInfo);
+ memcpy(result, rel, sizeof(RelOptInfo));
+ root->simple_grouped_rel_array[relid] = result;
+
+ /*
+ * The target for grouped paths will be initialized later.
+ */
+ result->reltarget = NULL;
+
+ /*
+ * Make sure that index paths have access to the parent rel's agg_info,
+ * which is used to indicate that the rel should produce grouped paths.
+ */
+ foreach(l, result->indexlist)
+ {
+ IndexOptInfo *src,
+ *dst;
+
+ src = lfirst_node(IndexOptInfo, l);
+ dst = makeNode(IndexOptInfo);
+ memcpy(dst, src, sizeof(IndexOptInfo));
+
+ dst->rel = result;
+ indexlist = lappend(indexlist, dst);
+ }
+ result->indexlist = indexlist;
+
+ /*
+ * This is very similar to child rel processing in build_simple_rel().
+ */
+ rte = root->simple_rte_array[relid];
+ if (rte->inh)
+ {
+ int nparts = rel->nparts;
+ int cnt_parts = 0;
+
+ if (nparts > 0)
+ result->part_rels = (RelOptInfo **)
+ palloc(sizeof(RelOptInfo *) * nparts);
+
+ foreach(l, root->append_rel_list)
+ {
+ AppendRelInfo *appinfo = (AppendRelInfo *) lfirst(l);
+ RelOptInfo *childrel;
+
+ /* append_rel_list contains all append rels; ignore others */
+ if (appinfo->parent_relid != relid)
+ continue;
+
+ /*
+ * The non-grouped child rel must already exist.
+ */
+ childrel = root->simple_rel_array[appinfo->child_relid];
+ Assert(childrel != NULL);
+
+ /*
+ * Create the copy.
+ */
+ childrel = copy_simple_rel(root, childrel);
+
+ /* Nothing more to do for an unpartitioned table. */
+ if (!rel->part_scheme)
+ continue;
+
+ Assert(cnt_parts < nparts);
+ result->part_rels[cnt_parts] = childrel;
+ cnt_parts++;
+ }
+
+ /* We should have seen all the child partitions. */
+ Assert(cnt_parts == nparts);
+ }
+
+ return result;
+ }
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c
new file mode 100644
index 95cbffb..7a25f22
*** a/src/backend/optimizer/plan/planagg.c
--- b/src/backend/optimizer/plan/planagg.c
*************** build_minmax_path(PlannerInfo *root, Min
*** 441,447 ****
subroot->tuple_fraction = 1.0;
subroot->limit_tuples = 1.0;
! final_rel = query_planner(subroot, tlist, minmax_qp_callback, NULL);
/*
* Since we didn't go through subquery_planner() to handle the subquery,
--- 441,447 ----
subroot->tuple_fraction = 1.0;
subroot->limit_tuples = 1.0;
! final_rel = query_planner(subroot, tlist, minmax_qp_callback, NULL, NULL);
/*
* Since we didn't go through subquery_planner() to handle the subquery,
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
new file mode 100644
index 7a34abc..ba16454
*** a/src/backend/optimizer/plan/planmain.c
--- b/src/backend/optimizer/plan/planmain.c
***************
*** 43,48 ****
--- 43,50 ----
* (this is NOT necessarily root->parse->targetList!)
* qp_callback is a function to compute query_pathkeys once it's safe to do so
* qp_extra is optional extra data to pass to qp_callback
+ * *partially_grouped receives relation that contains partial aggregate
+ * anywhere in the join tree.
*
* Note: the PlannerInfo node also includes a query_pathkeys field, which
* tells query_planner the sort order that is desired in the final output
***************
*** 52,62 ****
*/
RelOptInfo *
query_planner(PlannerInfo *root, List *tlist,
! query_pathkeys_callback qp_callback, void *qp_extra)
{
Query *parse = root->parse;
List *joinlist;
! RelOptInfo *final_rel;
Index rti;
double total_pages;
--- 54,66 ----
*/
RelOptInfo *
query_planner(PlannerInfo *root, List *tlist,
! query_pathkeys_callback qp_callback, void *qp_extra,
! RelOptInfo **partially_grouped)
{
Query *parse = root->parse;
List *joinlist;
! JoinSearchResult *final_rels;
! RelOptInfo *plain_rel;
Index rti;
double total_pages;
*************** query_planner(PlannerInfo *root, List *t
*** 66,73 ****
*/
if (parse->jointree->fromlist == NIL)
{
/* We need a dummy joinrel to describe the empty set of baserels */
! final_rel = build_empty_join_rel(root);
/*
* If query allows parallelism in general, check whether the quals are
--- 70,82 ----
*/
if (parse->jointree->fromlist == NIL)
{
+ JoinSearchResult *final_rels;
+ RelOptInfo *final_rel;
+
+ final_rels = (JoinSearchResult *) palloc0(sizeof(JoinSearchResult));
+
/* We need a dummy joinrel to describe the empty set of baserels */
! final_rels->plain = final_rel = build_empty_join_rel(root);
/*
* If query allows parallelism in general, check whether the quals are
*************** query_planner(PlannerInfo *root, List *t
*** 106,111 ****
--- 115,122 ----
*/
root->join_rel_list = NIL;
root->join_rel_hash = NULL;
+ root->join_grouped_rel_list = NIL;
+ root->join_grouped_rel_hash = NULL;
root->join_rel_level = NULL;
root->join_cur_level = 0;
root->canon_pathkeys = NIL;
*************** query_planner(PlannerInfo *root, List *t
*** 114,119 ****
--- 125,131 ----
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
*************** query_planner(PlannerInfo *root, List *t
*** 226,231 ****
--- 238,253 ----
extract_restriction_or_clauses(root);
/*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, setup root->grouped_var_list
+ * and create RelOptInfo for base relations capable to do the grouping.
+ *
+ * The base relations should be fully initialized now, so that we have
+ * enough info to decide whether grouping is possible.
+ */
+ add_grouped_base_rels_to_query(root);
+
+ /*
* We should now have size estimates for every actual table involved in
* the query, and we also know which if any have been deleted from the
* query by join removal; so we can compute total_table_pages.
*************** query_planner(PlannerInfo *root, List *t
*** 256,267 ****
/*
* Ready to do the primary planning.
*/
! final_rel = make_one_rel(root, joinlist);
/* Check that we got at least one usable path */
! if (!final_rel || !final_rel->cheapest_total_path ||
! final_rel->cheapest_total_path->param_info != NULL)
elog(ERROR, "failed to construct the join relation");
! return final_rel;
}
--- 278,292 ----
/*
* Ready to do the primary planning.
*/
! final_rels = make_one_rel(root, joinlist);
! plain_rel = final_rels->plain;
! if (partially_grouped != NULL)
! *partially_grouped = final_rels->grouped;
/* Check that we got at least one usable path */
! if (!plain_rel || !plain_rel->cheapest_total_path ||
! plain_rel->cheapest_total_path->param_info != NULL)
elog(ERROR, "failed to construct the join relation");
! return plain_rel;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
new file mode 100644
index de1257d..f1a21c0
*** a/src/backend/optimizer/plan/planner.c
--- b/src/backend/optimizer/plan/planner.c
*************** static void standard_qp_callback(Planner
*** 131,141 ****
static double get_number_of_groups(PlannerInfo *root,
double path_rows,
grouping_sets_data *gd);
- static Size estimate_hashagg_tablesize(Path *path,
- const AggClauseCosts *agg_costs,
- double dNumGroups);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *target,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd);
--- 131,139 ----
static double get_number_of_groups(PlannerInfo *root,
double path_rows,
grouping_sets_data *gd);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
+ RelOptInfo *partially_grouped_input_rel,
PathTarget *target,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd);
*************** static void add_paths_to_partial_groupin
*** 200,205 ****
--- 198,205 ----
grouping_sets_data *gd,
bool can_sort,
bool can_hash);
+ static void gather_partial_grouping_rel_paths(PlannerInfo *root,
+ RelOptInfo *partially_grouped_rel);
static bool can_parallel_agg(PlannerInfo *root, RelOptInfo *input_rel,
RelOptInfo *grouped_rel, const AggClauseCosts *agg_costs);
*************** grouping_planner(PlannerInfo *root, bool
*** 1688,1693 ****
--- 1688,1694 ----
List *activeWindows = NIL;
grouping_sets_data *gset_data = NULL;
standard_qp_extra qp_extra;
+ RelOptInfo *partially_grouped = NULL;
/* A recursive query should always have setOperations */
Assert(!root->hasRecursion);
*************** grouping_planner(PlannerInfo *root, bool
*** 1795,1801 ****
* of the query's sort clause, distinct clause, etc.
*/
current_rel = query_planner(root, tlist,
! standard_qp_callback, &qp_extra);
/*
* Convert the query's result tlist into PathTarget format.
--- 1796,1803 ----
* of the query's sort clause, distinct clause, etc.
*/
current_rel = query_planner(root, tlist,
! standard_qp_callback, &qp_extra,
! &partially_grouped);
/*
* Convert the query's result tlist into PathTarget format.
*************** grouping_planner(PlannerInfo *root, bool
*** 1983,1988 ****
--- 1985,1991 ----
{
current_rel = create_grouping_paths(root,
current_rel,
+ partially_grouped,
grouping_target,
&agg_costs,
gset_data);
*************** get_number_of_groups(PlannerInfo *root,
*** 3561,3600 ****
}
/*
- * estimate_hashagg_tablesize
- * estimate the number of bytes that a hash aggregate hashtable will
- * require based on the agg_costs, path width and dNumGroups.
- *
- * XXX this may be over-estimating the size now that hashagg knows to omit
- * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
- * grouping columns not in the hashed set are counted here even though hashagg
- * won't store them. Is this a problem?
- */
- static Size
- estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
- double dNumGroups)
- {
- Size hashentrysize;
-
- /* Estimate per-hash-entry space at tuple width... */
- hashentrysize = MAXALIGN(path->pathtarget->width) +
- MAXALIGN(SizeofMinimalTupleHeader);
-
- /* plus space for pass-by-ref transition values... */
- hashentrysize += agg_costs->transitionSpace;
- /* plus the per-hash-entry overhead */
- hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
-
- /*
- * Note that this disregards the effect of fill-factor and growth policy
- * of the hash-table. That's probably ok, given default the default
- * fill-factor is relatively high. It'd be hard to meaningfully factor in
- * "double-in-size" growth policies here.
- */
- return hashentrysize * dNumGroups;
- }
-
- /*
* create_grouping_paths
*
* Build a new upperrel containing Paths for grouping and/or aggregation.
--- 3564,3569 ----
*************** estimate_hashagg_tablesize(Path *path, c
*** 3605,3610 ****
--- 3574,3580 ----
* is, they need a Gather and then a FinalizeAggregate.
*
* input_rel: contains the source-data Paths
+ * partially_grouped_input_rel: contains Paths with aggregation pushed down.
* target: the pathtarget for the result Paths to compute
* agg_costs: cost info about all aggregates in query (in AGGSPLIT_SIMPLE mode)
* rollup_lists: list of grouping sets, or NIL if not doing grouping sets
*************** estimate_hashagg_tablesize(Path *path, c
*** 3622,3627 ****
--- 3592,3598 ----
static RelOptInfo *
create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
+ RelOptInfo *partially_grouped_input_rel,
PathTarget *target,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd)
*************** create_grouping_paths(PlannerInfo *root,
*** 3824,3837 ****
get_agg_clause_costs(root, (Node *) partial_grouping_target->exprs,
AGGSPLIT_INITIAL_SERIAL,
&agg_partial_costs);
-
- /* final phase */
- get_agg_clause_costs(root, (Node *) target->exprs,
- AGGSPLIT_FINAL_DESERIAL,
- &agg_final_costs);
- get_agg_clause_costs(root, parse->havingQual,
- AGGSPLIT_FINAL_DESERIAL,
- &agg_final_costs);
}
add_paths_to_partial_grouping_rel(root, input_rel,
--- 3795,3800 ----
*************** create_grouping_paths(PlannerInfo *root,
*** 3840,3845 ****
--- 3803,3862 ----
gd, can_sort, can_hash);
}
+ /*
+ * Paths generated due to aggregation push-down are passed in a separate
+ * relation. Unlike "partially grouped_rel", reltarget of which contains
+ * Aggrefs, this relation's reltarget contains GroupedVars.
+ */
+ if (partially_grouped_input_rel)
+ {
+ ListCell *lc;
+
+ Assert(enable_agg_pushdown);
+
+ /*
+ * Aggregation push-down could have produced partial paths as well.
+ * These are already aggregated, so only apply Gather / GatherMerge to
+ * them.
+ */
+ gather_partial_grouping_rel_paths(root, partially_grouped_input_rel);
+
+ /*
+ * If non-partial paths were generated above, and / or the aggregate
+ * push-down resulted in non-partial paths, just add them all to
+ * partially_grouped_rel for common processing.
+ *
+ * The only difference is that the paths we add here have GroupedVars
+ * in their pathtarget, while ones already contained in the pathlist
+ * of partially_grouped_rel (i.e. the paths resulting from parallel
+ * processing) have Aggrefs. This difference will be handled later by
+ * set_upper_references().
+ */
+ foreach(lc, partially_grouped_input_rel->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ add_path(partially_grouped_rel, path);
+ }
+ }
+
+ /*
+ * Prepare for the final aggregation if it's expected to take place.
+ */
+ if (partially_grouped_rel->pathlist)
+ {
+ /* Choose the best path(s) */
+ set_cheapest(partially_grouped_rel);
+
+ /* final phase */
+ get_agg_clause_costs(root, (Node *) target->exprs,
+ AGGSPLIT_FINAL_DESERIAL,
+ &agg_final_costs);
+ get_agg_clause_costs(root, parse->havingQual,
+ AGGSPLIT_FINAL_DESERIAL,
+ &agg_final_costs);
+ }
+
/* Build final grouping paths */
add_paths_to_grouping_rel(root, input_rel, grouped_rel, target,
partially_grouped_rel, agg_costs,
*************** add_paths_to_partial_grouping_rel(Planne
*** 6334,6339 ****
--- 6351,6375 ----
* Try adding Gather or Gather Merge to partial paths to produce
* non-partial paths.
*/
+ gather_partial_grouping_rel_paths(root, partially_grouped_rel);
+ }
+
+ /*
+ * Apply Gather or GatherMerge to partial paths of partially_grouped_rel. The
+ * input paths should already be partially aggregated.
+ */
+ static void
+ gather_partial_grouping_rel_paths(PlannerInfo *root,
+ RelOptInfo *partially_grouped_rel)
+ {
+ Path *cheapest_partial_path;
+
+ /* If there are no partial paths, there's nothing to do here. */
+ if (partially_grouped_rel->partial_pathlist == NIL)
+ return;
+
+ cheapest_partial_path = linitial(partially_grouped_rel->partial_pathlist);
+
generate_gather_paths(root, partially_grouped_rel, true);
/* Get cheapest partial path from partially_grouped_rel */
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
new file mode 100644
index 4617d12..f385792
*** a/src/backend/optimizer/plan/setrefs.c
--- b/src/backend/optimizer/plan/setrefs.c
*************** typedef struct
*** 40,45 ****
--- 40,46 ----
List *tlist; /* underlying target list */
int num_vars; /* number of plain Var tlist entries */
bool has_ph_vars; /* are there PlaceHolderVar entries? */
+ bool has_grp_vars; /* are there GroupedVar entries? */
bool has_non_vars; /* are there other entries? */
bool has_conv_whole_rows; /* are there ConvertRowtypeExpr
* entries encapsulating a whole-row
*************** set_upper_references(PlannerInfo *root,
*** 1739,1747 ****
--- 1740,1802 ----
indexed_tlist *subplan_itlist;
List *output_targetlist;
ListCell *l;
+ List *sub_tlist_save = NIL;
+
+ if (root->grouped_var_list != NIL)
+ {
+ if (IsA(plan, Agg))
+ {
+ Agg *agg = (Agg *) plan;
+
+ if (agg->aggsplit == AGGSPLIT_FINAL_DESERIAL)
+ {
+ /*
+ * convert_combining_aggrefs could have replaced some vars
+ * with Aggref expressions representing the partial
+ * aggregation. We need to restore the same Aggrefs in the
+ * subplan targetlist, but this would break the subplan if
+ * it's something else than the partial aggregation (i.e. the
+ * partial aggregation takes place lower in the plan tree). So
+ * we'll eventually need to restore the current
+ * subplan->targetlist.
+ */
+ if (!IsA(subplan, Agg))
+ sub_tlist_save = subplan->targetlist;
+ #ifdef USE_ASSERT_CHECKING
+ else
+ Assert(((Agg *) subplan)->aggsplit == AGGSPLIT_INITIAL_SERIAL);
+ #endif /* USE_ASSERT_CHECKING */
+
+ /*
+ * Restore the aggregate expressions that we might have
+ * removed when planning for aggregation at base relation
+ * level.
+ */
+ subplan->targetlist =
+ replace_grouped_vars_with_aggrefs(root, subplan->targetlist);
+ }
+ else if (agg->aggsplit == AGGSPLIT_INITIAL_SERIAL)
+ {
+ /*
+ * Partial aggregation node can have GroupedVar's on the input
+ * if those represent generic (non-Var) grouping expressions.
+ * Unlike above, the restored expressions should stay there.
+ */
+ subplan->targetlist =
+ replace_grouped_vars_with_aggrefs(root, subplan->targetlist);
+ }
+ }
+ }
subplan_itlist = build_tlist_index(subplan->targetlist);
+ /*
+ * The replacement of GroupVars by Aggrefs was only needed for the index
+ * build.
+ */
+ if (sub_tlist_save != NIL)
+ subplan->targetlist = sub_tlist_save;
+
output_targetlist = NIL;
foreach(l, plan->targetlist)
{
*************** build_tlist_index(List *tlist)
*** 1996,2001 ****
--- 2051,2057 ----
itlist->tlist = tlist;
itlist->has_ph_vars = false;
+ itlist->has_grp_vars = false;
itlist->has_non_vars = false;
itlist->has_conv_whole_rows = false;
*************** build_tlist_index(List *tlist)
*** 2016,2021 ****
--- 2072,2079 ----
}
else if (tle->expr && IsA(tle->expr, PlaceHolderVar))
itlist->has_ph_vars = true;
+ else if (tle->expr && IsA(tle->expr, GroupedVar))
+ itlist->has_grp_vars = true;
else if (is_converted_whole_row_reference((Node *) tle->expr))
itlist->has_conv_whole_rows = true;
else
*************** fix_join_expr_mutator(Node *node, fix_jo
*** 2299,2304 ****
--- 2357,2387 ----
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, GroupedVar))
+ {
+ GroupedVar *gvar = (GroupedVar *) node;
+
+ /* See if the GroupedVar has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_grp_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) gvar,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_grp_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) gvar,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+
+ /* No referent found for GroupedVar */
+ elog(ERROR, "grouped variable not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
*************** fix_upper_expr_mutator(Node *node, fix_u
*** 2461,2467 ****
/* If no match, just fall through to process it normally */
}
/* Try matching more complex expressions too, if tlist has any */
! if (context->subplan_itlist->has_non_vars ||
(context->subplan_itlist->has_conv_whole_rows &&
is_converted_whole_row_reference(node)))
{
--- 2544,2551 ----
/* If no match, just fall through to process it normally */
}
/* Try matching more complex expressions too, if tlist has any */
! if (context->subplan_itlist->has_grp_vars ||
! context->subplan_itlist->has_non_vars ||
(context->subplan_itlist->has_conv_whole_rows &&
is_converted_whole_row_reference(node)))
{
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
new file mode 100644
index 45d82da..d52f229
*** a/src/backend/optimizer/prep/prepjointree.c
--- b/src/backend/optimizer/prep/prepjointree.c
*************** pull_up_simple_subquery(PlannerInfo *roo
*** 911,916 ****
--- 911,917 ----
memset(subroot->upper_rels, 0, sizeof(subroot->upper_rels));
memset(subroot->upper_targets, 0, sizeof(subroot->upper_targets));
subroot->processed_tlist = NIL;
+ subroot->max_sortgroupref = 0;
subroot->grouping_map = NULL;
subroot->minmax_aggs = NIL;
subroot->qual_security_level = 0;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
new file mode 100644
index fe3b458..835623b
*** a/src/backend/optimizer/util/pathnode.c
--- b/src/backend/optimizer/util/pathnode.c
***************
*** 27,32 ****
--- 27,33 ----
#include "optimizer/planmain.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+ /* TODO Remove this if create_grouped_path ends up in another module. */
#include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "parser/parsetree.h"
*************** static List *reparameterize_pathlist_by_
*** 57,63 ****
List *pathlist,
RelOptInfo *child_rel);
-
/*****************************************************************************
* MISC. PATH UTILITIES
*****************************************************************************/
--- 58,63 ----
*************** compare_path_costs_fuzzily(Path *path1,
*** 243,248 ****
--- 243,249 ----
void
set_cheapest(RelOptInfo *parent_rel)
{
+ bool grouped = parent_rel->agg_info != NULL;
Path *cheapest_startup_path;
Path *cheapest_total_path;
Path *best_param_path;
*************** set_cheapest(RelOptInfo *parent_rel)
*** 252,258 ****
Assert(IsA(parent_rel, RelOptInfo));
if (parent_rel->pathlist == NIL)
! elog(ERROR, "could not devise a query plan for the given query");
cheapest_startup_path = cheapest_total_path = best_param_path = NULL;
parameterized_paths = NIL;
--- 253,273 ----
Assert(IsA(parent_rel, RelOptInfo));
if (parent_rel->pathlist == NIL)
! {
! if (!grouped)
! elog(ERROR, "could not devise a query plan for the given query");
! else
! {
! /*
! * Creation of grouped paths is not guaranteed.
! */
! if (IS_SIMPLE_REL(parent_rel) || IS_JOIN_REL(parent_rel))
! mark_dummy_rel(parent_rel);
! else
! Assert(false);
! return;
! }
! }
cheapest_startup_path = cheapest_total_path = best_param_path = NULL;
parameterized_paths = NIL;
*************** create_seqscan_path(PlannerInfo *root, R
*** 949,958 ****
Relids required_outer, int parallel_workers)
{
Path *pathnode = makeNode(Path);
pathnode->pathtype = T_SeqScan;
pathnode->parent = rel;
! pathnode->pathtarget = rel->reltarget;
pathnode->param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->parallel_aware = parallel_workers > 0 ? true : false;
--- 964,978 ----
Relids required_outer, int parallel_workers)
{
Path *pathnode = makeNode(Path);
+ bool grouped = rel->agg_info != NULL;
pathnode->pathtype = T_SeqScan;
pathnode->parent = rel;
! /* For grouped relation only generate the aggregation input. */
! if (!grouped)
! pathnode->pathtarget = rel->reltarget;
! else
! pathnode->pathtarget = rel->agg_info->input;
pathnode->param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->parallel_aware = parallel_workers > 0 ? true : false;
*************** create_index_path(PlannerInfo *root,
*** 1032,1041 ****
RelOptInfo *rel = index->rel;
List *indexquals,
*indexqualcols;
pathnode->path.pathtype = indexonly ? T_IndexOnlyScan : T_IndexScan;
pathnode->path.parent = rel;
! pathnode->path.pathtarget = rel->reltarget;
pathnode->path.param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->path.parallel_aware = false;
--- 1052,1066 ----
RelOptInfo *rel = index->rel;
List *indexquals,
*indexqualcols;
+ bool grouped = rel->agg_info != NULL;
pathnode->path.pathtype = indexonly ? T_IndexOnlyScan : T_IndexScan;
pathnode->path.parent = rel;
! /* For grouped relation only generate the aggregation input. */
! if (!grouped)
! pathnode->path.pathtarget = rel->reltarget;
! else
! pathnode->path.pathtarget = rel->agg_info->input;
pathnode->path.param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->path.parallel_aware = false;
*************** create_tidscan_path(PlannerInfo *root, R
*** 1183,1192 ****
Relids required_outer)
{
TidPath *pathnode = makeNode(TidPath);
pathnode->path.pathtype = T_TidScan;
pathnode->path.parent = rel;
! pathnode->path.pathtarget = rel->reltarget;
pathnode->path.param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->path.parallel_aware = false;
--- 1208,1222 ----
Relids required_outer)
{
TidPath *pathnode = makeNode(TidPath);
+ bool grouped = rel->agg_info != NULL;
pathnode->path.pathtype = T_TidScan;
pathnode->path.parent = rel;
! /* For grouped relation only generate the aggregation input. */
! if (!grouped)
! pathnode->path.pathtarget = rel->reltarget;
! else
! pathnode->path.pathtarget = rel->agg_info->input;
pathnode->path.param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->path.parallel_aware = false;
*************** create_append_path(RelOptInfo *rel,
*** 1218,1229 ****
{
AppendPath *pathnode = makeNode(AppendPath);
ListCell *l;
Assert(!parallel_aware || parallel_workers > 0);
pathnode->path.pathtype = T_Append;
pathnode->path.parent = rel;
! pathnode->path.pathtarget = rel->reltarget;
pathnode->path.param_info = get_appendrel_parampathinfo(rel,
required_outer);
pathnode->path.parallel_aware = parallel_aware;
--- 1248,1261 ----
{
AppendPath *pathnode = makeNode(AppendPath);
ListCell *l;
+ bool grouped = rel->agg_info != NULL;
Assert(!parallel_aware || parallel_workers > 0);
pathnode->path.pathtype = T_Append;
pathnode->path.parent = rel;
! pathnode->path.pathtarget = !grouped ? rel->reltarget :
! rel->agg_info->target;
pathnode->path.param_info = get_appendrel_parampathinfo(rel,
required_outer);
pathnode->path.parallel_aware = parallel_aware;
*************** append_startup_cost_compare(const void *
*** 1317,1327 ****
/*
* create_merge_append_path
* Creates a path corresponding to a MergeAppend plan, returning the
! * pathnode.
*/
MergeAppendPath *
create_merge_append_path(PlannerInfo *root,
RelOptInfo *rel,
List *subpaths,
List *pathkeys,
Relids required_outer,
--- 1349,1361 ----
/*
* create_merge_append_path
* Creates a path corresponding to a MergeAppend plan, returning the
! * pathnode. target can be supplied by caller. If NULL is passed, the field
! * is set to rel->reltarget.
*/
MergeAppendPath *
create_merge_append_path(PlannerInfo *root,
RelOptInfo *rel,
+ PathTarget *target,
List *subpaths,
List *pathkeys,
Relids required_outer,
*************** create_merge_append_path(PlannerInfo *ro
*** 1334,1340 ****
pathnode->path.pathtype = T_MergeAppend;
pathnode->path.parent = rel;
! pathnode->path.pathtarget = rel->reltarget;
pathnode->path.param_info = get_appendrel_parampathinfo(rel,
required_outer);
pathnode->path.parallel_aware = false;
--- 1368,1374 ----
pathnode->path.pathtype = T_MergeAppend;
pathnode->path.parent = rel;
! pathnode->path.pathtarget = target ? target : rel->reltarget;
pathnode->path.param_info = get_appendrel_parampathinfo(rel,
required_outer);
pathnode->path.parallel_aware = false;
*************** create_unique_path(PlannerInfo *root, Re
*** 1504,1510 ****
MemoryContext oldcontext;
int numCols;
! /* Caller made a mistake if subpath isn't cheapest_total ... */
Assert(subpath == rel->cheapest_total_path);
Assert(subpath->parent == rel);
/* ... or if SpecialJoinInfo is the wrong one */
--- 1538,1546 ----
MemoryContext oldcontext;
int numCols;
! /*
! * Caller made a mistake if subpath isn't cheapest_total.
! */
Assert(subpath == rel->cheapest_total_path);
Assert(subpath->parent == rel);
/* ... or if SpecialJoinInfo is the wrong one */
*************** calc_non_nestloop_required_outer(Path *o
*** 2125,2130 ****
--- 2161,2167 ----
* relations.
*
* 'joinrel' is the join relation.
+ * 'target' is the join path target
* 'jointype' is the type of join required
* 'workspace' is the result from initial_cost_nestloop
* 'extra' contains various information about the join
*************** calc_non_nestloop_required_outer(Path *o
*** 2139,2144 ****
--- 2176,2182 ----
NestPath *
create_nestloop_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
*************** create_nestloop_path(PlannerInfo *root,
*** 2179,2185 ****
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
! pathnode->path.pathtarget = joinrel->reltarget;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2217,2223 ----
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
! pathnode->path.pathtarget = target;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_nestloop_path(PlannerInfo *root,
*** 2211,2216 ****
--- 2249,2255 ----
* two relations
*
* 'joinrel' is the join relation
+ * 'target' is the join path target
* 'jointype' is the type of join required
* 'workspace' is the result from initial_cost_mergejoin
* 'extra' contains various information about the join
*************** create_nestloop_path(PlannerInfo *root,
*** 2227,2232 ****
--- 2266,2272 ----
MergePath *
create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
*************** create_mergejoin_path(PlannerInfo *root,
*** 2243,2249 ****
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = joinrel->reltarget;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2283,2289 ----
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_mergejoin_path(PlannerInfo *root,
*** 2279,2284 ****
--- 2319,2325 ----
* Creates a pathnode corresponding to a hash join between two relations.
*
* 'joinrel' is the join relation
+ * 'target' is the join path target
* 'jointype' is the type of join required
* 'workspace' is the result from initial_cost_hashjoin
* 'extra' contains various information about the join
*************** create_mergejoin_path(PlannerInfo *root,
*** 2293,2298 ****
--- 2334,2340 ----
HashPath *
create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
*************** create_hashjoin_path(PlannerInfo *root,
*** 2307,2313 ****
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = joinrel->reltarget;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
--- 2349,2355 ----
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
! pathnode->jpath.path.pathtarget = target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
*************** create_projection_path(PlannerInfo *root
*** 2389,2396 ****
* Note: in the latter case, create_projection_plan has to recheck our
* conclusion; see comments therein.
*/
! if (is_projection_capable_path(subpath) ||
! equal(oldtarget->exprs, target->exprs))
{
/* No separate Result node needed */
pathnode->dummypp = true;
--- 2431,2438 ----
* Note: in the latter case, create_projection_plan has to recheck our
* conclusion; see comments therein.
*/
! if ((is_projection_capable_path(subpath) ||
! equal(oldtarget->exprs, target->exprs)))
{
/* No separate Result node needed */
pathnode->dummypp = true;
*************** create_agg_path(PlannerInfo *root,
*** 2775,2782 ****
pathnode->path.pathtype = T_Agg;
pathnode->path.parent = rel;
pathnode->path.pathtarget = target;
! /* For now, assume we are above any joins, so no parameterization */
! pathnode->path.param_info = NULL;
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
--- 2817,2823 ----
pathnode->path.pathtype = T_Agg;
pathnode->path.parent = rel;
pathnode->path.pathtarget = target;
! pathnode->path.param_info = subpath->param_info;
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
*************** create_agg_path(PlannerInfo *root,
*** 2809,2814 ****
--- 2850,3004 ----
}
/*
+ * Apply partial AGG_SORTED aggregation path to subpath if it's suitably
+ * sorted.
+ *
+ * check_pathkeys can be passed FALSE if the function was already called for
+ * given index --- since the target should not change, we can skip the check
+ * of sorting during subsequent calls.
+ *
+ * agg_info contains both aggregate and grouping expressions.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+ AggPath *
+ create_partial_agg_sorted_path(PlannerInfo *root, Path *subpath,
+ bool check_pathkeys, double input_rows)
+ {
+ RelOptInfo *rel;
+ AggClauseCosts agg_costs;
+ double dNumGroups;
+ AggPath *result = NULL;
+ RelAggInfo *agg_info;
+
+ rel = subpath->parent;
+ agg_info = rel->agg_info;
+ Assert(agg_info != NULL);
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ if (check_pathkeys)
+ {
+ ListCell *lc1;
+ List *key_subset = NIL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+ }
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ Assert(agg_info->agg_exprs != NIL);
+ get_agg_clause_costs(root, (Node *) agg_info->agg_exprs,
+ AGGSPLIT_INITIAL_SERIAL, &agg_costs);
+
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ input_rows, NULL);
+
+ Assert(agg_info->group_clauses != NIL);
+ result = create_agg_path(root, rel, subpath, agg_info->target,
+ AGG_SORTED, AGGSPLIT_INITIAL_SERIAL,
+ agg_info->group_clauses, NIL, &agg_costs, dNumGroups);
+
+ return result;
+ }
+
+ /*
+ * Apply partial AGG_HASHED aggregation to subpath.
+ *
+ * Arguments have the same meaning as those of create_agg_sorted_path.
+ */
+ AggPath *
+ create_partial_agg_hashed_path(PlannerInfo *root, Path *subpath,
+ double input_rows)
+ {
+ RelOptInfo *rel;
+ bool can_hash;
+ AggClauseCosts agg_costs;
+ double dNumGroups;
+ Size hashaggtablesize;
+ Query *parse = root->parse;
+ AggPath *result = NULL;
+ RelAggInfo *agg_info;
+
+ rel = subpath->parent;
+ agg_info = rel->agg_info;
+ Assert(agg_info != NULL);
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ Assert(agg_info->agg_exprs != NIL);
+ get_agg_clause_costs(root, (Node *) agg_info->agg_exprs,
+ AGGSPLIT_INITIAL_SERIAL, &agg_costs);
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ input_rows, NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * Create the partial aggregation path.
+ */
+ Assert(agg_info->group_clauses != NIL);
+
+ result = create_agg_path(root, rel, subpath,
+ agg_info->target,
+ AGG_HASHED,
+ AGGSPLIT_INITIAL_SERIAL,
+ agg_info->group_clauses, NIL,
+ &agg_costs,
+ dNumGroups);
+
+ /*
+ * The agg path should require no fewer parameters than the plain
+ * one.
+ */
+ result->path.param_info = subpath->param_info;
+ }
+ }
+
+ return result;
+ }
+
+ /*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
*
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
new file mode 100644
index da8f0f9..6eaba9f
*** a/src/backend/optimizer/util/relnode.c
--- b/src/backend/optimizer/util/relnode.c
***************
*** 17,22 ****
--- 17,23 ----
#include <limits.h>
#include "miscadmin.h"
+ #include "catalog/pg_constraint_fn.h"
#include "catalog/partition.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
***************
*** 27,32 ****
--- 28,35 ----
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+ #include "optimizer/var.h"
+ #include "parser/parse_oper.h"
#include "utils/hsearch.h"
*************** static List *subbuild_joinrel_joinlist(R
*** 53,62 ****
List *new_joininfo);
static void set_foreign_rel_properties(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel);
! static void add_join_rel(PlannerInfo *root, RelOptInfo *joinrel);
static void build_joinrel_partition_info(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
List *restrictlist, JoinType jointype);
/*
--- 56,69 ----
List *new_joininfo);
static void set_foreign_rel_properties(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel);
! static void add_join_rel(PlannerInfo *root, RelOptInfo *joinrel,
! bool grouped);
static void build_joinrel_partition_info(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
List *restrictlist, JoinType jointype);
+ static void init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p);
/*
*************** setup_simple_rel_arrays(PlannerInfo *roo
*** 72,80 ****
/* Arrays are accessed using RT indexes (1..N) */
root->simple_rel_array_size = list_length(root->parse->rtable) + 1;
! /* simple_rel_array is initialized to all NULLs */
root->simple_rel_array = (RelOptInfo **)
palloc0(root->simple_rel_array_size * sizeof(RelOptInfo *));
/* simple_rte_array is an array equivalent of the rtable list */
root->simple_rte_array = (RangeTblEntry **)
--- 79,92 ----
/* Arrays are accessed using RT indexes (1..N) */
root->simple_rel_array_size = list_length(root->parse->rtable) + 1;
! /*
! * simple_rel_array / simple_grouped_rel_array are both initialized to all
! * NULLs
! */
root->simple_rel_array = (RelOptInfo **)
palloc0(root->simple_rel_array_size * sizeof(RelOptInfo *));
+ root->simple_grouped_rel_array = (RelOptInfo **)
+ palloc0(root->simple_rel_array_size * sizeof(RelOptInfo *));
/* simple_rte_array is an array equivalent of the rtable list */
root->simple_rte_array = (RangeTblEntry **)
*************** build_simple_rel(PlannerInfo *root, int
*** 111,117 ****
rel->reloptkind = parent ? RELOPT_OTHER_MEMBER_REL : RELOPT_BASEREL;
rel->relids = bms_make_singleton(relid);
rel->rows = 0;
! /* cheap startup cost is interesting iff not all tuples to be retrieved */
rel->consider_startup = (root->tuple_fraction > 0);
rel->consider_param_startup = false; /* might get changed later */
rel->consider_parallel = false; /* might get changed later */
--- 123,136 ----
rel->reloptkind = parent ? RELOPT_OTHER_MEMBER_REL : RELOPT_BASEREL;
rel->relids = bms_make_singleton(relid);
rel->rows = 0;
!
! /*
! * Cheap startup cost is interesting iff not all tuples to be retrieved.
! * XXX As for grouped relation, the startup cost might be interesting for
! * AGG_SORTED (if it can produce the ordering that matches
! * root->query_pathkeys) but not in general (other kinds of aggregation
! * need the whole relation). Yet it seems worth trying.
! */
rel->consider_startup = (root->tuple_fraction > 0);
rel->consider_param_startup = false; /* might get changed later */
rel->consider_parallel = false; /* might get changed later */
*************** build_simple_rel(PlannerInfo *root, int
*** 125,130 ****
--- 144,150 ----
rel->cheapest_parameterized_paths = NIL;
rel->direct_lateral_relids = NULL;
rel->lateral_relids = NULL;
+ rel->agg_info = NULL;
rel->relid = relid;
rel->rtekind = rte->rtekind;
/* min_attr, max_attr, attr_needed, attr_widths are set below */
*************** find_base_rel(PlannerInfo *root, int rel
*** 293,306 ****
}
/*
* build_join_rel_hash
* Construct the auxiliary hash table for join relations.
*/
static void
! build_join_rel_hash(PlannerInfo *root)
{
HTAB *hashtab;
HASHCTL hash_ctl;
ListCell *l;
/* Create the hash table */
--- 313,349 ----
}
/*
+ * find_grouped_base_rel
+ * Find a grouped base or other relation entry, which does not have to
+ * exist.
+ */
+ RelOptInfo *
+ find_grouped_base_rel(PlannerInfo *root, int relid)
+ {
+ RelOptInfo *rel;
+
+ Assert(relid > 0);
+
+ if (relid < root->simple_rel_array_size)
+ {
+ rel = root->simple_grouped_rel_array[relid];
+ if (rel)
+ return rel;
+ }
+
+ return NULL;
+ }
+
+ /*
* build_join_rel_hash
* Construct the auxiliary hash table for join relations.
*/
static void
! build_join_rel_hash(PlannerInfo *root, bool grouped)
{
HTAB *hashtab;
HASHCTL hash_ctl;
+ List *join_rel_list;
ListCell *l;
/* Create the hash table */
*************** build_join_rel_hash(PlannerInfo *root)
*** 316,322 ****
HASH_ELEM | HASH_FUNCTION | HASH_COMPARE | HASH_CONTEXT);
/* Insert all the already-existing joinrels */
! foreach(l, root->join_rel_list)
{
RelOptInfo *rel = (RelOptInfo *) lfirst(l);
JoinHashEntry *hentry;
--- 359,367 ----
HASH_ELEM | HASH_FUNCTION | HASH_COMPARE | HASH_CONTEXT);
/* Insert all the already-existing joinrels */
! join_rel_list = !grouped ? root->join_rel_list :
! root->join_grouped_rel_list;
! foreach(l, join_rel_list)
{
RelOptInfo *rel = (RelOptInfo *) lfirst(l);
JoinHashEntry *hentry;
*************** build_join_rel_hash(PlannerInfo *root)
*** 330,336 ****
hentry->join_rel = rel;
}
! root->join_rel_hash = hashtab;
}
/*
--- 375,384 ----
hentry->join_rel = rel;
}
! if (!grouped)
! root->join_rel_hash = hashtab;
! else
! root->join_grouped_rel_hash = hashtab;
}
/*
*************** build_join_rel_hash(PlannerInfo *root)
*** 339,352 ****
* or NULL if none exists. This is for join relations.
*/
RelOptInfo *
! find_join_rel(PlannerInfo *root, Relids relids)
{
/*
* Switch to using hash lookup when list grows "too long". The threshold
* is arbitrary and is known only here.
*/
! if (!root->join_rel_hash && list_length(root->join_rel_list) > 32)
! build_join_rel_hash(root);
/*
* Use either hashtable lookup or linear search, as appropriate.
--- 387,419 ----
* or NULL if none exists. This is for join relations.
*/
RelOptInfo *
! find_join_rel(PlannerInfo *root, Relids relids, bool grouped)
{
+ HTAB *join_rel_hash;
+ List *join_rel_list;
+
+ if (!grouped)
+ {
+ join_rel_hash = root->join_rel_hash;
+ join_rel_list = root->join_rel_list;
+ }
+ else
+ {
+ join_rel_hash = root->join_grouped_rel_hash;
+ join_rel_list = root->join_grouped_rel_list;
+ }
+
/*
* Switch to using hash lookup when list grows "too long". The threshold
* is arbitrary and is known only here.
*/
! if (!join_rel_hash && list_length(join_rel_list) > 32)
! {
! build_join_rel_hash(root, grouped);
!
! join_rel_hash = !grouped ? root->join_rel_hash :
! root->join_grouped_rel_hash;
! }
/*
* Use either hashtable lookup or linear search, as appropriate.
*************** find_join_rel(PlannerInfo *root, Relids
*** 356,367 ****
* so would force relids out of a register and thus probably slow down the
* list-search case.
*/
! if (root->join_rel_hash)
{
Relids hashkey = relids;
JoinHashEntry *hentry;
! hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
&hashkey,
HASH_FIND,
NULL);
--- 423,434 ----
* so would force relids out of a register and thus probably slow down the
* list-search case.
*/
! if (join_rel_hash)
{
Relids hashkey = relids;
JoinHashEntry *hentry;
! hentry = (JoinHashEntry *) hash_search(join_rel_hash,
&hashkey,
HASH_FIND,
NULL);
*************** find_join_rel(PlannerInfo *root, Relids
*** 372,378 ****
{
ListCell *l;
! foreach(l, root->join_rel_list)
{
RelOptInfo *rel = (RelOptInfo *) lfirst(l);
--- 439,445 ----
{
ListCell *l;
! foreach(l, join_rel_list)
{
RelOptInfo *rel = (RelOptInfo *) lfirst(l);
*************** set_foreign_rel_properties(RelOptInfo *j
*** 440,457 ****
* PlannerInfo. Also add it to the auxiliary hashtable if there is one.
*/
static void
! add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
{
! /* GEQO requires us to append the new joinrel to the end of the list! */
! root->join_rel_list = lappend(root->join_rel_list, joinrel);
/* store it into the auxiliary hashtable if there is one. */
! if (root->join_rel_hash)
{
JoinHashEntry *hentry;
bool found;
! hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
&(joinrel->relids),
HASH_ENTER,
&found);
--- 507,537 ----
* PlannerInfo. Also add it to the auxiliary hashtable if there is one.
*/
static void
! add_join_rel(PlannerInfo *root, RelOptInfo *joinrel, bool grouped)
{
! HTAB *join_rel_hash;
!
! if (!grouped)
! {
! /*
! * GEQO requires us to append the new joinrel to the end of the list!
! */
! root->join_rel_list = lappend(root->join_rel_list, joinrel);
! }
! else
! root->join_grouped_rel_list = lappend(root->join_grouped_rel_list,
! joinrel);
!
! join_rel_hash = !grouped ? root->join_rel_hash :
! root->join_grouped_rel_hash;
/* store it into the auxiliary hashtable if there is one. */
! if (join_rel_hash)
{
JoinHashEntry *hentry;
bool found;
! hentry = (JoinHashEntry *) hash_search(join_rel_hash,
&(joinrel->relids),
HASH_ENTER,
&found);
*************** add_join_rel(PlannerInfo *root, RelOptIn
*** 472,477 ****
--- 552,559 ----
* 'restrictlist_ptr': result variable. If not NULL, *restrictlist_ptr
* receives the list of RestrictInfo nodes that apply to this
* particular pair of joinable relations.
+ * 'grouped': does the join contain partial aggregate? (If it does, then
+ * caller is responsible for setup of reltarget.)
*
* restrictlist_ptr makes the routine's API a little grotty, but it saves
* duplicated calculation of the restrictlist...
*************** build_join_rel(PlannerInfo *root,
*** 482,491 ****
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
! List **restrictlist_ptr)
{
RelOptInfo *joinrel;
List *restrictlist;
/* This function should be used only for join between parents. */
Assert(!IS_OTHER_REL(outer_rel) && !IS_OTHER_REL(inner_rel));
--- 564,575 ----
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
! List **restrictlist_ptr,
! bool grouped)
{
RelOptInfo *joinrel;
List *restrictlist;
+ bool create_target = !grouped;
/* This function should be used only for join between parents. */
Assert(!IS_OTHER_REL(outer_rel) && !IS_OTHER_REL(inner_rel));
*************** build_join_rel(PlannerInfo *root,
*** 493,499 ****
/*
* See if we already have a joinrel for this set of base rels.
*/
! joinrel = find_join_rel(root, joinrelids);
if (joinrel)
{
--- 577,583 ----
/*
* See if we already have a joinrel for this set of base rels.
*/
! joinrel = find_join_rel(root, joinrelids, grouped);
if (joinrel)
{
*************** build_join_rel(PlannerInfo *root,
*** 516,526 ****
joinrel->reloptkind = RELOPT_JOINREL;
joinrel->relids = bms_copy(joinrelids);
joinrel->rows = 0;
! /* cheap startup cost is interesting iff not all tuples to be retrieved */
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
! joinrel->reltarget = create_empty_pathtarget();
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
joinrel->partial_pathlist = NIL;
--- 600,610 ----
joinrel->reloptkind = RELOPT_JOINREL;
joinrel->relids = bms_copy(joinrelids);
joinrel->rows = 0;
! /* See the comment in build_simple_rel(). */
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
! joinrel->reltarget = NULL;
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
joinrel->partial_pathlist = NIL;
*************** build_join_rel(PlannerInfo *root,
*** 534,539 ****
--- 618,624 ----
inner_rel->direct_lateral_relids);
joinrel->lateral_relids = min_join_parameterization(root, joinrel->relids,
outer_rel, inner_rel);
+ joinrel->agg_info = NULL;
joinrel->relid = 0; /* indicates not a baserel */
joinrel->rtekind = RTE_JOIN;
joinrel->min_attr = 0;
*************** build_join_rel(PlannerInfo *root,
*** 582,590 ****
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
! build_joinrel_tlist(root, joinrel, outer_rel);
! build_joinrel_tlist(root, joinrel, inner_rel);
! add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
--- 667,679 ----
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
! if (create_target)
! {
! joinrel->reltarget = create_empty_pathtarget();
! build_joinrel_tlist(root, joinrel, outer_rel);
! build_joinrel_tlist(root, joinrel, inner_rel);
! add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
! }
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
*************** build_join_rel(PlannerInfo *root,
*** 621,651 ****
/*
* Set estimates of the joinrel's size.
- */
- set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
- sjinfo, restrictlist);
-
- /*
- * Set the consider_parallel flag if this joinrel could potentially be
- * scanned within a parallel worker. If this flag is false for either
- * inner_rel or outer_rel, then it must be false for the joinrel also.
- * Even if both are true, there might be parallel-restricted expressions
- * in the targetlist or quals.
*
! * Note that if there are more than two rels in this relation, they could
! * be divided between inner_rel and outer_rel in any arbitrary way. We
! * assume this doesn't matter, because we should hit all the same baserels
! * and joinclauses while building up to this joinrel no matter which we
! * take; therefore, we should make the same decision here however we get
! * here.
*/
! if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
! is_parallel_safe(root, (Node *) restrictlist) &&
! is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
! joinrel->consider_parallel = true;
/* Add the joinrel to the PlannerInfo. */
! add_join_rel(root, joinrel);
/*
* Also, if dynamic-programming join search is active, add the new joinrel
--- 710,747 ----
/*
* Set estimates of the joinrel's size.
*
! * XXX The function claims to need reltarget but it does not seem to
! * actually use it. Should we call it unconditionally so that callers of
! * build_join_rel() do not have to care?
*/
! if (create_target)
! {
! set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
! sjinfo, restrictlist);
!
! /*
! * Set the consider_parallel flag if this joinrel could potentially be
! * scanned within a parallel worker. If this flag is false for either
! * inner_rel or outer_rel, then it must be false for the joinrel also.
! * Even if both are true, there might be parallel-restricted
! * expressions in the targetlist or quals.
! *
! * Note that if there are more than two rels in this relation, they
! * could be divided between inner_rel and outer_rel in any arbitrary
! * way. We assume this doesn't matter, because we should hit all the
! * same baserels and joinclauses while building up to this joinrel no
! * matter which we take; therefore, we should make the same decision
! * here however we get here.
! */
! if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
! is_parallel_safe(root, (Node *) restrictlist) &&
! is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
! joinrel->consider_parallel = true;
! }
/* Add the joinrel to the PlannerInfo. */
! add_join_rel(root, joinrel, grouped);
/*
* Also, if dynamic-programming join search is active, add the new joinrel
*************** build_join_rel(PlannerInfo *root,
*** 653,664 ****
* of members should be for equality, but some of the level 1 rels might
* have been joinrels already, so we can only assert <=.
*/
! if (root->join_rel_level)
{
Assert(root->join_cur_level > 0);
Assert(root->join_cur_level <= bms_num_members(joinrel->relids));
! root->join_rel_level[root->join_cur_level] =
! lappend(root->join_rel_level[root->join_cur_level], joinrel);
}
return joinrel;
--- 749,766 ----
* of members should be for equality, but some of the level 1 rels might
* have been joinrels already, so we can only assert <=.
*/
! if ((!grouped && root->join_rel_level) ||
! (grouped && root->join_grouped_rel_level))
{
Assert(root->join_cur_level > 0);
Assert(root->join_cur_level <= bms_num_members(joinrel->relids));
! if (!grouped)
! root->join_rel_level[root->join_cur_level] =
! lappend(root->join_rel_level[root->join_cur_level], joinrel);
! else
! root->join_grouped_rel_level[root->join_cur_level] =
! lappend(root->join_grouped_rel_level[root->join_cur_level],
! joinrel);
}
return joinrel;
*************** build_join_rel(PlannerInfo *root,
*** 677,692 ****
* 'restrictlist': list of RestrictInfo nodes that apply to this particular
* pair of joinable relations
* 'jointype' is the join type (inner, left, full, etc)
*/
RelOptInfo *
build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
RelOptInfo *inner_rel, RelOptInfo *parent_joinrel,
List *restrictlist, SpecialJoinInfo *sjinfo,
! JoinType jointype)
{
RelOptInfo *joinrel = makeNode(RelOptInfo);
AppendRelInfo **appinfos;
int nappinfos;
/* Only joins between "other" relations land here. */
Assert(IS_OTHER_REL(outer_rel) && IS_OTHER_REL(inner_rel));
--- 779,797 ----
* 'restrictlist': list of RestrictInfo nodes that apply to this particular
* pair of joinable relations
* 'jointype' is the join type (inner, left, full, etc)
+ * 'grouped': does the join contain partial aggregate? (If it does, then
+ * caller is responsible for setup of reltarget.)
*/
RelOptInfo *
build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
RelOptInfo *inner_rel, RelOptInfo *parent_joinrel,
List *restrictlist, SpecialJoinInfo *sjinfo,
! JoinType jointype, bool grouped)
{
RelOptInfo *joinrel = makeNode(RelOptInfo);
AppendRelInfo **appinfos;
int nappinfos;
+ bool create_target = !grouped;
/* Only joins between "other" relations land here. */
Assert(IS_OTHER_REL(outer_rel) && IS_OTHER_REL(inner_rel));
*************** build_child_join_rel(PlannerInfo *root,
*** 694,704 ****
joinrel->reloptkind = RELOPT_OTHER_JOINREL;
joinrel->relids = bms_union(outer_rel->relids, inner_rel->relids);
joinrel->rows = 0;
! /* cheap startup cost is interesting iff not all tuples to be retrieved */
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
! joinrel->reltarget = create_empty_pathtarget();
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
joinrel->partial_pathlist = NIL;
--- 799,809 ----
joinrel->reloptkind = RELOPT_OTHER_JOINREL;
joinrel->relids = bms_union(outer_rel->relids, inner_rel->relids);
joinrel->rows = 0;
! /* See the comment in build_simple_rel(). */
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
! joinrel->reltarget = NULL;
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
joinrel->partial_pathlist = NIL;
*************** build_child_join_rel(PlannerInfo *root,
*** 708,713 ****
--- 813,819 ----
joinrel->cheapest_parameterized_paths = NIL;
joinrel->direct_lateral_relids = NULL;
joinrel->lateral_relids = NULL;
+ joinrel->agg_info = NULL;
joinrel->relid = 0; /* indicates not a baserel */
joinrel->rtekind = RTE_JOIN;
joinrel->min_attr = 0;
*************** build_child_join_rel(PlannerInfo *root,
*** 744,754 ****
/* Compute information relevant to foreign relations. */
set_foreign_rel_properties(joinrel, outer_rel, inner_rel);
! /* Build targetlist */
! build_joinrel_tlist(root, joinrel, outer_rel);
! build_joinrel_tlist(root, joinrel, inner_rel);
! /* Add placeholder variables. */
! add_placeholders_to_child_joinrel(root, joinrel, parent_joinrel);
/* Construct joininfo list. */
appinfos = find_appinfos_by_relids(root, joinrel->relids, &nappinfos);
--- 850,864 ----
/* Compute information relevant to foreign relations. */
set_foreign_rel_properties(joinrel, outer_rel, inner_rel);
! if (create_target)
! {
! /* Build targetlist */
! joinrel->reltarget = create_empty_pathtarget();
! build_joinrel_tlist(root, joinrel, outer_rel);
! build_joinrel_tlist(root, joinrel, inner_rel);
! /* Add placeholder variables. */
! add_placeholders_to_child_joinrel(root, joinrel, parent_joinrel);
! }
/* Construct joininfo list. */
appinfos = find_appinfos_by_relids(root, joinrel->relids, &nappinfos);
*************** build_child_join_rel(PlannerInfo *root,
*** 756,762 ****
(Node *) parent_joinrel->joininfo,
nappinfos,
appinfos);
- pfree(appinfos);
/*
* Lateral relids referred in child join will be same as that referred in
--- 866,871 ----
*************** build_child_join_rel(PlannerInfo *root,
*** 783,796 ****
/* Set estimates of the child-joinrel's size. */
! set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
! sjinfo, restrictlist);
/* We build the join only once. */
! Assert(!find_join_rel(root, joinrel->relids));
/* Add the relation to the PlannerInfo. */
! add_join_rel(root, joinrel);
return joinrel;
}
--- 892,909 ----
/* Set estimates of the child-joinrel's size. */
! /* XXX See the corresponding comment in build_join_rel(). */
! if (create_target)
! set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
! sjinfo, restrictlist);
/* We build the join only once. */
! Assert(!find_join_rel(root, joinrel->relids, grouped));
/* Add the relation to the PlannerInfo. */
! add_join_rel(root, joinrel, grouped);
!
! pfree(appinfos);
return joinrel;
}
*************** build_joinrel_partition_info(RelOptInfo
*** 1751,1753 ****
--- 1864,2471 ----
joinrel->nullable_partexprs[cnt] = nullable_partexpr;
}
}
+
+ /*
+ * Check if the relation can produce grouped paths and return the information
+ * it'll need for it. The passed relation is the non-grouped one which has the
+ * reltarget already constructed.
+ */
+ RelAggInfo *
+ create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel)
+ {
+ List *gvis;
+ List *aggregates = NIL;
+ List *grp_exprs = NIL;
+ bool found_higher_agg;
+ ListCell *lc;
+ RelAggInfo *result;
+ PathTarget *target,
+ *agg_input;
+ List *grp_exprs_extra = NIL;
+ int i;
+ List *sortgroupclauses = NIL;
+
+ /*
+ * The function shouldn't have been called if there's no opportunity for
+ * aggregation push-down.
+ */
+ Assert(root->grouped_var_list != NIL);
+
+ /*
+ * The source relation has nothing to do with grouping.
+ */
+ Assert(rel->agg_info == NULL);
+
+ /*
+ * The current implementation of aggregation push-down cannot handle
+ * PlaceHolderVar (PHV).
+ *
+ * If we knew that the PHV should be evaluated in this target (and of
+ * course, if its expression matched some grouping expression or Aggref
+ * argument), we'd just let init_grouping_targets create GroupedVar for
+ * the corresponding expression (phexpr). On the other hand, if we knew
+ * that the PHV is evaluated below the current rel, we'd ignore it because
+ * the referencing GroupedVar would take care of propagation of the value
+ * to upper joins. (PHV whose ph_eval_at is above the current rel make the
+ * aggregation push-down impossible in any case because the partial
+ * aggregation would receive wrong input if we ignored the ph_eval_at.)
+ *
+ * The problem is that the same PHV can be evaluated in the target of the
+ * current rel or in that of lower rel --- depending on the input paths.
+ * For example, consider rel->relids = {A, B, C} and if ph_eval_at = {B,
+ * C}. Path "A JOIN (B JOIN C)" implies that the PHV is evaluated by the
+ * "(B JOIN C)", while path "(A JOIN B) JOIN C" evaluates the PHV itself.
+ */
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *expr = lfirst(lc);
+
+ if (IsA(expr, PlaceHolderVar))
+ return NULL;
+ }
+
+ if (IS_SIMPLE_REL(rel))
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return NULL;
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the Agg plan
+ * would receive different input at the base rel level.
+ *
+ * XXX For RELOPT_JOINREL, do not return if all the joins that can set any
+ * entry of the grouped target (do we need to postpone this check until
+ * the grouped target is available, and init_grouping_targets take care?)
+ * of this rel to NULL are provably below rel. (It's ok if rel is one of
+ * these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return NULL;
+
+ /*
+ * Use equivalence classes to generate additional grouping expressions for
+ * the current rel. Without these we might not be able to apply
+ * aggregation to the relation result set.
+ *
+ * It's important that create_grouping_expr_grouped_var_infos has
+ * processed the explicit grouping columns by now. If the grouping clause
+ * contains multiple expressions belonging to the same EC, the original
+ * (i.e. not derived) one should be preferred when we build grouping
+ * target for a relation. Otherwise we have a problem when trying to match
+ * target entries to grouping clauses during plan creation, see
+ * get_grouping_expression().
+ */
+ gvis = list_copy(root->grouped_var_list);
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+ int relid = -1;
+
+ /* Only interested in grouping expressions. */
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ while ((relid = bms_next_member(rel->relids, relid)) >= 0)
+ {
+ GroupedVarInfo *gvi_trans;
+
+ gvi_trans = translate_expression_to_rels(root, gvi, relid);
+ if (gvi_trans != NULL)
+ gvis = lappend(gvis, gvi_trans);
+ }
+ }
+
+ /*
+ * Check if some aggregates or grouping expressions can be evaluated in
+ * this relation's target, and collect all vars referenced by these
+ * aggregates / grouping expressions;
+ */
+ found_higher_agg = false;
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which includes
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * init_grouping_targets will handle plain Var grouping
+ * expressions because it needs to look them up in
+ * grouped_var_list anyway.
+ *
+ * XXX A plain Var could actually be handled w/o GroupedVar, but
+ * thus init_grouping_targets would have to spend extra effort
+ * looking for the EC-related vars, instead of relying on
+ * create_grouping_expr_grouped_var_infos. (Processing of
+ * particular expression would look different, so we could hardly
+ * reuse the same piece of code.)
+ */
+ if (IsA(gvi->gvexpr, Var))
+ continue;
+
+ /*
+ * Accept the aggregate / grouping expression.
+ *
+ * (GroupedVarInfo is more convenient for the next processing than
+ * Aggref, see add_aggregates_to_grouped_target.)
+ */
+ if (IsA(gvi->gvexpr, Aggref))
+ aggregates = lappend(aggregates, gvi);
+ else
+ grp_exprs = lappend(grp_exprs, gvi);
+ }
+ else if (bms_overlap(gvi->gv_eval_at, rel->relids) &&
+ IsA(gvi->gvexpr, Aggref))
+ {
+ /*
+ * Remember that there is at least one aggregate expression that
+ * needs more than this rel.
+ */
+ found_higher_agg = true;
+ }
+ }
+
+ /*
+ * Grouping makes little sense w/o aggregate function and w/o grouping
+ * expressions.
+ */
+ if (aggregates == NIL)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Give up if some other aggregate(s) need multiple relations including
+ * the current one. The problem is that grouping of the current relation
+ * could make some input variables unavailable for the "higher aggregate",
+ * and it'd also decrease the number of input rows the "higher aggregate"
+ * receives.
+ *
+ * In contrast, grp_exprs is only supposed to contain generic grouping
+ * expression, so it can be NIL so far. If all the grouping keys are just
+ * plain Vars, init_grouping_targets will take care of them.
+ */
+ if (found_higher_agg)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Create target for grouped paths as well as one for the input paths of
+ * the partial aggregation paths.
+ */
+ target = create_empty_pathtarget();
+ agg_input = create_empty_pathtarget();
+ init_grouping_targets(root, rel, target, agg_input, gvis,
+ &grp_exprs_extra);
+ list_free(gvis);
+
+ /*
+ * Add (non-Var) grouping expressions (in the form of GroupedVar) to
+ * target_agg.
+ *
+ * Follow the convention that the grouping expressions should precede
+ * aggregates.
+ */
+ add_grouped_vars_to_target(root, target, grp_exprs);
+
+ /*
+ * Partial aggregation makes no sense w/o grouping expressions.
+ */
+ if (list_length(target->exprs) == 0)
+ return NULL;
+
+ /*
+ * If the aggregation target should have extra grouping expressions, add
+ * them now. This step includes assignment of tleSortGroupRef's which we
+ * can generate now (the "ordinary" grouping expression are present in the
+ * target by now).
+ */
+ if (list_length(grp_exprs_extra) > 0)
+ {
+ Index sortgroupref;
+
+ /*
+ * Always start at root->max_sortgroupref. The extra grouping
+ * expressions aren't used during the final aggregation, so the
+ * sortgroupref values don't need to be unique across the query. Thus
+ * we don't have to increase root->max_sortgroupref, which makes
+ * recognition of the extra grouping expressions pretty easy.
+ */
+ sortgroupref = root->max_sortgroupref;
+
+ /*
+ * Generate the SortGroupClause's and add the expressions to the
+ * target.
+ */
+ foreach(lc, grp_exprs_extra)
+ {
+ Var *var = lfirst_node(Var, lc);
+ SortGroupClause *cl = makeNode(SortGroupClause);
+ int i = 0;
+ ListCell *lc2;
+
+ /*
+ * TODO Verify that these fields are sufficient for this special
+ * SortGroupClause.
+ */
+ cl->tleSortGroupRef = ++sortgroupref;
+ get_sort_group_operators(var->vartype,
+ false, true, false,
+ NULL, &cl->eqop, NULL,
+ &cl->hashable);
+ sortgroupclauses = lappend(sortgroupclauses, cl);
+ add_column_to_pathtarget(target, (Expr *) var,
+ cl->tleSortGroupRef);
+
+ /*
+ * The aggregation input target must emit this var too. It can
+ * already be there, so avoid adding it again.
+ */
+ foreach(lc2, agg_input->exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc2);
+
+ if (equal(expr, var))
+ {
+ /*
+ * The fact that the var is in agg_input does not imply
+ * that it has sortgroupref set. For example, the reason
+ * that it's there can be that a generic grouping
+ * expression references it, so grouping by the var alone
+ * hasn't been considered so far.
+ */
+ if (agg_input->sortgrouprefs == NULL)
+ {
+ agg_input->sortgrouprefs = (Index *)
+ palloc0(list_length(agg_input->exprs) *
+ sizeof(Index));
+ }
+ if (agg_input->sortgrouprefs[i] == 0)
+ agg_input->sortgrouprefs[i] = cl->tleSortGroupRef;
+
+ break;
+ }
+
+ i++;
+ }
+ if (lc2 != NULL)
+ continue;
+
+ /*
+ * Add the var if it's not in the target yet.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) var,
+ cl->tleSortGroupRef);
+ }
+ }
+
+ /*
+ * Add aggregates (in the form of GroupedVar) to the grouping target.
+ */
+ add_grouped_vars_to_target(root, target, aggregates);
+
+ /*
+ * Make sure that the paths generating input data for partial aggregation
+ * include non-Var grouping expressions.
+ */
+ add_grouped_vars_to_target(root, agg_input, grp_exprs);
+
+ /*
+ * Since neither target nor agg_input is supposed to be identical to the
+ * source reltarget, compute the width and cost again.
+ */
+ set_pathtarget_cost_width(root, target);
+ set_pathtarget_cost_width(root, agg_input);
+
+ result = makeNode(RelAggInfo);
+ result->target = target;
+ result->input = agg_input;
+
+ /*
+ * Build a list of grouping expressions and a list of the corresponding
+ * SortGroupClauses.
+ */
+ i = 0;
+ foreach(lc, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(lc);
+
+ if (IsA(texpr, GroupedVar) &&
+ IsA(((GroupedVar *) texpr)->gvexpr, Aggref))
+ {
+ /*
+ * texpr should represent the first aggregate in the targetlist.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides being an aggregate, the target expression should have no
+ * other reason then being a column of a relation functionally
+ * dependent on the GROUP BY clause. So it's not actually a grouping
+ * column.
+ */
+ if (sortgroupref == 0)
+ continue;
+
+ cl = get_sortgroupref_clause_noerr(sortgroupref,
+ root->parse->groupClause);
+
+ /*
+ * If query does not have this clause, it must be target-specific.
+ */
+ if (cl == NULL)
+ cl = get_sortgroupref_clause(sortgroupref, sortgroupclauses);
+
+ result->group_clauses = list_append_unique(result->group_clauses,
+ cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ result->group_exprs = list_append_unique(result->group_exprs,
+ texpr);
+ }
+
+ /* Finally collect the aggregates. */
+ while (lc != NULL)
+ {
+ GroupedVar *gvar = castNode(GroupedVar, lfirst(lc));
+
+ /*
+ * Aggregate GroupedVarInfo should always point to the partial
+ * aggregate.
+ */
+ Assert(gvar->agg_partial != NULL);
+ result->agg_exprs = lappend(result->agg_exprs, gvar->agg_partial);
+ lc = lnext(lc);
+ }
+
+ return result;
+ }
+
+ /*
+ * Initialize target for grouped paths (target) as well as a target for paths
+ * that generate input for partial aggregation (agg_input).
+ *
+ * gvis a list of GroupedVarInfo's possibly useful for rel.
+ *
+ * The *group_exprs_extra_p list may receive additional grouping expressions
+ * that the query does not have. These can make the aggregation of base
+ * relation / join less efficient, but can allow for join of the grouped
+ * relation that wouldn't be possible otherwise.
+ */
+ static void
+ init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p)
+ {
+ ListCell *lc;
+ List *vars_unresolved = NIL;
+
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Var *tvar;
+ GroupedVar *gvar;
+
+ /*
+ * Given that PlaceHolderVar currently prevents us from doing
+ * aggregation push-down, the source target cannot contain anything
+ * more complex than a Var. (As for generic grouping expressions,
+ * add_grouped_vars_to_target will retrieve them from the query
+ * targetlist and add them to "target" outside this function.)
+ */
+ tvar = lfirst_node(Var, lc);
+
+ gvar = get_grouping_expression(gvis, (Expr *) tvar);
+ if (gvar != NULL)
+ {
+ /*
+ * It's o.k. to use the target expression for grouping.
+ *
+ * The actual Var is added to the target. If we used the
+ * containing GroupedVar, references from various clauses (e.g.
+ * join quals) wouldn't work.
+ */
+ add_column_to_pathtarget(target, gvar->gvexpr,
+ gvar->sortgroupref);
+
+ /*
+ * As for agg_input, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(agg_input, gvar->gvexpr,
+ gvar->sortgroupref);
+
+ /* Process the next expression. */
+ continue;
+ }
+
+ /*
+ * Further investigation involves dependency check, for which we need
+ * to have all the plain-var grouping expressions gathered. So far
+ * only store the var in a list.
+ */
+ vars_unresolved = lappend(vars_unresolved, tvar);
+ }
+
+ /*
+ * Check for other possible reasons for the var to be in the plain target.
+ */
+ foreach(lc, vars_unresolved)
+ {
+ Var *var;
+ RangeTblEntry *rte;
+ List *deps = NIL;
+ Relids relids_subtract;
+ int ndx;
+ RelOptInfo *baserel;
+
+ var = lfirst_node(Var, lc);
+ rte = root->simple_rte_array[var->varno];
+
+ /*
+ * Dependent var is almost the same as one that has sortgroupref.
+ */
+ if (check_functional_grouping(rte->relid, var->varno,
+ var->varlevelsup,
+ target->exprs, &deps))
+ {
+
+ Index sortgroupref = 0;
+
+ add_column_to_pathtarget(target, (Expr *) var, sortgroupref);
+
+ /*
+ * The var shouldn't be actually used as a grouping key (instead,
+ * the one this depends on will be), so sortgroupref should not be
+ * important. But once we have it ...
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) var, sortgroupref);
+
+ /*
+ * The var may or may not be present in generic grouping
+ * expression(s) or aggregate arguments, but we already have it in
+ * the targets, so don't care.
+ */
+ continue;
+ }
+
+ /*
+ * Isn't the expression needed by joins above the current rel?
+ *
+ * The relids we're not interested in do include 0, which is the
+ * top-level targetlist. The only reason for relids to contain 0
+ * should be that arg_var is referenced either by aggregate or by
+ * grouping expression, but right now we're interested in the *other*
+ * reasons. (As soon as GroupedVars are installed, the top level
+ * aggregates / grouping expressions no longer need direct reference
+ * to arg_var anyway.)
+ */
+ relids_subtract = bms_copy(rel->relids);
+ bms_add_member(relids_subtract, 0);
+
+ baserel = find_base_rel(root, var->varno);
+ ndx = var->varattno - baserel->min_attr;
+ if (bms_nonempty_difference(baserel->attr_needed[ndx],
+ relids_subtract))
+ {
+ /*
+ * The variable is needed by upper join. This includes one that is
+ * referenced by a generic grouping expression but couldn't be
+ * recognized as grouping expression on its own at the top of the
+ * loop.
+ *
+ * The only way to bring this var to the aggregation output is to
+ * add it to the grouping expressions too.
+ *
+ * Since root->parse->groupClause is not supposed to contain this
+ * expression, we need to construct special SortGroupClause. Its
+ * tleSortGroupRef needs to be unique within "target", so postpone
+ * creation of the SortGroupRefs until we're done with the
+ * iteration of rel->reltarget->exprs.
+ */
+ *group_exprs_extra_p = lappend(*group_exprs_extra_p, var);
+ }
+ else
+ {
+ /*
+ * As long as the query is semantically correct, arriving here
+ * means that the var is referenced either by aggregate argument
+ * or by generic grouping expression. The per-relation aggregation
+ * target should not contain it, as it only provides input for the
+ * final aggregation.
+ */
+ }
+
+ /*
+ * The var is not suitable for grouping, but agg_input ought to stay
+ * complete.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) var, 0);
+ }
+ }
+
+
+ /*
+ * Translate RelAggInfo of parent relation so it matches given child relation.
+ */
+ RelAggInfo *
+ translate_rel_agg_info(PlannerInfo *root, RelAggInfo *parent,
+ AppendRelInfo **appinfos, int nappinfos)
+ {
+ RelAggInfo *result;
+
+ result = makeNode(RelAggInfo);
+
+ result->target = copy_pathtarget(parent->target);
+ result->target->exprs = (List *)
+ adjust_appendrel_attrs(root,
+ (Node *) result->target->exprs,
+ nappinfos, appinfos);
+
+ result->input = copy_pathtarget(parent->input);
+ result->input->exprs = (List *)
+ adjust_appendrel_attrs(root,
+ (Node *) result->input->exprs,
+ nappinfos, appinfos);
+
+ result->group_clauses = parent->group_clauses;
+
+ result->group_exprs = (List *)
+ adjust_appendrel_attrs(root,
+ (Node *) parent->group_exprs,
+ nappinfos, appinfos);
+ result->agg_exprs = (List *)
+ adjust_appendrel_attrs(root,
+ (Node *) parent->agg_exprs,
+ nappinfos, appinfos);
+ return result;
+ }
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
new file mode 100644
index 32160d5..6d74323
*** a/src/backend/optimizer/util/tlist.c
--- b/src/backend/optimizer/util/tlist.c
*************** get_sortgrouplist_exprs(List *sgClauses,
*** 408,414 ****
return result;
}
-
/*****************************************************************************
* Functions to extract data from a list of SortGroupClauses
*
--- 408,413 ----
*************** apply_pathtarget_labeling_to_tlist(List
*** 783,788 ****
--- 782,900 ----
}
/*
+ * Replace each GroupedVar in the source targetlist with the original
+ * expression --- either Aggref or a non-Var grouping expression.
+ *
+ * Even if the query targetlist has the Aggref wrapped in a generic
+ * expression, any subplan should emit the corresponding GroupedVar
+ * alone. (Aggregate finalization is needed before the aggregate result can be
+ * used for any purposes and that happens at the top level of the query.)
+ * Therefore we do not have to recurse into the target expressions here.
+ */
+ List *
+ replace_grouped_vars_with_aggrefs(PlannerInfo *root, List *src)
+ {
+ List *result = NIL;
+ ListCell *l;
+
+ foreach(l, src)
+ {
+ TargetEntry *te,
+ *te_new;
+ Expr *expr_new = NULL;
+
+ te = lfirst_node(TargetEntry, l);
+
+ if (IsA(te->expr, GroupedVar))
+ {
+ GroupedVar *gvar;
+
+ gvar = castNode(GroupedVar, te->expr);
+ if (IsA(gvar->gvexpr, Aggref))
+ {
+ /*
+ * Partial aggregate should appear in the targetlist so that
+ * it looks as if convert_combining_aggrefs arranged it.
+ */
+ expr_new = (Expr *) gvar->agg_partial;
+ }
+ else
+ expr_new = gvar->gvexpr;
+ }
+
+ if (expr_new != NULL)
+ {
+ te_new = flatCopyTargetEntry(te);
+ te_new->expr = (Expr *) expr_new;
+ }
+ else
+ te_new = te;
+ result = lappend(result, te_new);
+ }
+
+ return result;
+ }
+
+ /*
+ * For each aggregate add GroupedVar to the grouped target.
+ *
+ * Caller passes the aggregates in the form of GroupedVarInfos so that we
+ * don't have to look for gvid.
+ */
+ void
+ add_grouped_vars_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions)
+ {
+ ListCell *lc;
+
+ /* Create the vars and add them to the target. */
+ foreach(lc, expressions)
+ {
+ GroupedVarInfo *gvi;
+ GroupedVar *gvar;
+
+ gvi = lfirst_node(GroupedVarInfo, lc);
+ gvar = makeNode(GroupedVar);
+ gvar->gvid = gvi->gvid;
+ gvar->gvexpr = gvi->gvexpr;
+ gvar->agg_partial = gvi->agg_partial;
+ add_column_to_pathtarget(target, (Expr *) gvar, gvi->sortgroupref);
+ }
+ }
+
+ /*
+ * Return GroupedVar containing the passed-in expression if one exists, or
+ * NULL if the expression cannot be used as grouping key.
+ */
+ GroupedVar *
+ get_grouping_expression(List *gvis, Expr *expr)
+ {
+ ListCell *lc;
+
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (equal(gvi->gvexpr, expr))
+ {
+ GroupedVar *result = makeNode(GroupedVar);
+
+ Assert(gvi->sortgroupref > 0);
+ result->gvexpr = gvi->gvexpr;
+ result->gvid = gvi->gvid;
+ result->sortgroupref = gvi->sortgroupref;
+ return result;
+ }
+ }
+
+ /* The expression cannot be used as grouping key. */
+ return NULL;
+ }
+
+ /*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
*
diff --git a/src/backend/optimizer/util/var.c b/src/backend/optimizer/util/var.c
new file mode 100644
index b16b1e4..459dc30
*** a/src/backend/optimizer/util/var.c
--- b/src/backend/optimizer/util/var.c
*************** alias_relid_set(PlannerInfo *root, Relid
*** 840,842 ****
--- 840,864 ----
}
return result;
}
+
+ /*
+ * Return GroupedVarInfo for given GroupedVar.
+ *
+ * XXX Consider better location of this routine.
+ */
+ GroupedVarInfo *
+ find_grouped_var_info(PlannerInfo *root, GroupedVar *gvar)
+ {
+ ListCell *l;
+
+ foreach(l, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, l);
+
+ if (gvi->gvid == gvar->gvid)
+ return gvi;
+ }
+
+ elog(ERROR, "GroupedVarInfo not found");
+ return NULL; /* keep compiler quiet */
+ }
diff --git a/src/backend/parser/parse_func.c b/src/backend/parser/parse_func.c
new file mode 100644
index 2a4ac09..267c0b9
*** a/src/backend/parser/parse_func.c
--- b/src/backend/parser/parse_func.c
*************** ParseFuncOrColumn(ParseState *pstate, Li
*** 98,103 ****
--- 98,104 ----
Oid vatype;
FuncDetailCode fdresult;
char aggkind = 0;
+ Oid aggcombinefn = InvalidOid;
ParseCallbackState pcbstate;
/*
*************** ParseFuncOrColumn(ParseState *pstate, Li
*** 350,355 ****
--- 351,357 ----
elog(ERROR, "cache lookup failed for aggregate %u", funcid);
classForm = (Form_pg_aggregate) GETSTRUCT(tup);
aggkind = classForm->aggkind;
+ aggcombinefn = classForm->aggcombinefn;
catDirectArgs = classForm->aggnumdirectargs;
ReleaseSysCache(tup);
*************** ParseFuncOrColumn(ParseState *pstate, Li
*** 695,700 ****
--- 697,703 ----
aggref->aggstar = agg_star;
aggref->aggvariadic = func_variadic;
aggref->aggkind = aggkind;
+ aggref->aggcombinefn = aggcombinefn;
/* agglevelsup will be set by transformAggregateCall */
aggref->aggsplit = AGGSPLIT_SIMPLE; /* planner might change this */
aggref->location = location;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
new file mode 100644
index 3697466..4b49d32
*** a/src/backend/utils/adt/ruleutils.c
--- b/src/backend/utils/adt/ruleutils.c
*************** get_rule_expr(Node *node, deparse_contex
*** 7686,7691 ****
--- 7686,7708 ----
get_agg_expr((Aggref *) node, context, (Aggref *) node);
break;
+ case T_GroupedVar:
+ {
+ GroupedVar *gvar = castNode(GroupedVar, node);
+ Expr *expr = gvar->gvexpr;
+
+ if (IsA(expr, Aggref))
+ get_agg_expr(gvar->agg_partial, context, (Aggref *) gvar->gvexpr);
+ else if (IsA(expr, Var))
+ (void) get_variable((Var *) expr, 0, false, context);
+ else
+ {
+ Assert(IsA(gvar->gvexpr, OpExpr));
+ get_oper_expr((OpExpr *) expr, context);
+ }
+ break;
+ }
+
case T_GroupingFunc:
{
GroupingFunc *gexpr = (GroupingFunc *) node;
*************** get_agg_combine_expr(Node *node, deparse
*** 9171,9180 ****
Aggref *aggref;
Aggref *original_aggref = private;
! if (!IsA(node, Aggref))
elog(ERROR, "combining Aggref does not point to an Aggref");
- aggref = (Aggref *) node;
get_agg_expr(aggref, context, original_aggref);
}
--- 9188,9205 ----
Aggref *aggref;
Aggref *original_aggref = private;
! if (IsA(node, Aggref))
! aggref = (Aggref *) node;
! else if (IsA(node, GroupedVar))
! {
! GroupedVar *gvar = castNode(GroupedVar, node);
!
! aggref = gvar->agg_partial;
! original_aggref = castNode(Aggref, gvar->gvexpr);
! }
! else
elog(ERROR, "combining Aggref does not point to an Aggref");
get_agg_expr(aggref, context, original_aggref);
}
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
new file mode 100644
index fcc8323..72e14e5
*** a/src/backend/utils/adt/selfuncs.c
--- b/src/backend/utils/adt/selfuncs.c
***************
*** 114,119 ****
--- 114,120 ----
#include "catalog/pg_statistic_ext.h"
#include "catalog/pg_type.h"
#include "executor/executor.h"
+ #include "executor/nodeAgg.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
*************** estimate_hash_bucket_stats(PlannerInfo *
*** 3863,3868 ****
--- 3864,3902 ----
ReleaseVariableStats(vardata);
}
+ /*
+ * estimate_hashagg_tablesize
+ * estimate the number of bytes that a hash aggregate hashtable will
+ * require based on the agg_costs, path width and dNumGroups.
+ *
+ * XXX this may be over-estimating the size now that hashagg knows to omit
+ * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
+ * grouping columns not in the hashed set are counted here even though hashagg
+ * won't store them. Is this a problem?
+ */
+ Size
+ estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
+ double dNumGroups)
+ {
+ Size hashentrysize;
+
+ /* Estimate per-hash-entry space at tuple width... */
+ hashentrysize = MAXALIGN(path->pathtarget->width) +
+ MAXALIGN(SizeofMinimalTupleHeader);
+
+ /* plus space for pass-by-ref transition values... */
+ hashentrysize += agg_costs->transitionSpace;
+ /* plus the per-hash-entry overhead */
+ hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
+
+ /*
+ * Note that this disregards the effect of fill-factor and growth policy
+ * of the hash-table. That's probably ok, given default the default
+ * fill-factor is relatively high. It'd be hard to meaningfully factor in
+ * "double-in-size" growth policies here.
+ */
+ return hashentrysize * dNumGroups;
+ }
/*-------------------------------------------------------------------------
*
*************** examine_variable(PlannerInfo *root, Node
*** 4793,4799 ****
if (varRelid == 0)
{
/* treat it as a variable of a join relation */
! vardata->rel = find_join_rel(root, varnos);
node = basenode; /* strip any relabeling */
}
else if (bms_is_member(varRelid, varnos))
--- 4827,4833 ----
if (varRelid == 0)
{
/* treat it as a variable of a join relation */
! vardata->rel = find_join_rel(root, varnos, false);
node = basenode; /* strip any relabeling */
}
else if (bms_is_member(varRelid, varnos))
*************** find_join_input_rel(PlannerInfo *root, R
*** 5651,5657 ****
rel = find_base_rel(root, bms_singleton_member(relids));
break;
case BMS_MULTIPLE:
! rel = find_join_rel(root, relids);
break;
}
--- 5685,5691 ----
rel = find_base_rel(root, bms_singleton_member(relids));
break;
case BMS_MULTIPLE:
! rel = find_join_rel(root, relids, false);
break;
}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
new file mode 100644
index 1db7845..0caa05e
*** a/src/backend/utils/misc/guc.c
--- b/src/backend/utils/misc/guc.c
*************** static struct config_bool ConfigureNames
*** 923,928 ****
--- 923,937 ----
NULL, NULL, NULL
},
{
+ {"enable_agg_pushdown", PGC_USERSET, QUERY_TUNING_METHOD,
+ gettext_noop("Enables aggregation push-down."),
+ NULL
+ },
+ &enable_agg_pushdown,
+ false,
+ NULL, NULL, NULL
+ },
+ {
{"enable_parallel_append", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables the planner's use of parallel append plans."),
NULL
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
new file mode 100644
index 74b094a..2914fc8
*** a/src/include/nodes/nodes.h
--- b/src/include/nodes/nodes.h
*************** typedef enum NodeTag
*** 218,223 ****
--- 218,224 ----
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_RelAggInfo,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
*************** typedef enum NodeTag
*** 258,267 ****
--- 259,270 ----
T_PathTarget,
T_RestrictInfo,
T_PlaceHolderVar,
+ T_GroupedVar,
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PartitionedChildRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
new file mode 100644
index 1b4b0d7..6af31f2
*** a/src/include/nodes/primnodes.h
--- b/src/include/nodes/primnodes.h
*************** typedef struct Aggref
*** 296,301 ****
--- 296,302 ----
Oid aggcollid; /* OID of collation of result */
Oid inputcollid; /* OID of collation that function should use */
Oid aggtranstype; /* type Oid of aggregate's transition value */
+ Oid aggcombinefn; /* combine function (see pg_aggregate.h) */
List *aggargtypes; /* type Oids of direct and aggregated args */
List *aggdirectargs; /* direct arguments, if an ordered-set agg */
List *args; /* aggregated arguments and sort expressions */
*************** typedef struct Aggref
*** 306,311 ****
--- 307,313 ----
bool aggvariadic; /* true if variadic arguments have been
* combined into an array last argument */
char aggkind; /* aggregate kind (see pg_aggregate.h) */
+
Index agglevelsup; /* > 0 if agg belongs to outer query */
AggSplit aggsplit; /* expected agg-splitting mode of parent Agg */
int location; /* token location, or -1 if unknown */
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
new file mode 100644
index db8de2d..1f10816
*** a/src/include/nodes/relation.h
--- b/src/include/nodes/relation.h
*************** typedef struct PlannerInfo
*** 179,185 ****
* unreferenced view RTE; or if the RelOptInfo hasn't been made yet.
*/
struct RelOptInfo **simple_rel_array; /* All 1-rel RelOptInfos */
! int simple_rel_array_size; /* allocated size of array */
/*
* simple_rte_array is the same length as simple_rel_array and holds
--- 179,193 ----
* unreferenced view RTE; or if the RelOptInfo hasn't been made yet.
*/
struct RelOptInfo **simple_rel_array; /* All 1-rel RelOptInfos */
!
! /*
! * The same like simple_rel_array, but for grouped rels. In addition to
! * the meanings explained above, NULL can also mean that the relation
! * cannot be grouped alone, regardless its kind.
! */
! struct RelOptInfo **simple_grouped_rel_array; /* The same for grouped
! * rels. */
! int simple_rel_array_size; /* allocated size of the arrays above */
/*
* simple_rte_array is the same length as simple_rel_array and holds
*************** typedef struct PlannerInfo
*** 217,222 ****
--- 225,234 ----
List *join_rel_list; /* list of join-relation RelOptInfos */
struct HTAB *join_rel_hash; /* optional hashtable for join relations */
+ /* The same for grouped joins. */
+ List *join_grouped_rel_list;
+ struct HTAB *join_grouped_rel_hash;
+
/*
* When doing a dynamic-programming-style join search, join_rel_level[k]
* is a list of all join-relation RelOptInfos of level k, and
*************** typedef struct PlannerInfo
*** 225,230 ****
--- 237,244 ----
* join_rel_level is NULL if not in use.
*/
List **join_rel_level; /* lists of join-relation RelOptInfos */
+ List **join_grouped_rel_level; /* lists of grouped join-relation
+ * RelOptInfos */
int join_cur_level; /* index of list being extended */
List *init_plans; /* init SubPlans for query */
*************** typedef struct PlannerInfo
*** 259,264 ****
--- 273,280 ----
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
*************** typedef struct PlannerInfo
*** 285,290 ****
--- 301,312 ----
*/
List *processed_tlist;
+ /*
+ * The maximum ressortgroupref among target entries in processed_list.
+ * Useful when adding extra grouping expressions for partial aggregation.
+ */
+ int max_sortgroupref;
+
/* Fields filled during create_plan() for use in setrefs.c */
AttrNumber *grouping_map; /* for GroupingFunc fixup */
List *minmax_aggs; /* List of MinMaxAggInfos */
*************** typedef struct PartitionSchemeData *Part
*** 440,445 ****
--- 462,469 ----
* direct_lateral_relids - rels this rel has direct LATERAL references to
* lateral_relids - required outer rels for LATERAL, as a Relids set
* (includes both direct and indirect lateral references)
+ * gpi - GroupedPathInfo if the relation can produce grouped paths, NULL
+ * otherwise.
*
* If the relation is a base relation it will have these fields set:
*
*************** typedef struct RelOptInfo
*** 611,616 ****
--- 635,643 ----
Relids direct_lateral_relids; /* rels directly laterally referenced */
Relids lateral_relids; /* minimum parameterization of rel */
+ /* Information needed to apply partial aggregation to this rel's paths. */
+ struct RelAggInfo *agg_info;
+
/* information about a base rel (not set for join rels!) */
Index relid;
Oid reltablespace; /* containing tablespace */
*************** typedef struct ParamPathInfo
*** 1009,1014 ****
--- 1036,1088 ----
List *ppi_clauses; /* join clauses available from outer rels */
} ParamPathInfo;
+ /*
+ * RelAggInfo
+ *
+ * RelOptInfo needs information contained here if its paths should be
+ * partially aggregated.
+ *
+ * "target" will be used as pathtarget of grouped paths produced by "explicit
+ * aggregation" of a relation, but also --- if the relation is a join --- by
+ * joining grouped path to a non-grouped one.
+ *
+ * The target contains plain-Var grouping expressions, generic grouping
+ * expressions wrapped in GroupedVar structure, or Aggrefs which are also
+ * wrapped in GroupedVar. Once GroupedVar is evaluated, its value is passed to
+ * the upper paths w/o being evaluated again. If final aggregation appears to
+ * be necessary above the final join, the contained Aggrefs are supposed to
+ * provide the final aggregation plan with input values, i.e. the aggregate
+ * transient state.
+ *
+ * Note: There's a convention that GroupedVars that contain Aggref expressions
+ * are supposed to follow the other expressions of the target. Iterations of
+ * target->exprs may rely on this arrangement.
+ *
+ * "input" contains Vars used either as grouping expressions or aggregate
+ * arguments, plus grouping expressions which are not plain vars. Paths
+ * providing the partial aggregation plan with input data should use this
+ * target.
+ *
+ * "group_clauses" and "group_exprs" are lists of SortGroupClause and the
+ * corresponding grouping expressions respectively. "agg_exprs" is a list of
+ * Aggref nodes to be evaluated by the relation.
+ *
+ * "rows" is the estimated number of result tuples produced by grouped paths.
+ */
+ typedef struct RelAggInfo
+ {
+ NodeTag type;
+
+ PathTarget *target; /* target of grouped paths. */
+ PathTarget *input; /* pathtarget of paths that generate input for
+ * the partial aggregation. */
+
+ List *group_clauses;
+ List *group_exprs;
+ List *agg_exprs;
+
+ double rows;
+ } RelAggInfo;
/*
* Type "Path" is used as-is for sequential-scan paths, as well as some other
*************** typedef struct HashPath
*** 1486,1497 ****
--- 1560,1575 ----
* ProjectionPath node, which is marked dummy to indicate that we intend to
* assign the work to the input plan node. The estimated cost for the
* ProjectionPath node will account for whether a Result will be used or not.
+ *
+ * force_result field tells that the Result node must be used for some reason
+ * even though the subpath could normally handle the projection.
*/
typedef struct ProjectionPath
{
Path path;
Path *subpath; /* path representing input source */
bool dummypp; /* true if no separate Result is needed */
+ bool force_result; /* Is Result node required? */
} ProjectionPath;
/*
*************** typedef struct PlaceHolderVar
*** 1957,1962 ****
--- 2035,2065 ----
Index phlevelsup; /* > 0 if PHV belongs to outer query */
} PlaceHolderVar;
+
+ /*
+ * Similar to the concept of PlaceHolderVar, we treat aggregates and grouping
+ * columns as special variables if grouping is possible below the top-level
+ * join. Likewise, the variable is evaluated below the query targetlist (in
+ * particular, in the targetlist of AGGSPLIT_INITIAL_SERIAL aggregation node
+ * which has base relation or a join as the input) and bubbles up through the
+ * join tree until it reaches AGGSPLIT_FINAL_DESERIAL aggregation node.
+ *
+ * gvexpr is either Aggref or a generic (non-Var) grouping expression. (If a
+ * simple Var, we don't replace it with GroupedVar.)
+ *
+ * agg_partial also points to the corresponding field of GroupedVarInfo if
+ * gvexpr is Aggref.
+ */
+ typedef struct GroupedVar
+ {
+ Expr xpr;
+ Expr *gvexpr; /* the represented expression */
+ Aggref *agg_partial; /* partial aggregate if gvexpr is aggregate */
+ Index sortgroupref; /* SortGroupClause.tleSortGroupRef if gvexpr
+ * is grouping expression. */
+ Index gvid; /* GroupedVarInfo */
+ } GroupedVar;
+
/*
* "Special join" info.
*
*************** typedef struct PartitionedChildRelInfo
*** 2131,2138 ****
Index parent_relid;
List *child_rels;
! bool part_cols_updated; /* is the partition key of any of
! * the partitioned tables updated? */
} PartitionedChildRelInfo;
/*
--- 2234,2241 ----
Index parent_relid;
List *child_rels;
! bool part_cols_updated; /* is the partition key of any of the
! * partitioned tables updated? */
} PartitionedChildRelInfo;
/*
*************** typedef struct PlaceHolderInfo
*** 2174,2179 ****
--- 2277,2301 ----
} PlaceHolderInfo;
/*
+ * Likewise, GroupedVarInfo exists for each distinct GroupedVar.
+ */
+ typedef struct GroupedVarInfo
+ {
+ NodeTag type;
+
+ Index gvid; /* GroupedVar.gvid */
+ Expr *gvexpr; /* the represented expression. */
+ Aggref *agg_partial; /* if gvexpr is aggregate, agg_partial is the
+ * corresponding partial aggregate */
+ Index sortgroupref; /* If gvexpr is a grouping expression, this is
+ * the tleSortGroupRef of the corresponding
+ * SortGroupClause. */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+ int32 gv_width; /* estimated width of the expression */
+ } GroupedVarInfo;
+
+ /*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
* path, the list is stored into root->minmax_aggs for use during setrefs.c.
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
new file mode 100644
index ba4fa4b..6901f10
*** a/src/include/optimizer/clauses.h
--- b/src/include/optimizer/clauses.h
*************** extern Node *estimate_expression_value(P
*** 84,88 ****
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
!
#endif /* CLAUSES_H */
--- 84,89 ----
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
! extern GroupedVarInfo *translate_expression_to_rels(PlannerInfo *root,
! GroupedVarInfo *gvi, Index relid);
#endif /* CLAUSES_H */
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
new file mode 100644
index 132e355..9d56761
*** a/src/include/optimizer/cost.h
--- b/src/include/optimizer/cost.h
*************** extern PGDLLIMPORT double parallel_tuple
*** 54,60 ****
extern PGDLLIMPORT double parallel_setup_cost;
extern PGDLLIMPORT int effective_cache_size;
extern PGDLLIMPORT Cost disable_cost;
! extern PGDLLIMPORT int max_parallel_workers_per_gather;
extern PGDLLIMPORT bool enable_seqscan;
extern PGDLLIMPORT bool enable_indexscan;
extern PGDLLIMPORT bool enable_indexonlyscan;
--- 54,60 ----
extern PGDLLIMPORT double parallel_setup_cost;
extern PGDLLIMPORT int effective_cache_size;
extern PGDLLIMPORT Cost disable_cost;
! extern PGDLLIMPORT int max_parallel_workers_per_gather;
extern PGDLLIMPORT bool enable_seqscan;
extern PGDLLIMPORT bool enable_indexscan;
extern PGDLLIMPORT bool enable_indexonlyscan;
*************** extern PGDLLIMPORT bool enable_gathermer
*** 70,76 ****
extern PGDLLIMPORT bool enable_partitionwise_join;
extern PGDLLIMPORT bool enable_parallel_append;
extern PGDLLIMPORT bool enable_parallel_hash;
! extern PGDLLIMPORT int constraint_exclusion;
extern double clamp_row_est(double nrows);
extern double index_pages_fetched(double tuples_fetched, BlockNumber pages,
--- 70,77 ----
extern PGDLLIMPORT bool enable_partitionwise_join;
extern PGDLLIMPORT bool enable_parallel_append;
extern PGDLLIMPORT bool enable_parallel_hash;
! extern PGDLLIMPORT bool enable_agg_pushdown;
! extern PGDLLIMPORT int constraint_exclusion;
extern double clamp_row_est(double nrows);
extern double index_pages_fetched(double tuples_fetched, BlockNumber pages,
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
new file mode 100644
index ef7173f..8cafd54
*** a/src/include/optimizer/pathnode.h
--- b/src/include/optimizer/pathnode.h
*************** extern AppendPath *create_append_path(Re
*** 71,76 ****
--- 71,77 ----
List *partitioned_rels, double rows);
extern MergeAppendPath *create_merge_append_path(PlannerInfo *root,
RelOptInfo *rel,
+ PathTarget *target,
List *subpaths,
List *pathkeys,
Relids required_outer,
*************** extern Relids calc_non_nestloop_required
*** 123,128 ****
--- 124,130 ----
extern NestPath *create_nestloop_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
*************** extern NestPath *create_nestloop_path(Pl
*** 134,139 ****
--- 136,142 ----
extern MergePath *create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
*************** extern MergePath *create_mergejoin_path(
*** 148,153 ****
--- 151,157 ----
extern HashPath *create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
*************** extern AggPath *create_agg_path(PlannerI
*** 197,202 ****
--- 201,213 ----
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+ extern AggPath *create_partial_agg_sorted_path(PlannerInfo *root,
+ Path *subpath,
+ bool check_pathkeys,
+ double input_rows);
+ extern AggPath *create_partial_agg_hashed_path(PlannerInfo *root,
+ Path *subpath,
+ double input_rows);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
*************** extern void setup_simple_rel_arrays(Plan
*** 266,278 ****
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
! extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
! List **restrictlist_ptr);
extern Relids min_join_parameterization(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
--- 277,292 ----
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
! extern RelOptInfo *find_grouped_base_rel(PlannerInfo *root, int relid);
! extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids,
! bool grouped);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
! List **restrictlist_ptr,
! bool grouped);
extern Relids min_join_parameterization(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
*************** extern ParamPathInfo *find_param_path_in
*** 300,305 ****
extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
! SpecialJoinInfo *sjinfo, JoinType jointype);
!
#endif /* PATHNODE_H */
--- 314,324 ----
extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
! SpecialJoinInfo *sjinfo, JoinType jointype,
! bool grouped);
! extern RelAggInfo *create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel);
! extern RelAggInfo *translate_rel_agg_info(PlannerInfo *root,
! RelAggInfo *agg_info,
! AppendRelInfo **appinfos,
! int nappinfos);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
new file mode 100644
index 94f9bb2..8352899
*** a/src/include/optimizer/paths.h
--- b/src/include/optimizer/paths.h
***************
*** 21,29 ****
* allpaths.c
*/
extern PGDLLIMPORT bool enable_geqo;
! extern PGDLLIMPORT int geqo_threshold;
! extern PGDLLIMPORT int min_parallel_table_scan_size;
! extern PGDLLIMPORT int min_parallel_index_scan_size;
/* Hook for plugins to get control in set_rel_pathlist() */
typedef void (*set_rel_pathlist_hook_type) (PlannerInfo *root,
--- 21,30 ----
* allpaths.c
*/
extern PGDLLIMPORT bool enable_geqo;
! extern PGDLLIMPORT bool enable_agg_pushdown;
! extern PGDLLIMPORT int geqo_threshold;
! extern PGDLLIMPORT int min_parallel_table_scan_size;
! extern PGDLLIMPORT int min_parallel_index_scan_size;
/* Hook for plugins to get control in set_rel_pathlist() */
typedef void (*set_rel_pathlist_hook_type) (PlannerInfo *root,
*************** typedef void (*set_join_pathlist_hook_ty
*** 41,66 ****
JoinPathExtraData *extra);
extern PGDLLIMPORT set_join_pathlist_hook_type set_join_pathlist_hook;
/* Hook for plugins to replace standard_join_search() */
! typedef RelOptInfo *(*join_search_hook_type) (PlannerInfo *root,
! int levels_needed,
! List *initial_rels);
extern PGDLLIMPORT join_search_hook_type join_search_hook;
! extern RelOptInfo *make_one_rel(PlannerInfo *root, List *joinlist);
extern void set_dummy_rel_pathlist(RelOptInfo *rel);
! extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
! List *initial_rels);
extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages, int max_workers);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
Path *bitmapqual);
extern void generate_partitionwise_join_paths(PlannerInfo *root,
! RelOptInfo *rel);
#ifdef OPTIMIZER_DEBUG
extern void debug_print_rel(PlannerInfo *root, RelOptInfo *rel);
--- 42,89 ----
JoinPathExtraData *extra);
extern PGDLLIMPORT set_join_pathlist_hook_type set_join_pathlist_hook;
+ /*
+ * Result of standard_join_search() or join_search_hook().
+ *
+ * 'plain' is a join of two plain (non-grouped) relation.
+ *
+ * 'grouped' is either a join of one plain relation to one grouped, or a join
+ * of two plain relations whose (the join relation's) paths have all been
+ * subject to partial aggregation.
+ */
+ typedef struct JoinSearchResult
+ {
+ RelOptInfo *plain;
+ RelOptInfo *grouped;
+ } JoinSearchResult;
+
/* Hook for plugins to replace standard_join_search() */
! typedef JoinSearchResult *(*join_search_hook_type) (PlannerInfo *root,
! int levels_needed,
! List *initial_rels,
! List *initial_rels_grouped);
extern PGDLLIMPORT join_search_hook_type join_search_hook;
! extern JoinSearchResult *make_one_rel(PlannerInfo *root, List *joinlist);
extern void set_dummy_rel_pathlist(RelOptInfo *rel);
! extern JoinSearchResult *standard_join_search(PlannerInfo *root,
! int levels_needed,
! List *initial_rels,
! List *initial_rels_grouped);
extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
+
+ extern bool create_grouped_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, bool precheck,
+ bool partial, AggStrategy aggstrategy);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages, int max_workers);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
Path *bitmapqual);
extern void generate_partitionwise_join_paths(PlannerInfo *root,
! RelOptInfo *rel);
#ifdef OPTIMIZER_DEBUG
extern void debug_print_rel(PlannerInfo *root, RelOptInfo *rel);
*************** extern Expr *adjust_rowcompare_for_index
*** 92,98 ****
* tidpath.h
* routines to generate tid paths
*/
! extern void create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel);
/*
* joinpath.c
--- 115,122 ----
* tidpath.h
* routines to generate tid paths
*/
! extern void create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel,
! bool grouped);
/*
* joinpath.c
*************** extern void create_tidscan_paths(Planner
*** 101,114 ****
extern void add_paths_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, SpecialJoinInfo *sjinfo,
! List *restrictlist);
/*
* joinrels.c
* routines to determine which relations to join
*/
extern void join_search_one_level(PlannerInfo *root, int level);
! extern RelOptInfo *make_join_rel(PlannerInfo *root,
RelOptInfo *rel1, RelOptInfo *rel2);
extern bool have_join_order_restriction(PlannerInfo *root,
RelOptInfo *rel1, RelOptInfo *rel2);
--- 125,138 ----
extern void add_paths_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, SpecialJoinInfo *sjinfo,
! List *restrictlist, bool do_aggregate);
/*
* joinrels.c
* routines to determine which relations to join
*/
extern void join_search_one_level(PlannerInfo *root, int level);
! extern JoinSearchResult *make_join_rel(PlannerInfo *root,
RelOptInfo *rel1, RelOptInfo *rel2);
extern bool have_join_order_restriction(PlannerInfo *root,
RelOptInfo *rel1, RelOptInfo *rel2);
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
new file mode 100644
index 7132c88..2bd1135
*** a/src/include/optimizer/planmain.h
--- b/src/include/optimizer/planmain.h
*************** typedef void (*query_pathkeys_callback)
*** 38,44 ****
* prototypes for plan/planmain.c
*/
extern RelOptInfo *query_planner(PlannerInfo *root, List *tlist,
! query_pathkeys_callback qp_callback, void *qp_extra);
/*
* prototypes for plan/planagg.c
--- 38,45 ----
* prototypes for plan/planmain.c
*/
extern RelOptInfo *query_planner(PlannerInfo *root, List *tlist,
! query_pathkeys_callback qp_callback, void *qp_extra,
! RelOptInfo **partially_grouped);
/*
* prototypes for plan/planagg.c
*************** extern void add_base_rels_to_query(Plann
*** 76,81 ****
--- 77,84 ----
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
Relids where_needed, bool create_new_ph);
+ extern void add_grouped_base_rels_to_query(PlannerInfo *root);
+ extern void add_grouped_vars_to_rels(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
new file mode 100644
index 9fa52e1..68c32e1
*** a/src/include/optimizer/tlist.h
--- b/src/include/optimizer/tlist.h
***************
*** 16,22 ****
#include "nodes/relation.h"
-
extern TargetEntry *tlist_member(Expr *node, List *targetlist);
extern TargetEntry *tlist_member_ignore_relabel(Expr *node, List *targetlist);
--- 16,21 ----
*************** extern Node *get_sortgroupclause_expr(So
*** 41,47 ****
List *targetList);
extern List *get_sortgrouplist_exprs(List *sgClauses,
List *targetList);
-
extern SortGroupClause *get_sortgroupref_clause(Index sortref,
List *clauses);
extern SortGroupClause *get_sortgroupref_clause_noerr(Index sortref,
--- 40,45 ----
*************** extern void split_pathtarget_at_srfs(Pla
*** 65,70 ****
--- 63,75 ----
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+ /* TODO Find the best location (position and in some cases even file) for the
+ * following ones. */
+ extern List *replace_grouped_vars_with_aggrefs(PlannerInfo *root, List *src);
+ extern void add_grouped_vars_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions);
+ extern GroupedVar *get_grouping_expression(List *gvis, Expr *expr);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
diff --git a/src/include/optimizer/var.h b/src/include/optimizer/var.h
new file mode 100644
index 43c53b5..5a795c3
*** a/src/include/optimizer/var.h
--- b/src/include/optimizer/var.h
*************** extern bool contain_vars_of_level(Node *
*** 36,40 ****
--- 36,42 ----
extern int locate_var_of_level(Node *node, int levelsup);
extern List *pull_var_clause(Node *node, int flags);
extern Node *flatten_join_alias_vars(PlannerInfo *root, Node *node);
+ extern GroupedVarInfo *find_grouped_var_info(PlannerInfo *root,
+ GroupedVar *gvar);
#endif /* VAR_H */
diff --git a/src/include/utils/selfuncs.h b/src/include/utils/selfuncs.h
new file mode 100644
index 299c9f8..e033a90
*** a/src/include/utils/selfuncs.h
--- b/src/include/utils/selfuncs.h
*************** extern void estimate_hash_bucket_stats(P
*** 210,215 ****
--- 210,219 ----
Node *hashkey, double nbuckets,
Selectivity *mcv_freq,
Selectivity *bucketsize_frac);
+ extern Size estimate_hashagg_tablesize(Path *path,
+ const AggClauseCosts *agg_costs,
+ double dNumGroups);
+
extern List *deconstruct_indexquals(IndexPath *path);
extern void genericcostestimate(PlannerInfo *root, IndexPath *path,
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
new file mode 100644
index ...09b380d
*** a/src/test/regress/expected/agg_pushdown.out
--- b/src/test/regress/expected/agg_pushdown.out
***************
*** 0 ****
--- 1,316 ----
+ BEGIN;
+ CREATE TABLE agg_pushdown_parent (
+ i int primary key);
+ CREATE TABLE agg_pushdown_child1 (
+ j int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+ CREATE INDEX ON agg_pushdown_child1(parent);
+ CREATE TABLE agg_pushdown_child2 (
+ k int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+ INSERT INTO agg_pushdown_parent(i)
+ SELECT n
+ FROM generate_series(0, 7) AS s(n);
+ INSERT INTO agg_pushdown_child1(j, parent, v)
+ SELECT 64 * i + n, i, random()
+ FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+ INSERT INTO agg_pushdown_child2(k, parent, v)
+ SELECT 64 * i + n, i, random()
+ FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+ ANALYZE;
+ SET enable_agg_pushdown TO on;
+ -- Perform scan of a table and partially aggregate the result.
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+ ------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+ (9 rows)
+
+ -- Scan index on agg_pushdown_child1(parent) column and partially aggregate
+ -- the result using AGG_SORTED strategy.
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (8 rows)
+
+ SET enable_seqscan TO on;
+ -- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+ -- and partially aggregate the result.
+ SET enable_nestloop TO on;
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: (k = c1.j)
+ Filter: (c1.parent = parent)
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (14 rows)
+
+ -- The same for hash join.
+ SET enable_nestloop TO off;
+ SET enable_hashjoin TO on;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ----------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Hash
+ -> Seq Scan on agg_pushdown_child2 c2
+ (15 rows)
+
+ -- The same for merge join.
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO on;
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: (c1.j = c2.k)
+ Join Filter: (c1.parent = c2.parent)
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ (14 rows)
+
+ -- Generic grouping expression.
+ EXPLAIN (COSTS off)
+ SELECT p.i / 2, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i / 2;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: (((c1.parent / 2)))
+ -> Sort
+ Sort Key: (((c1.parent / 2)))
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: (c1.parent / 2), c1.parent, c2.parent
+ -> Merge Join
+ Merge Cond: (c1.j = c2.k)
+ Join Filter: (c1.parent = c2.parent)
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ (16 rows)
+
+ -- The same tests for parallel plans.
+ RESET ALL;
+ SET parallel_setup_cost TO 0;
+ SET parallel_tuple_cost TO 0;
+ SET min_parallel_table_scan_size TO 0;
+ SET min_parallel_index_scan_size TO 0;
+ SET max_parallel_workers_per_gather TO 4;
+ SET enable_agg_pushdown TO on;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Gather
+ Workers Planned: 2
+ -> Parallel Hash Join
+ Hash Cond: (c1.parent = p.i)
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Parallel Seq Scan on agg_pushdown_child1 c1
+ -> Parallel Hash
+ -> Parallel Seq Scan on agg_pushdown_parent p
+ (11 rows)
+
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 2
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Parallel Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (10 rows)
+
+ SET enable_seqscan TO on;
+ SET enable_nestloop TO on;
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Parallel Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: (k = c1.j)
+ Filter: (c1.parent = parent)
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+ (16 rows)
+
+ SET enable_nestloop TO off;
+ SET enable_hashjoin TO on;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ----------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Gather
+ Workers Planned: 1
+ -> Parallel Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Parallel Seq Scan on agg_pushdown_parent p
+ -> Parallel Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Parallel Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Parallel Seq Scan on agg_pushdown_child1 c1
+ -> Parallel Hash
+ -> Parallel Seq Scan on agg_pushdown_child2 c2
+ (17 rows)
+
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO on;
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 2
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: (c1.j = c2.k)
+ Join Filter: (c1.parent = c2.parent)
+ -> Parallel Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ (16 rows)
+
+ EXPLAIN (COSTS off)
+ SELECT p.i / 2, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i / 2;
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: (((c1.parent / 2)))
+ -> Sort
+ Sort Key: (((c1.parent / 2)))
+ -> Gather
+ Workers Planned: 2
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: (c1.parent / 2), c1.parent, c2.parent
+ -> Merge Join
+ Merge Cond: (c1.j = c2.k)
+ Join Filter: (c1.parent = c2.parent)
+ -> Parallel Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ (18 rows)
+
+ ROLLBACK;
diff --git a/src/test/regress/expected/sysviews.out b/src/test/regress/expected/sysviews.out
new file mode 100644
index 759f7d9..68d0407
*** a/src/test/regress/expected/sysviews.out
--- b/src/test/regress/expected/sysviews.out
*************** select count(*) >= 0 as ok from pg_prepa
*** 72,77 ****
--- 72,78 ----
select name, setting from pg_settings where name like 'enable%';
name | setting
---------------------------+---------
+ enable_agg_pushdown | off
enable_bitmapscan | on
enable_gathermerge | on
enable_hashagg | on
*************** select name, setting from pg_settings wh
*** 87,93 ****
enable_seqscan | on
enable_sort | on
enable_tidscan | on
! (15 rows)
-- Test that the pg_timezone_names and pg_timezone_abbrevs views are
-- more-or-less working. We can't test their contents in any great detail
--- 88,94 ----
enable_seqscan | on
enable_sort | on
enable_tidscan | on
! (16 rows)
-- Test that the pg_timezone_names and pg_timezone_abbrevs views are
-- more-or-less working. We can't test their contents in any great detail
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
new file mode 100644
index ad9434f..611aeb4
*** a/src/test/regress/parallel_schedule
--- b/src/test/regress/parallel_schedule
*************** test: rules psql_crosstab amutils
*** 98,103 ****
--- 98,106 ----
test: select_parallel
test: write_parallel
+ # this one runs parallel workers too
+ test: agg_pushdown
+
# no relation related tests can be put in this group
test: publication subscription
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
new file mode 100644
index 27cd498..fe8108e
*** a/src/test/regress/serial_schedule
--- b/src/test/regress/serial_schedule
*************** test: rules
*** 137,142 ****
--- 137,143 ----
test: psql_crosstab
test: select_parallel
test: write_parallel
+ test: agg_pushdown
test: publication
test: subscription
test: amutils
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
new file mode 100644
index ...05e2f55
*** a/src/test/regress/sql/agg_pushdown.sql
--- b/src/test/regress/sql/agg_pushdown.sql
***************
*** 0 ****
--- 1,137 ----
+ BEGIN;
+
+ CREATE TABLE agg_pushdown_parent (
+ i int primary key);
+
+ CREATE TABLE agg_pushdown_child1 (
+ j int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+
+ CREATE INDEX ON agg_pushdown_child1(parent);
+
+ CREATE TABLE agg_pushdown_child2 (
+ k int primary key,
+ parent int references agg_pushdown_parent,
+ v double precision);
+
+ INSERT INTO agg_pushdown_parent(i)
+ SELECT n
+ FROM generate_series(0, 7) AS s(n);
+
+ INSERT INTO agg_pushdown_child1(j, parent, v)
+ SELECT 64 * i + n, i, random()
+ FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+
+ INSERT INTO agg_pushdown_child2(k, parent, v)
+ SELECT 64 * i + n, i, random()
+ FROM generate_series(0, 63) AS s(n), agg_pushdown_parent;
+
+ ANALYZE;
+
+ SET enable_agg_pushdown TO on;
+
+ -- Perform scan of a table and partially aggregate the result.
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+ -- Scan index on agg_pushdown_child1(parent) column and partially aggregate
+ -- the result using AGG_SORTED strategy.
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+ SET enable_seqscan TO on;
+
+ -- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+ -- and partially aggregate the result.
+ SET enable_nestloop TO on;
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO off;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ -- The same for hash join.
+ SET enable_nestloop TO off;
+ SET enable_hashjoin TO on;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ -- The same for merge join.
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO on;
+ SET enable_seqscan TO off;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ -- Generic grouping expression.
+ EXPLAIN (COSTS off)
+ SELECT p.i / 2, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i / 2;
+
+ -- The same tests for parallel plans.
+ RESET ALL;
+
+ SET parallel_setup_cost TO 0;
+ SET parallel_tuple_cost TO 0;
+ SET min_parallel_table_scan_size TO 0;
+ SET min_parallel_index_scan_size TO 0;
+ SET max_parallel_workers_per_gather TO 4;
+
+ SET enable_agg_pushdown TO on;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+ SET enable_seqscan TO off;
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+ AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+ SET enable_seqscan TO on;
+
+ SET enable_nestloop TO on;
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO off;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ SET enable_nestloop TO off;
+ SET enable_hashjoin TO on;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ SET enable_hashjoin TO off;
+ SET enable_mergejoin TO on;
+ SET enable_seqscan TO off;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+ EXPLAIN (COSTS off)
+ SELECT p.i / 2, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+ agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i / 2;
+
+ ROLLBACK;
Re: [HACKERS] WIP: Aggregation push-down
Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Feb 23, 2018 at 11:08 AM, Antonin Houska <ah@cybertec.at> wrote:
I spent some more time thinking about this. What about adding a new strategy
number for hash index operator classes, e.g. HTBinaryEqualStrategyNumber? For
most types both HTEqualStrategyNumber and HTBinaryEqualStrategyNumber strategy
would point to the same operator, but types like numeric would naturally have
them different.Thus the pushed-down partial aggregation can only use the
HTBinaryEqualStrategyNumber's operator to compare grouping expressions. In the
initial version (until we have useful statistics for the binary values) we can
avoid the aggregation push-down if the grouping expression output type has the
two strategies implemented using different functions because, as you noted
upthread, grouping based on binary equality can result in excessive number of
groups.One open question is whether the binary equality operator needs a separate
operator class or not. If an opclass cares only about the binary equality, its
hash function(s) can be a lot simpler.Hmm. How about instead adding another regproc field to pg_type which
stores the OID of a function that tests binary equality for that
datatype? If that happens to be equal to the OID you got from the
opclass, then you're all set.
I suppose you mean pg_operator, not pg_type. What I don't like about this is
that the new field would only be useful for very little fraction of
operators.
On the other hand, the drawback of an additional operator classes is that we'd
have to modify almost all the existing operator classes for the hash AM. (The
absence of the new strategy number in an operator class cannot mean that the
existing equality operator can be used to compare binary values too, because
thus we can't guarantee correct behavior of the already existing user-defined
operator classes.)
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com
Re: [HACKERS] WIP: Aggregation push-down
Antonin Houska <ah@cybertec.at> wrote:
I've reworked the patch so that separate RelOptInfo is used for grouped
relation. The attached patch is only the core part. Neither postgres_fdw
changes nor the part that tries to avoid the final aggregation is included
here. I'll add these when the patch does not seem to need another major rework.
This is the next version. After having posted a few notes to the [1]/messages/by-id/c165b72e-8dbb-2a24-291f-113aeb67b76a@iki.fi thread,
I'm returning to the original one so it can be referenced from my entry in the
CF application. As proposed in the other thread, it uses only "simple
aggregation". The 2-stage aggregation, which gives more power to the feature
and which also enables paralle processing for it, will be coded in a separate
patch.
I eventually decided abandon the Var substitution proposed by Heikki in
[2]: /messages/by-id/113e9594-7c08-0f1f-ad15-41773b56a86b@iki.fi
only simple column reference as grouping expression so far.
[1]: /messages/by-id/c165b72e-8dbb-2a24-291f-113aeb67b76a@iki.fi
[2]: /messages/by-id/113e9594-7c08-0f1f-ad15-41773b56a86b@iki.fi
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com
Attachments:
agg_pushdown_v8.tgzapplication/x-gzipDownload
� ,T�[ �<ks����*�
L���6%��HJJN����z��������3�%6��aG�����(QO+����9+"��o,� ��_~�Y�� ���p�O�|�'7q�������k�� ���a��'f�m[�cZ ��a��� �r��o�����c�|��X�4����\�c��cv�]�c�@+�����m��m���������� �m�a?�+�@�8~� ����L �.���v��<a�_A��S�? �rC��r�_�,���5H3��L�M=��w�"�~����C�M�5:���!�y�o:]�� ���cY ��t�vG�
h5;����_��w�����Y��{���&�+��iZ��$����=�L�?��^2 �8f`� �g�������zm3���x�A��=�;Ge1J���(.�8��OI��� |��7����������co�� �C�6��~nu��6� ��4�2�� a�_�$+��A�@����#���"�����=}g����ZC�� ��w0��'�I
� *;H# ��Xd��[����/���; �0yW���/:��63�uwZ�����h����?��Z��� ���e���=�'%L�
E���1,l? ��,)�|��Dp ����h��>U/�Q?Kn�i� �a?E��y�����6��y��Ap�����i)J.��M{� c�8ds������P
��"��*����cO�[UD���� M���BD�i!g�e)�N���s��8-�~&�Y3~ES���� i����
����l�
�� v���b�� ���
�#�$z�l0)D�80:�� :gb�������0��L|.�L������������}��c���:" Z��W��l�'l Xr-�����0"�,Nn$ijR��j�� !\%����/��q�� K��aJ����<aA��q�E��q��)�2d�&����p�I��x�
F` ����#_��@�� �*J �M?+����V}���\N��4K����x0��;����n\����q�F���� Y���}v\1L
�� -�ay�=���eJ�[��:Mc {�� �ztvz�nIn �_���C���4$s
�%��0���+ @{S����
��)���`��D����yX����<*E��,���a[��^���Em����#����5x�E ���>�Ym,�;��|:�|�.!��%E��� 0�%"�W�� ���{�P$[��7�i��D���[[��� ���^� >5�0D��E���� ��K} p�����p86.��(�# �q,8��aPF R��%����Za���x:Gu�EH'X\�Le�����q.,���z�x V�il�� �0�A�bq�����rS� &�� Me��
��Z�B_�
�E��/re��K��*������ut}��]��
w���������F��� � a�E%(�� �( �`�Q��{}���GO���T,��/�+a���`���= r��}<@�7 p�bK�{� ���{�� ��'!� �dS�]�(|��vm��J�JX��f����@�}�$&�J{���9`��_x��]�kX �(�4k���f����i��j��z_�Y�G����i��5��$�l�v_6Y�d��?�}�M����l��M6Y��&+���&/�u�M����+�nm��l����d�^l�v�6Y�
��5��na�5����!�dw��W&nj�F��k��nG���5<�nn�����v-���] ����v?���RX��$��������&�����k�������~ va-�B ;
p�C)P�~M(>�jwR�/����2K� B��^��M�\�(�=������E
X���� �2� X��Y�������z�� S�M����!�����n;N�tlw��_�������z��{f��6����;������������w�����x����� +� �k
�6�����1�A����a[� �w<�i�������[�I0���C��e���5#
��{�\
���C��N�t�;� p�. �D���4�6��?��a�KS�_�:
�$�p�����'�e��&��)�\��)W9���>�9D�������]B�^����T�`0�s���x��9�{<�pF2M��7��e\/� �����&�^�� �� 0���Ai��e��a^9S����ko[�YE����a>1�<�c2j���?r����Z�����w�g{�!�C���bH��e�c��m�d� �[V�y�a��OP��[:�mR�����-��,��CP �)8�x�����,�x!0�D�c�T4��R�-�V��������? _ 3�H�u�vU���L!a��R O)������I�a5�]��U;�G !���D��\!Rq6�.�8�����{1���(<]�#v��,|xf�?/��������j�"����X�6���1"�U_����fA�>�pZ�~N[��.����0 ���z�f�{ ��@�@���`_���y�� ��@@�h6d~�s��%�\�,�,��-\�� rv#�o`� . �%n>�R�X����_��@��}��8�-asLQf�T �� 9 U6.����UjQ�@U���z ,�(3�0!��$Y������g������G''SY
��G
F����Aa���U����� 0�.!���}�D��������Qj.��� ���Uj���
Q��E%^���I�#��G���pTT2����T�+4���D��4Vm�tm���;:�p -�jqk����^��~(�}y\�m��~`�)�O{K&�J���7(��E���h�A�n1(�� ���hw6(�=�� ���e&6[�+ �&
;J�h2��(���4z?�c(�L�_y��D<,F�����|�L�
�}���aZ����j�n�\3�*��e7��,��� ��1� ���L#�\�;���&S�����?�`~"r<WBYD!���`�����l�n�gh+��Y ��N��X��V���o��u�h+n���VT����N\����b y����>��
���a+"<5��54XU~0�5������R�
I:����������� "�kw��M� �����< �� ox����������v�z��V�r���}���=>�<���w���������E���c��! ��u;�^�1Dw������t<�� �kw}��c�i������s�x��K�������0l��C�$�M�, M��F:���� �pE�u�G���!�-��@��!�.tY_�����&��e$C�#���� c ���B ���]dI]������C���XV��enJ"����R�qSo�M0h8��"2���0 �8�� �H���<����)���H��� L�0.e^�
�E����g0 @��&�>�yV�`�D��&)�����B?���*ln����8�H��Z���2�,�hK�(H/�0nHa�Zl��8�t0 kC<���A�Or�����%(u,0p��p��u�m:]�i�{���.c[W%� ���t)���`x�K �. +���'{�+T� J��z�F��j��M<
��h&?�VL����;Z
}�*�'2���F���F���)x��.'�+���{Q����&; ��u���TG9�%\`7���N�����(�w�t� 8T��q��9�����v[���������WW���C@8 �P�W%bT��D�:Ts$pUfZMa�FUdM�4��t�2k<���x���4�Ge�}��EU(=�'J������
�j�Y�LM����
�j�Sh�����@)!�{w�}��u]t�A��^�Y_���r��:H����A�V���E/������_��d<H"zV�>����B �{�R*��X������/O�y< �H�]1}G��W�
���"�(@��frg���%�RF��4�.6���3��B$I����8��]}1 B|�R��cL1Q>J���b�kxo �~�+)P��@A};�7� l���S����{�������(t��,/Jm�oY���q�nr%Lu@
��2m���n�6��gX���U���P ]m��"�WGWGg�'�5�����f�cT���>�%����G=��)+X������ �/�*y��?�|�}��������G_������>4���(����{t�35d<��Q� 0�O �vxmn�%�QMPMc%*e�iv���J'��-����rK���CX]k[kW�" Gp�h;����{^�+<��x�U��a�[1��(|tj���FN�~^d�W�\�Y�(�4A��9'�`ps�`b�R���2���4+��@�\�� 1|?5�5+�-,�����G ��>�.�o��������IR�65�}��1]�����%����C<���dw;z�hw�n������r;��S����O�}���=>�<�����l��9 ���$��c��A���y��K���K�K����x]�2� o����vz��`C��2����2 �e[���^��<~��A���4���_�_�}����i��G��
J!Ho�1�c�����-Y��[�eJ?y�p� �~�B]��~7������5�78����� �� r�sHeN�U �
�H�s�IS 6S�[������
]�I����0L������-y�m��_�i��L� O�J"_�9(.Yx;e�*��ku�X�B���/\u��h���j�EX9 e���`�%���x�lw D�
�:��������7��g�w��XrAI)j�|utvt������3�^��5���X �^ _
��4���W���)�c�$��!�8�i ������.�YI��j��.��- ���/��sYj-
XF��� G(���1��D9D�n~��S=�<��R��2��Q�X������c�\r�>;.�lR�4�����;<���Y� �;���
?���f������xCf�����>@��5���2�� �Mu�[^%D|�_ /N�@L����*y �_�)��l����[�S��e�")@*� ��O������2?�B�Os����:���^}~�b�uS��k���6.OSXY�
�~�iP������|����G��� J��_�l�H���fU�7[�04P F�{[�{W���3��f
���v ���)��0f�:����0Q�t�������������!����70�@�o�_\���������� N�U� m������E^%���5��w �V ��R�������j� ��Uw,�iY3��u�U@k0?M��*A]�1*G�Ao�Ka*� c��/��0�.��S�x�������YN?'#��AG%F\2�rx�3N�)�������������gG''����?Uvsi��n��R�����-�Z��s
;mh��Y���WJ��L�m\��t��� �=b ��ay�L����
p�kS�I�.m���\�r����T�x�������[s��dr�+[�<3t~~�������&�
����f����m[����w��$3�����6�~m�u�#���z<�Q%�����zTu� ����J!4�U:R5O��3�O�F5���A�a`u 3��2��k����r9�� �c���X%"�/�*���B2n_�9V��4�fk(���&���"y�`���y8��:��t��A`�����.�_��������7`)� ?�Q^ ��"P10�%�x � �q�b����w*�"���(�N���W�n����8�~��_�S��� ��Vd-Q2o�������PC`@N�-@o������O�� %g{�T�f������;��� ek+��E����6�����pO��������
�O$� Z1+
����6���17��!h����gH{����Pap�Q6"�� D��)�n�~ V��(�fyv��+.��T����c!�����B��C�����.�T��w�M,eB��;
i�dOI�����N����� &=6����/q2���b���l<����nl���Z��n�9&+�{�z����c�s��`e�1U-�%[9��bE'�b������>�#"�!���vVf����n�4}B\�����H? ; W��r�!T�n H:�����Jb����j�VD\.���)]�a<�@���#�R��HW���T���yd
s���B#� ���NN�f5_�?�^�uJ�_tH�^fw��"�!��7�{��vEW�k� ����Wh�%&�:�d��BV��e�������3ST�h���������`V�9>�����n�3����"Fzi�6#E� �jAy�MD��)����K�� ���
a3ch�Y�RU�e Z:���d�w�ur
fc������ ;�Yd���%o|��� kfe�O��p %���@�X/��S�W��� 0�����|���6���������,����mR���"��]��Lvx�`,)8 bY�C� AM@�w>Y2���qa�[�{�� 1r�`�� &��=��Y�6�@�} e~5��G��z\ Nq�a ��N�H ����g����w�/z� ����F����q�K��8��Z� s��9���Y0�?�P)�j��(�w�0���S������ G�%Zgi���AM�AN\ns���j��&�������W�)�M� ��!��
���cY��
1���i�l|�}v��UXGN �9�= �v8�c:�p)�����il�����BFc%���EP�_�N����;�X!L��,�g��r
� �K� ��p=�Om\]����m�Q���A����W� ��8y! "�Ta?�L��f}D @��S��0��i���M\���@���p3�/[
��9&4�%�������3,��9Z�#j��uX���l�'�|xk�T7���
��.)py"�QS�|B��~�����g0�!M �0�9`S�F
������j4��]:|o����wD��=)�;���'z��J.�m��A�<� ���M���������VG�V�?��Z���f@B� ��wf�E�L
�q��"�K�.����h��-\�n���i�u�����V�� Yu���������O^�iA�����QM�G������L�'X
��cb�,���@�[�l�KO���A<s�z��j%,���%qo���V,���} aH�n����#� Qzv�_��tw�(%y�,��k
/&2�����UF`a8#R9�p0k���d'���b����v� � ��.N���H��){������� ���O�n�� �B��� ������v�� 6�]��;�7�^� �C����k��X�) ��[�!y��yz�� ���s`o���-��k ��b5�`����QJ����nqCA����K~ XF����t�G�K���n{04Q�����ySQ��N����|wvr��m�����'��iQ�(��h^ �l"�X�����+�g��=0�[US��� � ��}S���� ��� ����g���� ��#����������,���>u�84�^FBk�FZ 0�J�b;� �)y� �`� ���5�&��x �R���U� &��Y$����. �|�����k�L��K�,��zbJ������Y=A�C�(b��DM�k��4�������.������l$���|���>�����xr��o��i5�{���a�����a�e���-y[bWv�-'�0zA2.�m@%����q� S�� A7���)_�����y���o�� ������F�3��"��������%0#��a�ug� �����������1b�n����G�F��O��� w��-�����8[��5�L����Kb�Y����� vYT���F��u�G�C��,2���/�d ��&����RakA
����0i�� ?�F����5FJ�
�O<�
�|�h����<2��k�� g
����x��"�Ngc�:�LX�^����0����scf�3�� 2����d���o �I��@�b3��]��.z�jL�O�a�8���d������;g8.j������H�=�!6�� �*����~c�� �������g�K 8��o�{��o_�>�0�(��=�� ��_9������#��� u
���� �ja����A����`��O{��� Y�q�YRu_)�U�H���M�O%GArs���� M�L5�����<�v��S�bwF���Zw���f��9����E`��d D\RC��C�k��<S�e���0�+��f�y���d�,z ��n}���q��y�Ay�9�^V]�S}5���wE��$ ��� ��3�
+7Zu���������W�l��N0�P0�1�����J�kK"���Nq����P �f����:�f���^��|qK��c^ Lr ������R��(�6 ���J���E��Q.����FgWS�=���4������"�q���<>�=�S��_��< @ p6���Q��p�����QS��4�3�dZNs���
��D��rH���e�������
������.JB�����B )��Ij�es�gBw^J�� q���
�vP-
?77*�}j�H>jM���n�!%���/HS"���,S�s�����"�P@5�]f�OG��D07��j�)o�� �������#nE'`&Y�0
!\�{�*�VvM����
I� � �^�V`* ��o���?)��/r*&` R�_,)�����+�������P'w ��uD��.O���p����Ab�2�)�@J�j�L�.��s�h]� %<���������+=t�,d"�r1'�I #��o�P�Hz{���%/�d�NAQ��P-)���9����8u< t�g�p��\u�U���Tb�U���A�S���Zm�O��/x�������P�����.<;��
5��fG�r�:����T�����=�V])3�:�V��V`�q[ ��� O<���7zZ/�.J*�9s\��`����E����X^N��C����9�Y 2%R����������|�0�XZ���v���; ����RS�`)������E Ru2�h(lz�04�� �u�= �X�I-A]p��Ca���O2B�q��z�mV����:1���� $�I@V�l��� ��BT$O������,SP+5��T�7�
|`g\��D&ag��g���
��Tr���=�E�eK��ag����;B�0�� F��'�T$X�Jn��b�8�?&=�����9d3�@9r�c��L��H���"�:f��1��� �;�2�X� ��(�� tC$��{� ���ktx�o�u�#S
��Z Cy�b6��%� �i��6�7|���h�l�f�t=�e���1[$x�u��`�m���}����>:��(a~^
��<f� 1�H���_AR�x�^���"".�Uf��N[��C l�~
�+zP��L4�?��
�P�Q��#H1r��8�D�4� �N�ml�p�� K���3#`R �$��1���O����c�� 4��
Zy�7���v|# ��$ +����` k�fG;�e���MZ����v����uJ�L��� 6X�r��C8� s� �. �6�F�i
2/��#����3����d H��4��� �7,������|B����� ���%�q��x /��TD4,i���y��0���l�������&yz��{���l��3_�e�3 �C���<���Y�Q���Nj&����|�0�?&p��,�T��������>^<���p���5����J��<��Ai��R�{>,��t6}^� S\�u>�!�E�@��� �4��� �� ^3w�U�����Y#y����dv��
6<B�@����L l4z+ s,��kB���Y&�Ap�����e-���4S�f$�+ �,L~��o��n�:���=lSz,�nC���Ji,d��G(�g���O*���Ub���� Tg��N*��}�;�T�0t��
�IQ��L0U�A� �I��� G`1.����5T�����r ���p=����� W���^��������������h�����^��{�N>E���RsN�f��<F8�X{��
���������X ���#8I��:G��
������a �����|�,�����A���]`R������������@ �Q%��!���������re�:G�9�� �d�2�lr��UqA�����h~�` t��M�j�+��1G�h���~Nz2T�LF��L�~��a�5�T�n��� �>l�O��w����XI :g�`w ������$<������%`t���xy�= ���%���#�����*/�V� 3[���~� Q������_q�jys���J�o:s�VdV=e���"~� )�x�����=��^� �7��a���8}s�N�z/N����� �\BE�����������7��=��� �����r����:G�f�`/�O�����@:���@:�] �C�9$ X(*:�<T����������c�n*_�3�Hg��v����<h AIa��? ����RY���"��yH �4_��#.��AD��!��#@ ^�'q�� ��*?AL�����]d?G��IsR:��E���5���8h�} n��q�1���R���G �kz b�#k�d�u/+��Ks�nmM�T$ ~sJ�9�������7����H�� &����d9����S&�w�a4t��7��( �`4=`Q����iH�9��� �"Tr1 �ML�p�r��{���� �9��\�bR���.}�� ���w[��v���n�K�+Tu��.��o�3��(��;��{<���m!��3����V:�#��X��,���� YC|,����� +\��Xo�V��i�V��f(�WnK��r���}�R xF+� ����y��o����y��K�yxpP�L,���O"�� �s���<��MU����~�j�7k�Klv��/������� ��w��Vi���T��?�g��kg�u����[��n�)���n{������\��Qw�y|0y�l���Q< ����#mq�:B��p�� �k r�[����) �U�i��Wcr��4 ' zo����w�o{�o���y?��]�1��� ���� � G �~���S�Z��� Z��2������)x zA��� ��\���`��0H,���E�{3� �78���&�E�����i
��?(��|u��>?���z���5����� x��1��,���Y++*�04� ��l� K�����x��� � !�k_������d�X�� k���������2�,��^��yb�1���!|��8��PfA���C�O����e�jA�}��]5nV�<?� ����]���p��w�o_)Y<�`4��oi����Y��,�{�����u!��&���F���qU(L���c����)���$Q ���
#�� �-� ��E6�A(��iq\��� �9�gr ��'2��F���� ��� ����n�s;U l,����Y6�A��VPD ��x�Mo;���#�� =����>�P�W�P��|�:���L��2�������o�l#����%r�a=j�R�|j �|�����D
��##x!��aN01i�I �����G��-ngPO�z�+�J3���Nu��@*JQ�N�Z��h�A�X!�I[K�R��V��wd.&�;��
�0���%����#�����3��h��~�����qq;�p�p�����~��iJ[��z�=��="fO75�A��G����(��M����\v��~+k6;P�<���:�������1A\k�� ��X�Sb=1���f�bP�U ��_�-��~����w� ���==����'��M$xi� 3�����ib�a 7�%��� ."�2�W�h�]����TL�>������ �,z��/s� \��N��# -y�]4�b�N
��G���A!x
� 2�1OPA��Z�w6+
�G
���8�PW s#�`5R���
��!� r-5(���c��j�S���!���eA8�u����<qeB�I���������H���&y�0H�@!� (�!F��L|��e�k���F�r��U� �j���*�*�@�M2�����f��_~10��<������=;T���C5�m���\���!���'��� �������\��P0�n~!!��TY����(2+� ����������%?�T$� F���F� ���lF&~L���m��^mQ3�4��q�V�9<J g}3G��zp���W��M���� �&��;6+�&�(O�'&��-T�A�����bd}3�>�yl�f�c��!I�F�j�h>5�y� �� � 0.���s��P�3^���9������i/��PW6�p�4��Y�u��vE���=��@�� r�CS�"3�Hi#�9������H_tad8\� �� ���yu"Ct���F `�'� [k��T:�$��� ����V���B�b��db��i�Dc ����A&| %6�gY����� _�\Ox �����i�A��t.
��a���l��������8��(4Tu�h�8}_�MJf��)Q�n��K~F8
L3 �I8.c�cc�4�w�� �"k�F���a�E_�~@�Tbx�s�X �6����V=^�0V]l
~�}[[���' �^������A,�E�6�B��`��c� ]�XV�A��&/ �.q��2s� W��t�`��l :���4�
s�s���i �� �����
���ZP�4��=r/��-a<�G����K�S9d9���+9<���E�V� ua���z������FR��S���`
�Gr��H8KY!�a�(�L<��0:�d2fy�w}�}���w��zT�9�`[Q�A��%9hu�-�������rK�*����@�Y%5������ � \�0'�P� %�3q���o��tbK�/P|����r=8��]�$4R7����X�U�H+M� ����������`����vfT�����j�jo����%�����UQ,F�����0�?/�e�UM�\(F����"��v���r������E�l�nE:zJu
pc�?�� ����bx
����zz� +z�zg����#?eK*���3���.������*D�<�s���{ �I~@��D87s�z�d�G��8t�����+�����+Q��I�/��La���GF��r�0w�u$2����������������>H2�[KI�� *y �����6�2;�/��b�%��,�1��>���=px~��Y|N�Wi&�������y?����
�������LI���p���.�W(�� �*�����WR0�{$�w���E31��MNw�#k
7
��nQj�2 #� �I$���Jd�*�(�{����Z��Z"��W����;7���
}R���n���R��.j�3&hM��G�F��V�5?X?d��|���J�F�%������H���SG�m���>�N��bz[�(sc�fB�k(�
9�{�A� �& ��! ��������OR����&�o$u�c�'���F��� �j��o ���s�� ��#X�Z1%�r��`�9 �!5d��X�� � �<S� �� �R�(�]2����9�v��W��H��� &A]�u�h��d) 9M���g���K����Z��v@U-��|��w�t��f��61�L6���O���(� ���&�
��t����Pc�� �p) @�A��� 6f�U3!�.���~3�� S�
��r*F ���&5A [{�3�0��q?DX1K0Xf �8�
����������t:sH�GXy�n�Z�
�_iC�����>ZyIz���a���XA�Zg��-�vb����4J��8��Wr K�,�,xt��S &e�PEClt�^���n�1�R���{�c�}��O2��V�����p9��6���${��@ ��N#�'�}��o�%X`��c-����p���M�����g �T�$�����" ��9q���m�����@�EF��
����/�N�G_���A�AnZU�� /���6E�����0�]^��>z<"���O���!h��o�l]����d��Q@���n�Xm2�
@�!�j�3�nu��[FF�����t� �����L����Z:� �����j ��'K ��c�B�����&W�6|Sn� d>����B9o��z��T��T0w9�\��O�/�-��E�_��'2^�\{d$��K������^�s����
b1V?�`� �rx?K�)����Rn��,��h�����
���$^�Q�& ��-��!�B�aC=��� �H��y�e�e
Y��r+�����Y�^�[aUN��T~����@�\G2�����\�H��������0U�=~�Y��*��i�[[:K��� �`x!����| o���)�=W��VQ�O� E=����Q�����]v����bqG����-6y��k�� � !���y6�a�V9z{����� ���W�t��o�����M[y���~������H-�#�bD��F�1U�4�r�M�6`U ��I��E6)D����P)E�X�=�6��C�DPn�|
:�R�����/�f�n�������;%8�6��.�\���4h$���]�a��� P�����>���`A� ����X 6��w� Id�m���$#2���������Z�$��h�` ���K �������)�{����� �����n���]v������nJ��Z7�z��w�`�� �H\t��!�Q�f:a��u����#����A�����`+N L��t<F\IXl���)2��W��6�����)/� G�}��l� ,�=�
��y�*]Zn^���X���]
�>� O#4p���Y���"�cK-O���=�/�x��8}
pGo1]�#�/�^r���� Q�s�*%���l�n�W8�-��$�j"lwZt�1Z���N�����bE+��<94�t/���_u6��������F �XX�=-�n�����F�^����v���%Q��D�M[� �f�O�s������ p���r���K���
=Y�P��71�g����q�b���~D���~X7}��������B8�Zi�1
#z��)ZN�1���zw���Oh�Y�t�,:��w� �O^���=��[��0��I�>� e@����/��\���3��� f��������:��-��!���s%���e$��I\�o��Z���;� S� ���Y�3�g����F<�}���'g5�wQ6��;y��Vb�K��
�X����#�����% W������~5�h���
���F�.���a��4�b���F��S*~�[�Fb�D�;7j�5{��H[��&Z��� e,��Q�_�g����GP��>C{����:f<$#&��Zq��������?�&�1�c�*�^�Q�Jrj:�.�1��]�;v���n�Z��� ����#�� ��@6���T({gdR~(R�+���o�����%+6m���w�5�]��������Cw�dE?����l��-c�!��P{&��Xg��>�2��Iu/�@�J�E� �$6m�n8����&��^���D���3��T�#
V �
�N<��"���g��\�� �����KB1������@C�� �e�! ���A��6<��\��w��l^ � ��koN1Ipmh6��
��%@�!��?��*F�� �R��~W ��P��{� �,z���w�
�!�tl!��/q�< � ~tK�����{�)I��^%��^�r+�:����~V�"���-� ��K~V.y0��y��!���jz��Tc%�+f�Q ���+e�� �67�h�s�r����m�`�E�R$�F� e�p^T������$ �UFcJL�f�3� �s Rqx���
���B%��t+~n��[`y!� ���94C�b7��~Z� � �n=���?������T��z%v���N|uIh��|� ���5��Ob���r�����4^����R�����U�) s�{��P� ��La�pt6���[G�fSk�U]
�B����(C�Q@X��\ �_TB���N�����t��6�@��M� ]�1�[0Q����$H!����<��<��)e���;w�DnHz���f�7�z����H��4A���� ���D��3�qp�0"5�����w��8X6�J��� �f��i&T�_�8 ��73o�R���% \��CH����8hXg:�����d|n�������T�I�� Y���8���e���o3����`.��'j��S�61�����;��m� �k|w������7� ��F!����� BM���� �AY������J��R���4�J��RgY��Th-a�V�������d �<cM�?�����u�(���E�vW���j1�!V���:�Jb��X�L��O���:�����S�/6�~7����<����K��N�f(��;�eU�/�^z����� ����n$�Qq��
�ci\���{��+�o�������������ug�{�X����w��]-|k���K�J���-|��d��� at>�M �����|4
M��D�S6]pZ� ��T�������� �Y�A�^� ���,���t� �c�$;�f��u�^@�!���T:��2��Ln��!��e���"��9�_�bA��unPG)��`�|�f��U 9�"���o���@�����>7���>" �GZ ���g��������:�WyU-��
�*��
��Xu �_\�2<>�p�g�\��|Am�����!��\�6XO0���=�R�V)]����:iP���T��<�?��F���H���Jk� ���W]��Zr�"���.e_hh����K�����X��W��_�w vv)�[
�j�c���zG�������;�;{ \�a:� ^��H��
=$���o�i�t��� 7"_Z�L��m 8P��k�5�B�}�����'����E,i��/��"�_���~t���������
�CV��7j\E�E����N�z�"���-(��s��v .V �>���w �����Gw])�
���(����g
��"����_g��~����w�hl��wv2Y��,F6P:��c�4�-e��]C�����0��/�W �������xB��+_|I���/�r�����=���;� '�ER]�HW%�l��x�7�FY�Z�oS����� o]��
~V\�����J<���~gU��}���%qi��pi�n���_�����3����E����m�.I���I'��I����� f��A�9h��]^�;��wJ���6J��w�6��!
��}D�����������b���V" f���a(A�54��/�P�e ���mQz ��Q��R�����X��K5q�t_���������!����d9��h� �����d��������wjl��#(cS����<n1�<��$�I=�hB)c��%g �
1��HN�8{������v)i������[�����+K?�k A n��&���fF��Q���E�����`9 �b��5��cu����B(��b+'����P�,E �~CiE�s$J^A0���-�� ��[#���]: �H�b?oDw�wdg��r$�b�Xy���f�
���C���H5w81J���9[�pm�I U(����*�_fU�������O��LD��;���}���!#��N 0���i[��Q�P��RH��:�������,(d��^d���2�d=Ld����@��� k�56���6�����,L� ����G |O��Fa�="=����R� 0�y� �� �$�(�7�!��6������qMV���-4�������%��"�7~1 B�X���C���k�
18��W�������� �7�X+
��zY ��ca�� ��
(�?OE����Nl3�� D�d��z;���6 �
������(R ��"��y��v���� d/�$ ��=� R� ��GBCPn�2/� ����!d�D�c �����f��HP|S%�2P��@��� �\RW-�5{�'�� P
�=�iA� �[�:%���5��y�4\'�l*��� �fs�Q� }�]��@Q��f����,aZ#vHm1�3��s��1"@��[��LV�*?��y5��F��Ha��s�1��f�Z{�2f���
<���
� �= �oL�� ����
a�������T��S�
��:�6�0"���@,s�(�y�� ��|L44�!��O�9m�D����[���|��kr]���Z��KS�wp�� ��c���6fiN���C�1�}9c�p�����R��h��{\���C�o���L Nn<[H>��>�Xj3A�Z��t�-�#i:���a1h�xs�q� O��_O�xz~q���#{�����h*�� ��9?���������$�]� ������9�����}��`�.� H9�����#��Eq�Z�}�,�R, �7S����B���e>������ �8!M\��/���������������,3 Od�F%�.�o�� �g���?}vU���`A��w4�z�����1�zT�}��/�r��G�+��B�|� `]�8��5��Vt��*�J�� ���PBQ�)�1tG�:.6S6��i"�J��.�ax w�� �sj~�F �JG2�m�������]���`�R����Q����^�Y>���K����� ����?�!o����znk���U@`�TI:����8:�ro�INU���#����:�0��
8��n�-K@ ��Rb7^= �-hb���'�EIt��D_W�2�.R/d p�^[L��Y��XB 2��2!�N�%^=��d _~�� ���!xB���o���P��_�^���_��,��������a�I�gb�z]�.�������k�x�R�����~�� �� J �n����@ xUf�����|�W��(��]��eE�eU#��(�T�F����.z���y��,�JGf�dA��h �E�������F�����F{��Y���� t���I�,'����^UA(���/�|�DF��D��9�T����Iq��
�b[m��1�IS�}�����dW �#
�gX.�m�
���z�
]\G`{���k�3}&�|F��HXd�`c��z����H�TY�5��k�l�2���U���)��V���OCq��9^�OJ��6J���o�R `�t�]$ �y�d-�}�m��^�T��H�,3;�}���P�g�z�-Be��~<�I��P�����u��~�����7�&X ���s>p��@l������>�Z 3A
Na
*�?nRU��R*���N�1��.mmH�1#tG=Ln�M�#[��z/<u_uJ_u��:�e[��^o��Dc�����? "�}��������:�1�K�?b�:L��V���$���xb�@���1�) /�h������� �������:
J�i�
�O ��
��m8R(}��C�>H���=���� � 8^��UG#�����Cd��Vl�:�#L �q6w�����>Ua�[C,!����`<T��#�wT�[n��/i0 �E�� ����I: �y*�W���.��:(l
����5 �X
~G�����������}�����Q[x�|aT>�ee���l��6\��^`c�2�|T���i�:����b��0�D��L@��$�:q��>�T����`�7� %���=��1:��������������������hD��O��}�F����PM�X]E ��})��������B��(������Hs���nc6�-�M�j ��Jx��4���v�h���D{S��< Jp�D����>�l6��3����O'��l2�:��� 3����jj�D�l�$�*Z�`��?��F�nm����v�M���:��J��x��PLR���6��'�9�^���j!'!b�5C6����C{����F�O���)�){4�k)���T�e�.�30 ��'
�O�/�gTG�H?�sFw �+��}���$*',P0V����H�F���]�d��isb���cU@�&%uu�_]/l?����0�=���%����f���wL���^4�<9 ��D�����aCyAn
m��1 3���tv�����F)U/��J���; S�A�� ���O��(�\�bk[ �Z-v��P���wZ[U]�n�K~� h�^�S��2��GK�gW�E�R�������Y��'���J <[�ihk����)�b��- ����t��p�O���x+�6�J
x��4� W������@�.����|�B�h��_��*b �~7 0�pl��::�I_���m�!V��B���H���U����9��@8��I�"�� �#M�ux� �v�+�����R��h��jk����t���oO�"��R,O�A�V���:<���.
���E�
�y}�������Y�a!B&���x�v����4��w���Vm)���jG)��QH}s;�R�� ���4�������s����M��z@�?V"g} E�BH��� -(M�l����TM��G����C���2��k��{7y��O��FV�z�6�PW��������|�k�K-��V���Z���V9����
v������#��nw��$��bw������l���&n6�R�i4�f����vF�a�ZP>�� ��r�o��\�����,X������@ 6665)��"7c�3���:���l`z�A������@s����`0����g���Ou�!Zo���3C��.�o �T��+~'4z�=ST zM��zb;|�� ��F -��( � �C�c������n�G�T��������m����? P������^���_��w�V��;�j�H�,3P����� + 8���4yz��dM��}�S����{�����7����4�� �7N�v����v�b��^��_ T���7����t���,�$g,G�V�%o���� A*u� ��'�b~T�C4��H��k����d
�a��'T~��g��u�/9��zB��f��L�
�@Ke�����5��"��%��.��"L�M@��<���� >��}������6:�y��J�B����p(���K��Dd���U�1� ������P�=��s�Nudf��5i����*��< �Kn}���Zw����/��� X?���3��Z.t=(oZ ���k��}3�� �k�q� R�
�����$%���f0X���K��:� ��� '��;�#��� ���A�o2�N��v����f�h��n t;��xt���l���%��Puj@A�]�y� V��b���]���p�^:����u�s��mh�}���m��1�M���O ��Pu�&��b��8�� v��-?d� ��;y!y�]4B�PFM�/1L��d�3�������Ph�����=��&�0��@��P�,��hb�j:�8�K��M�q:jnL�sXlB��E��V���\����f�����l�9�����~�DB���� _�
qdZ, )0�B
I��Iq[���<����t4�z2��BM�SsA!| KO��_
UD�qvy�Q<:$k�V�����(�{�1
�6�ua ��tt���u�g7�|�=���I'�Lf��e|w�3�5�99�������p]&���'�G�����u=��� G��Q"��D�����0����9a���h�uO����:�Ir�g��.��Ra�Af�XM Fk����5t�<��CtH��g�m��!y� ���.���I6X�R �U��&���x�q�� c5S �%�z^][O�b�e(BcC�A�(~��J��^���N��h��R�1]S ���r��� �����2�l!>d��!(A��n�(� 3l���Q7����<�{���C��� �_���p4l�\)��_��?>?15���P���Vn��x[zh� >Ts/8.(����P������ ��7���t�u��q+%��MW�����i��/!H��A�_��Qb�b����;ub��m�����n�-�����H���T�@�� �h�X���>[��C�c���m��l�ei�
� �A���Y4S �q��G�
�\<6���{\���.aU�
�FxP��x��0Y6�� �'�� 0$�`E��Z��d9 ����y�T��jhv ��.V���� 9 ��������U�X��|�C�q�����{��se��E���F��+����� [�V�C9����"�Q{�W
�#4����+�x~�R��Q��-�97K �����EM%/s����('���- R��2 `�� ���VHbb��a1�3�|�����-k$ ���qz)Uhu�������0�E3yn��Zu,�l,
/Gj�h�E��+�Xxs��T��_�{��E^H��e�-7"��l� �%��O��NoP ��s�r�����HQw{��
d1��:���^���At�ww�d�`26��e�L�z>1.�Eg���$�W`���m���Eap���Y�~���o���������-������1%0���-�����P��A�� �L s��|�I��X�.�k����V�I������]���'�,�f}K� �������L@
��g�� �i-<���`L��AO��]Qd�7!�1��Jk��lnSi�p~ sF��a>�W�`�`�8�8}�V*h�;��1�a��������wg'����.��}���F�"��ux�Z�p���B"�N������9�s����k>�^�G���X#�8���rRs�c��� /%���r���z ^~� �H�sP@]r"��#D�F��,�H��
7k����+&
a�^`� &R]���c0�� �W
:Ls���H�;���/� ��{�:F�0\����� Y�"��1 �e�s���+�S �/���]�j1���T�}:����<_
d� � �He������.���]��}: �I)��&��+ns�@� ���Y�eH�{�TH&w~&��@���� ��A�l��G���������� �5���<s� ��B��$y�������_����w����������0���Fc�H� &�Z#'b�*�����0�F'!N2W:�G�Y8 a5��: ���CP� 1�F�F��n�d���e���=D ��d4�����r���] �RO�k�X�"��v��`�`)���?��}%�� B�@#�.�<��PK7`�e
z
8oJ�^)�~4�_)���c�
���6 ��� ���lh���t
L�p� �O��EM=�zo�{���NG�B� ��"�Q���`�@ ���Y�q� ��$g�l�D2|*[�� 8����9�Ny��(�b�E=z�Q�p���v�F�i� �� ��. Q@�#�qu��TV-yR��
1����L����Az�!n�w���9��������������2I��� �8_���Q���D���H��f Z�Tf�m@3v ����Mlsj�wJX��� ������i��by ���;n��5U� @���Ae��<�pD��EQ!VfN Czb]��K�!(��4*�P��V�<#����X�G�HL;^�-B !6_��?"Qz���\�,5w[>�W�aJ��D���azU~mX��l���+�NM��pv��,���-������4s�Og��.��� �&f� l 5f�|� ��8��`$�O����$����x�
,/z������V����Nz���I8��U)���X�)xD�S����P���E��P&�
?����
�� �l����)'`��l� L�R�JNhk$�D �����Z4I��\�#I����� H��)BDH�2�3J�B4-���M;V��w��99��W�����4k���1*�;D�`gp���J�r(X�2�l'�-��Prl7"2��<�y?7�|�~7����!I�5��8l�A����� ��4`�A�/����(b�����H�b����F\��� �e�A�� �
��o�VI'}k��������fT|U�V�r�^}� J"
x"l��b(��3`����K�D�����Z ��6���3Y0��4��3�W��$��
/r�m�{�G�9�^\s�
� ��h��M�3pg�s��b�C ���(�*���D�3K���
���-i��hJJ��E��� �i���l��c��B�hz}����]O��l���%$���n�[lB������ W^�h�=�h��z�Qkc����&�
���l/v�aV�+� �k?�prviD�����[���A�{z9��� �L�������_����o�=M}0E}sjc����[m��)h��;�JF\ b���k�G��B���@v�b#�C�����C pF�E���?�wx�]#,�XC;4ZTT��Kva�
;�o���� ��=�8=~�S01�2O�[����eB }:�Cg��n�����q�����V��������-�^6��# &��g c�����.�B�<h��y�������i�� � �Ej ���\���o��0k)�b�������� �����\�qY_�6+����
�<&�C��([L'0��oQ�I|�D��E���0�i��K�?0����t�q��ek/ �Z��f��sy���w���.U{kc.��F��}��1��� �.V����U�%I��Qh�H��o���9C��s��� <Yf#eJ�mM0���?�/�A5���pDu�m����o��0w��Q�X�� k@t9�;�D ���>!��������#(�����|�vod��F � ���}t�QT����u�0#�r���0��K�\KI�R��o��3�A^0��M���{���&��+�b�Z����S����������%Sd� �I����JMPa&���7|�`��:|� =�!0�Dp��OK#��`x��m� ��q�CF�#�� �dkQI�;��D���sS1������j� �[���
-HE+v�[���TB���=��S�Y�� p,�����p~\��L�HSO'�i�
��?T��K��5&���5D
et*� 8����%zt�M��d�&D.B�
E�,(gAh�+oDb ��Z*����}��>�0�d��!�\��"������x`�{��D$���ay:z�N�-�5w{8� �A��*H��W&�&i2{�����0 6�������~�����I�mta��Xvm���u���*5���r��LD� �9[��r�_�O�u��{�7R7��'�5_PX| PRB��icQS9�(�� }hv�-���~��m�qz����R���������Nmd��'� L�����0��������\�~t�CJF" �
K)����a%4T���}|_k��R��M$�l�5�C���
-������~�=X)���� ��<����������V�P���q�m ����� ���zn��fK���*�QSK"�����0\�9I1��OH�G�{ �lw��$y
���q
�I�P�}� ����������"���4aB�#3�jm$�2�!mS�l�8(f�"^���[�n�X��9807��UV ���@�<��&3���l�� !R��m��yK�?K� �]�B��O��*L��� s�T�����]�O��3&>9]1�IiX��8m��:\8��.�y��u�� �J-�_8��S�"�����6��dg��x���o�U�^��@F�7P�S� ���,��L��Sz� �����.���S��&[>����NP4�D*���������8�� kr{��b���, # �b�n��;(� b��K#��8e�C#k|^L'����u�-���;`V��������������)z��Qf57jXo��a�E�7��5
����� ��o��>`�6?F�� ��A���}S.��U��[:� �� ��9��5� I������^h�A��8���a�w����5n�_����&�<�I������-��ZG(� ��G�����l"�mB��!,�P����������;7��WV*X�IUx-rK����.85��
���t��jb*�[�QM�k�� ������a��l��;Y��r��1�5�V5q��1�sHy���.���Od�Bo�x� ��]��D2G��Rg� Zk� 4�.'[��t��l4������;k$1�
� �i��F{,��0f��_<d��m�M�A)O*
���X������d��� _ ����&������=d8 �dwE�xZ����
��b��x�*��jK����~��r1Y�+������ /��<!��=dFX�wyy ������xA&H
$�C���;�"�G/��2�!� �z����KO��d7��*�x4$ �g�G>�Y�=ooy�x��������3�]���0W��!l�=UZ��h����~��]�s�=� ��5p�j�S���qy������=B���=�?������7J��G���i�6b��E>B�*L�����{���{i����N�y��0����od{
�[���w�f� ����<��1 �\ ��u�F��I�!Q����#b���� -4��j�d8�Igy3��'d� ���H
�\}L
�*>C��ec�9(�"� e4���%�E��':��x]�5s�a�9bL�,�0�Bh��ij�q��8m$���%p�9�k �����je��[��*l}�r�
�H?���������m��[2)��J��,HrHKb#�+ qN �E��[[oN��7���C�����~OO� �c�
�������%��)�=C��N�`�`�&lEUKi wgO>�&����y��s6��� b�Y�<��E��| @� ��hx�F���^D�;�����
nxO&=P�1i�dkI��C����
��0���[I�U2��*^��|'���i�{:�$2 _�B����'j6Rp �-l �1���g�u��H�����Hp<���6�E���VN����?��M��-
������R l�M1��/�-�fV�Ss�f=� >�E��b)�����v��
��`$�|�WA)���|�lE���xd� �}�AB��s�Y� �`��E4\MK��|�+��G�J�hG@+��x���H%W�u������(K�^>� c�Q����]�L��%��!��u�# "�M�4��s�<���*Y��w��v+��� ��� `�L�v�f-S�^$���?]f�]#���xo)���
��z�� 8O3e�\)fS*��������,#@�%��� &�w���5�1*X@<'� "�1��(��=$ 0�$�rs #J�����t���V��c�M��G{����[�� �s�Ctc��=� t����_��l�Y���,)����s.���V��
�U}�~
�,��^��g{�hVjw�p�x������v����1����9[1��N`�w1Y��OG�S���"��,�R�����c���>G����PPO�������q��%�EV����:�X?-��D�g����#:d��������X���F-������h���Q���Y��@�oO�9���S1E� ��f"���; ���Z���S�\$z�������� �� � ����}�2G��uT�%�S���C�Ay��i��=]^�Iz9ed/��Wu�K�z������| �*�^�=\R�m�2������V��{z�O���c���K���qz�a=��@��v�����Oj�� ��4�J���FW�����
E��"�3� :��<'E
"��2��r �n�=��`y����K(�h���G�_2�V�#�YG��GH���� �������5�o�\uW �����C�� \���y��y$�W{����cxZ���#������.v<��|�C�m� ����u]#b6Z�"�6������������� oA5����%wI�������`�n������b�����Imt�a���j�`eRv����q9�)��A2F5��S��k� T��j����
�|�a)Cc��SC Q4ZBf�S���z� lG� �I8(�K���9��F*j(`��lm���$�I26�?�*&*7i� 6l���� ]��
�D�s #���Dq��c���f��6�����9��U�_���34��\��@�� �]� ���������&99�;b�� ��(�cL���O� q���� <��3������������|���1�nl8�~HHL�������_�"<���.I�{�������i�y�����������%�F,r"F�!e�0|��+���4T��1���(��<~}~"J����4 �"�q��N��)��*�\���Vl���g�2�^�eY0�F��A��t�v��r�h�����! L�Y���9�,��R- O�Q�amPG����I���Z�J}aMZ��������"�KJ �8��M�`8�� �<�6���<�!�Ch��������BG�Rq�YY��_i [[���1�2^������ <b �=<� ������(O���}�d|�+�J_rd]X'�
b���b/����PG�v,�X���
�����@\s��`��J����|���]JR�O6� < .���D| �j�$A;)t_�����' �]�8�t�v#Z�@7��.�E������.w�]��r�:�D%�u����$ k;��]�n9~�,�!s� ���-�@ � �
G�iH9T��z��nO�Y��m�����>�
��&����\��($ �c�,lo��5�����&�g���*����������"�W��e���o��� S�(��7����W�'j 3���T�z@���!�Tr<�R�����QInpo� S��������n�����T5��C�C%\�E���+P����=��_�Q�p�:�����q�U�K)�� @�9Q�kjQ�IANJ8��%R[� ��Gw�`�N�I\X-*�r~Q�R%+B���/���Q����4A��P��<�=0 ��� 1<�K�����Z�� rL![J� �!�T�F��� �Mi_��4
;}7G@L�r�^����2$Z�b����x�.�eCf���}%�/�;�?���5��c�V�d%����AG�M$����v���:����!��ie�����e��o/}m�_etex?�o�M�������W=��j�k��y���F�d� 3L���� ��7�
��R��~�K�5bn�-�����AI��x#����1�� �� �n`�������sM>� A8�=���� �S$ ~M�[ ���kL.�vA��z F]��
���EX��W�X'��k�t������4��� ��B4��k��� {��n�����6�{���^�� �(���-� % �� ����T5�����BP�����QP
�� ����5k VRY�� ����
k�>2�'I�S*���m��?�F�< �t��B=
{��N�c�cP
�~H��e�w�` 1k9���~�����ns�dL�M��X��bj����S����L�,c�&�q�N�s�#�����U/w�����`;�`������r���&����9�,��4�8����2��F�{���t�#�E7/B���h<�pTU�&���O�� \<6�+�����BU����#��P) D�d�_.N�����O�l �H1mP+��h$����`��� S�S:@���^I*���N����c ��vr���Z�-��-�V�G9�fqn- �D��r4�Dg3��xBU�� G��Z��n��
�z���"��a�$\a��H9�rd9����� =�I�1@w@�����c��J��-U�*�������X�[Je_m;�z�������;|�}iM��N^B���7� �!Kg�B7�W�Sk�+�,��',CJv����� �����������-�>�1��9L�O ���3�o�D:�m���jiX�ak�$!OS�r>HK$ .Nm�vE����41�jAE��mS�#)�����Vmp)u��1����")�������p��2���M2�dB����y��w�_P! �GT��fQn����KII�$P{���@b�bh����C�����T��R>�=�LJ��|�#��:�Q,:A� �H��e������~� �.0���A�b�s�
�
�`1�;�4A��]M$FK42���F*QD ���|(��18��,R6 �����zL�H��K���,>o�Q�� �UQ����ZG4�vV�}9T�/���r���W���5���H-g>���O�2��z��Y����)��^}h���W��AY���S���$AS�o�q����M�a��Fb-�C` �K��������?��|��Z�y�P�Z��u�^f������� ��Fj&�^${�[�\���u�u��*���KB�j.�ZFF�8�^ v����7�@W��h�F��uH�6���r�zxO����u(ZYtw���YJ-H�����& W UQ���\�x�!���g0���L� �?:G����$;G����������"3�-�J�-����2��T����m�|�����mc ��_�B���!W ����(�\W�`� � $z��W�^���rx���j����p� �k�p���R@������<���l�]Z2}]'��A�tW] ��UQU�P�eG����$o�[����=����q����~��uC�:G��BQY��+�Bq��}�"��\ � �4bCq�N/.h �Gg�q�����fp�R����S������H,J4�Z�k �\@)X�aRTQ�Dqa�r5}�W2����j�*A�=:�{�*U{y�� �t�[� �7�cK����=j����5sQ %��� _������K<����[x]��@�� en��Aa�����N������l��� cm����*�\!�l��l��;���>]�0�>����>��>���d����d����;B
�%�V����Nc��IB{�t��(4�����]J�q$��p�I��� �N+@�(�$,�+3xh�V��H�~@�Q^%�*�-YzH�^W������k���t������)�0$������T
�P�tn G b_�"*.; nR������� i������-[ ��oW�������g���q<y������.�<tr���5�Bx�a#^�']u�L
�7����S]n? DQ����lVg�&7�ef"P��L x����3S��������mb.�k����*I�� rM0!~o����� ��J����~��p6E������=j G�P ����� �gaz�J���J�-�9Y�R4c _��'��l�-���{����7���K�G����B���'B�<��4� X@�7�����S��#�M��Q�'�+�=VW�|A*C�p8�FD� �����4�O5W{�D�H�^v �j�������
�\�
��%~���nm��o��Y�q�tB�xB�����k���H'��D�7����;c3�1L��,zR]I�Lk�� (��[��wX6�N������aJ�[�����<J�@P�6~�e��(������jf�wD!�����rL�0T*X�U� �0���P� = w�.��S��,dm
��D65 ���*F�<�4Yu{m�=j�r�Q�#�v��{����v��?t��n{��1��kn-��{��
�9 �E�1��&��n U�|�bS��,�(����S�� ��s{��v ���?�����dh�Z\���lX���.|F����T;i)�P�=�* �rJU<�cW�}&>n���s�~��ku��d>�o �;��y`��Y����I��J��7
��-�()7W ��p�y���TX��z�`b� !s!�D�{��"N(������+o������!��������R��YdeGIX8���:7���)(My���j>k�U�S�i�m�"��a�7���m�1����V��g��_���a�Iu��tQ�q�M��N�.N
%�P��" o96�\*:��y�����H' sv��]`��>�2�����@oI��u| ��7|U
�
1�9p�4-�f�w�������2�e�M�xLKx)@���R_��#��#ebH�
�����t�Ncg/'�
D��� v�(�m������ys��{� �[�M��OU3p��5me��U��`�7LdF,�Dj�`-0y�3�N��)*$b��@��H9�A��g�I���p�T��%4=��1z
������� ����� ���&(ll
�����3@Z-{w�4��hjl�EC'�-�������(��^l�:F(�M�So&�0d���qfWp��om`��B��-�&.I\��joY3de��?�o���@������<�(�X�6 \�OV��h�PZ ����E��1���!�:�!���K={r����*�noyi6��B�Ac��i��E������� �gDT��C�O�7�� �J�?�|)[;K%Oy��m���� 5����b�+��x�7c��m�1�%�`J'�L�)�� �p���,(�d�9�
{!�����YpU.n\��.���vM�A/����M��*�F!������=��I��7���H���e��`���J�����M�T�����Z[*e�P��z3V��l��D��e��������%i� ��l20� r �,���]�\<�������o?$e����RE
"s��
�0�N�_���H"�-��0�q�#�� />*n �y���)���n�� ����i~�O�;k* u�w�&�DaA_B ��b����? f �J@���� �����f�0��� �*W�&muc �z��hKt�A:����7*l ��w�n��E���uGP0[k��E� �� 4y������*5�Vk�U���
n+�D�����$�����s�����^#X��| ��o�I/��<�U@ ��
i���+����E���O�(��z����� ���� D��%d2�owh������&�t-��G�H��k&`V�[��� ��P#�d�����kw��b�X���2L���}����r�B�\>�;\mS�A3�yDEN��R�CC��A:2 �0�^`�� �0��'S������{t�����w^����*(=�)4C (q�6���4�T�� ��n,��a����E�~f �b����Bq�|�!�],n)�Dm��q�C|�8���]��������P��eh���hw�������b%(?� %�4DM~\���� zNV���N^��!���d+I#S� �P�p��"�j��� >��~.� ��W��4��h��eF,F��`eA�� Zj�
���9+�P-��6����]�:���" �������+�\�=�t��Day%9C ^���-*�h�cbM�\ �<�������|�p�3����Ae������~� +l���T1]p4���=g� ���O�� Y/�N���p�RIi�3q��Y/��+_��nG%����
�=cWCR�~!��>��W�?�G���I�;���t�0��Ft��8!O��B{������2�`�> �� K�I �v#��1b���F�?��-���� a]b
�)��.N�8y����N :\� �n t��vw����x,�8a0�2�ZPr��(8�����R���l�B����K�C��| I6���!���w��;�<�2�|�7k���Bi�k,�����jeO��#h����6v�Cx�)o��?�/2i����)&g1�~��8J�]��%��?jt��rp��E�7�J�~]X@�.����M��R��!�����r�U ��uk��<���}� ��� \{8J9 ��R�j�94�b/GTQC�j �I Wc@�m��ZI������n�������
��n� (�/H�5G�}�V���29A�K#�L` �>ml�X�e�;�H���� eL*L���9"�0W��\B����[�M�H��N����=pK�Km��
��9�������]�� ����b�%p%�O�1 ���� d �'�a�@\x7D^��! ��Hw���R %7�i��cDV���L��,nxX���V L
5����"�����m P��$��]0^~�h>F1
�����R������ecug� �����xN#��1��t �P��n[O��p����S���+9��}�����C�u;_�xk �Hk����c~S�5 v���j����W�I�������=?��K0>j�Kp>j�K�>j�������~E� ���|.��f��<���q;��� e5���>xw��Da 1�\F��w�$�|�����������;l�S2Y���g?��J�(��}-� ��� � �N�q�����WF}C�,���9���%�h�: �L�(k �����@im���RB��� ���p������(i��D�@M��t '��e����"Z�R4[%�}�G����8�=C��3���`�����9�/���/��L4 ��K�e c.��<8h�V��p�aT~�����I(�G�]93���5����U�� ��|bx�Bd'��$>�07������\� (.�0�{�Da������8�1g�� ����"�)G6y�����j�B���!�n�i4�B������
��l�C��� Nz-A���l��<�����A��eh���rl�p�����>��S��Sg��<�eq�@��
���d�b����Qf>E3h_/g���o�r��� ^m�
������,)G���b�L"�t6�,HtOS��_� T��V+�rN�
���4dZ}L���-` `�n�2n�`��d�V��/WIvdq�P&I.����l`� �4��b����0�Y^��L7V,/9�Y� �k��\z�*���� �3�v��br��*�����O��N^S�����Wy�"G�6��=&�i'O{�
3='<G���8��eT8��L��FYS� 8��c�M�
��(�0H�_"?Z �[��p �9Fc'[r��>1��{����v1�1����u� w��� �����9�" �ay�7���9��F����� �B��M�C��R3�Op�u���+h<�������,I�!R�EJ��0�{�`6��7%3r7
L[�EF � �8G �6��iXT18{< �lBA��!�"�� [j ��B
��u�
��v>5\dL7,� ���b�m N&o�����Xv`&O6t�3� ���y �C���P� �MV�V�hp�����|�H��Lf�!�Wa��8�������?7 C��q����=��y�!�z��v��M�����:B;<�q����������D��j+��y�����v��)\U��;g�st�\�B�KX��!_��S 6_����s|q��-�{�H����f�P���a++���Hk�.��AG%��h�e��
HP)��$��
�P r�K-�UGb���B��"h��1`F�R��a
�I �-#.����ZfK�Y�� �� lC ���f�~��$VI��5�Adr1��.�{�����8 y<�e,'�|$��� HLN�!�\�s[��]����aHmH��}�bY�����9
�4E���4��;fX/A�n�H����|��br�M����r!� D��$��������X���N0��f��)�tn�x���)g�c�UY�gK��Y��4&w��: �� � �%�|`�=� `�H����r��y{�����,����Lo������5M$F �0Nz��&W�R����y��"0��I���`� c$�����$�D�^�s0�����
yv\�)D�.�i� �/nr��� ��|'�:^cp�8?X�BJ�a�J zk���Z �ochijuw_�~+(�W��O��-��\2�����H�5�������-Vh� C��v9g� %A0$�DA=��y�����G��%���7���O{�� �~�Px9 :2��b�N�`����I�������� �����u���aV�
���e��O����� ;�������2oHrR��"���$e�$B>�u����k-���cr���D�T [%�� !���M?�:Kf �������J������ly��`5JW��
�#F�>��������4��M�)�wTsP%4+�Dq� �9��! c��%�nE��@B|����t2�
8�<�H�-$r�Um�������K*Y��\B�0� Vv��C����E��� ���}�}��R���fPG� �gx�3�~D@y��<0������U�(�<����O�����Q ����P�� qf�c���`�?k�� g������9�+��
���U����O�~Ab��� �/_��� ��f��� ������b���i Y�UNF�da�Eq[ :�$�WjT<X r8�|�Y�vP+�64B4��:*Y ����y�-��c
KZbf�� vj��C���"XEQ����IqZ
�Tr�9�X=�Z� ^ Z��CVQ�hL�F�[v&��c!]
#{���~W�\&v�T.��39�M(@ n��� ��*���.:7)�� ��
v w������?UJE*'Z�(����w��S�'����Z��o�h}3k^�S �j�p��qS,6�}����_�a;T� +xY�
��� ������&N7�*.W��������jq]��eXn�^
n}�Cp|� ������7���Z<~%�6y
�q#��f)���) r� ���� ���~0��? �� F^\��A��2������u � l��Kg���o���L�v/�{�O ��Q4�A��lo��t��j�%<����v{���W��`�3�|�(d�7�8��zj cl&�?U�,7p5]' ��U�� -��z�jQ�D�B�e
:�I�0�W���>4�+9qC3��@!T x������q�JFZ���b1lQ��Y2Y/�K��Yd�����I�aH�=O�a+$1OY{)%G�r*�^�z�$k�.��oo;4 ��� <�/� ������(�Y��"dc�$��R���A6�QR��nDn�HCn�|1$�a�� M��� +���G��!����y
� �R}�9�Z[� ��u���5��V������w���{[m2�Z����4$�/������ ��I7"!q.jatt���mwX��T���3�����W' ���k�F���4!8olG���Kc�\�6KG�M�k�m�7��_���v���M�0���:������uY1w
�Y��l���]!o &�bl������! ��M�����6�:|����$�����,,�<9c�TK�/K���'E~|s�E�X�������_�+����=��>��j�������S�:�F�lP]O]������kNXpx(h�t����6f-H����(Y�7���� E��uz1��5Z^9��� �� /Pm91{: }�$=
T�<��GN�h��e��s �QSR����]���D��5 m��'��J-S ��+,��d���4������,s�"
��i4��E�:6��Xi�Ukn�A� e��:��Q���� �!������5E�S��Bb�d���b#�� ���&C�J���=g5�_�u���A��XT��������0M�����Vx�f�[%�� ��s�Vf"Y!&����f�X 4��S?���;-M�����B�A�`�o�R�o�]cC^$�
���P&[.�M���E@��Y#s�nAd�FX|��[W<dq-F+?:G<%d������Y�Dc��_��-�n@���N���'�����0A�D{ 8%�M� �8��3:�� �-�&�� >T����n!��p���G��O��R��r��]F��� ���a���A{�����Xu���������p�w�����u�l�x�} uP��: ����f��Yh�2m�V��w�uV9����k � ���s���#�YZ�\�vMG�����+3��=� \�K�J���vq *4�U&}������h� ���M������{g'�<;?�����W�'��NnG��D*zZ�r+IT^:]W)���# ������
����-�h��3�g�TE,��H��8���^���E��BY���3�&�� � �������Z�u�H6��BJ,��Zv�Ml�eVkW;��C�N�
����]�x���'-H����}!ji���z��s�����2m���B�{ �%W��' !m���#H�������|��)��2�@"���������EWX��� �TQ���
5�
h�fQ�z-����
JZH;��2l���DD� m��u0��.�g���j�e�f�(\�����@��(������0��2� g( �A! ��Q!S��'y�AzS`��JV�����d��r��@K���:h����2bb����fI<0KR6�WqV�A��C�dQ_k���R<E=g#���W�]]��N�;�����S�Q[�UD�\0������ 0d{X�Z>)Y:hX��"��'#� )N�:�0�Jx��4c�7�rc%�+I�����c��K��S���������R2������%�8>H���W`�( ��!�"��Lk�������������\P�-d;��V��%�x0�8���p� �f���v(q� ��_9�"�V���T������ ����r/0�]���7�����{���[�3O�� ,�c��6���:H��t�_�c��� ������-�� �VKHBH[n�M�e�B��Z0]������X`���6��*0X�|`W6�:�D��n��_����kG� <�S���j�F&HA?��/���M���-�J��0i��KK�r� I� k<J3��a�iL�� R��/����bA($N� v�#~����H���iYz,� � ��. g�vfC��� �;�E��4�(l U�>$s�mR�t:�i��<��ADC����+Bmn���V6Qq3D[!=AJ F��'\�����������������x@�0K�� ��
���t8xa�q�<��N�V7�F9�`�ds&�
�/��"�y!ES%�J�<�+FZk����]����t�����g��$�S+9��2{�B`@o�*�M��G�P����t
+�ya��T��- (G���.O#�?b��r����e����6-��h+��[� 3�����+@�v����TRV�N��wzfA��$Y��0�P ��cx� F^���4P�4�*0���a��
�*�H���+OH� p|��h��I�@��x��!�;���2�!����t��Dw�0�9�|C���n)U���� ��"v8����A��B�-("��G��KN��[d��l�6�o� 0�Tn�����H���V+�9?� �(���t����j�+<���)���)cf0�{ ����X��"|�i��m8�a�_���d� ���M��7��&Tt���� �p�[��r���4��-f\��E��(e�.�i��e]N4���� �j~��1k J��y�b\ ���V�%X26����L�
s�X5x���hH��'��� �U��#�@��y%0d�`�i��(�e��������[�%w�:*P� Et��8�RjJ`�V�Z� �tEL-E��S�� Y��~��8���l��7a[W*��^����_��"�R�{��}��+���e}UM���)�$��� ���p_>!�)�B��A(�cTu�����.��t�<��K�}�:d��3a �,�r�JI<��B _�w���q�OwZz���d8YS�_���{{��������f��;�{GY:��$�4�� ����Y: ����5U�^T%� ]�*T\q�p�c��v^#I�p/�}����� ������(dg��t2H)��x�0� a��[X����j�c)
�^�����2a8����y��Q�r_��B
k��*�*o��X$'�18��s������i@K+�����J� XL��X���
Q����9
�m��2��G��D��9�q�(#�Z��:����\f��g�on��Nk�������u�(��pE��)"��h����z8�� �)z"���l,�?&|�f�Z��q4����������x-z�Hh�� C�0�H�Zy�����~���;��;o������ �h��<z �����zU�#/�?�je�^���D{��)�D�Q�����]�"���b��B�����.�!Wa:����U*��Z��B�=� 0��{~��0�t��n��W^�TM*�?�Z�K�q�����~g/;��f�rx�������k��L�.��� V�����TYo�i(i�_����������Q����O�K� �G �������b�q=���P@����a����~<?9;?�h$�����O�� ���}�{sr���t��^, �b2��jO&���� �� ���H�� i�V�6}���v����6�BR9������l?��&��\�
^(p^���Dg�zx����������gLa�G��� �6������Qz��.SX�
G^��@[�N��v����5 �p�l� c����L����6�_^Re��e����� ��S��x�S������Zu�$��Y:��p���8���5# �wQ�^z��e�!� _|�O�����5� O��~=3��r����u{o�x��� 3��K#r��v��V�u�f����u���"�
B
! ��Uf �8]�`k���C1�����g�#% �:-(� ��\�
# 0'[��y �}�����n�����K Pa -s�j���9�S�@?a���igdl��]*��{��*��yA`�P
W/�� ���� ��aug�
I����������� `"��Ob�B�2OU��[`]O�<�!���%L����8�� ;T^�s���R�Fw���`�+��h\�A,h��8����_G�o8�K�=4��{-����8~%r\\sd�9�� ,~�&_��bAL���Ou0=�E;�;�Y@�X��G��6/��?�����F �uXM3�S��H�R�dC���O� .�o���I������ ��H Uu�� ����J &[�����=.���L~��h�GL7a������F ���,��� J�< O��4A��'���WX���g
�>��4W������nG�3�-��{��n'i�e]=7��r��"�7��+�����m�!��0A�&@�a��C���-�A���������HU~~�ys�O�0"gq��^b SY#���������N��v���v�P/!�E13���9m�DNc��GW[����o����X<�5�%c�Q�d�I�$��D���������"��6��@�~����gg������� m 6cm/���+A]�)[��G��C�D�6�b��sF��K���v,�d1B�1�����cr%�0<��� �7A�c���-��n�5��+��Q|��+�!0\��Qi_D �p���� U��lq��#�&� � ����N�4x�P���D�p�j?�H�����G��.(� 8�U��"�?����[�v�a�� �����ED`���w+������W��g�u ��B�5����.����U�T�"�%� �h��$�7D�Ls����-��)��]�����b>\�)q �������)��rY?��Y�.D��� j�7�E� �iL��f������ �������G�pH$�[xl �2���=�7�u.�KH�.F/eE�_f� �� �� ��y
�Cl�@�s�������Wr���`z���>�J�C
����B5m��� ������WQ,[�J ��5^�ysq�����&K����B_*.�� �� we[r��(R����W~�����R"��e�e��"�%��5��`.T+���'`:�0��<��A���m'/���iK��(-�t�� � � �(���{�\z|����� �f�gP�x��|k 4������G`���TA 2A�& ��F������O ��s�x���f��
� �[P��Ub�xA�F�I�#i>� L ���'�l��}rPS(]��4td��6�+ ���f�/�^M36AY��4GT��5Dhn��>��X\9�s��5a���T �ePn����It��@�eO�UO
�@*"� Ec>�4�Jz{�Xz+ ou���H�3[�c"{��N6��$D "�bI'8���*�E^W������*Q�.����� Z��dm�Yg:����n� �
8"������w�T
1
�����gM#+��+*S�*u���C�!U�#6���D|�M��* @,����&Tk�>H
%E�)8{��y�#G��i��D*P��G��CE����fz�aE0�Ih!�h����7,�(��|� ���Z�*1u�!~P�����r��`�O�iBT}2�� |�Ty��s��[�St�N���px*�oN�>ys�GE�����Hk��%.�T��R� �6S �I� ��Jy.r�Qa@�7���q� 7tt@�e<x��#�q�I���&W�n����g���L �/
��� �
byTT����n���rE�Ot��P@��a\�h�1��������3a�XC+�B����7�O��^�>���� ��:/�aa=BdA�t������������$LU^���% �b@<�>�M��]��V��w�#�� �8'���t; ��N|'"_n���1m �e"���2�����;�S
�cXwi��+],.|�����%�;Z�����p0�m5���aw���l]V;Zb-�}-��0�����l��Gc'X�����dq�'�|�A Y�<�!;Z����a3�H�zV��P
�o#o��L�=�|r^b�x�.��#4A% �~�6w�E�, ���RCXf��:��@���4�/ ������'�� ��7!(U��
�1���A?=�����~�o
w���R3+��^���x�����������7���]P���h/0�*wx'd���~����z��i`�� � ���O�{?� RqAsYAH�YF�'<��>���@�g=#�B���8����������^
��
� ��6��Q��iw���M(D5��J�k@)]T$��kJ���2�#,�Gf2L��y�5V^�����?��ta���MQ'�z�Qvx�6F� ���\,��C��9�FU5rEm�
$K�T� G ��wV"Qb����.F�;����'`$����� �@��Fj4H�XMr�B�� $,;���
oR���)� �������U�:^�h5�o4��'@ ?�Eg'����{�����1�p�vw������=�h��Mf��}<�_��f�V.��5Z�H����e�=�e�k{�t������0n�c���,��%! Q�O��hw��F#s�Ke����Al%�� �C*(��@�%���_� e9F1��W���������a��D��N�0b����� _q�F��9����h��\��F��:��\�5�U����!�����h�B�~���2�;����U^���c��#�T��%i ����f}S���:�6��azAC�nP,�5a��wS����X�E}?�K]����V�!���tL�U�����@,�,T�f��v s*��>+���-g�0�,��"�!#P����[��J�+��@� �_�5�29a�����,�U���b3r����Y5�U
H�\�8H�c��U[�
G�� � �����w��+c���pYv<�#��V�\v�9 ���i3���m��-�� �����_1XI�sa��7b�b��1�P��{[ \������u�����kN,t����:[������WS/���j��"�Y� }�l�W �`���H� _��������$� ^�;KY? �n� ����h8����T ^g'�w�&0��C"22s�*���}WY ���M��]��-���C@k�{c>1��b<Z�S�E��+\h����������{�#<E����|=�~@�^�W��kr�{w:���D��c
���`9 �" �.a&d�^w"]�t2H�|�i�U�%�0jT#z}m����V�\��� -��y'.�=���o��/�"�"�x������������f� DjP�0�+�O-��j?�
�%�z�!&����
�Y
��U����"����z�)b���d&(��K����b�V��u��nJ L�"+ �3��gDB:m����s���e�
d;�|�n�k�GK��|�6/�OY�Nd^�k���h�f,�����E
���v x.��z���S��!����b�5cMU7|�����M��,���/�b�g�Zk�Vi�������6�2�P�w�}E������9l
:�f�?�� �]v7�f��V���5�j�����b`�E�������fV�_���C���Cy�+�K�7�-b7���e�7UeS�n�j�1r_��FL�7��uxv5�d5V}/�_�1���9�N���C��|��i\.U_��
2r�,�$@/ �#���O0���������^'kw����a{7��=��G#��"zy�$"b� ��o1�a�c�� .x�| ����� ������� �a &������n�sF�Q�P>�FD/o�T�O?��W�idSi1��[��������� d��6;��H7m�7���&l S��������$2Wd����#�x�7����z� ������{ ���9W���l��K���2�d���c����h� ��a>��s�A.����)���_<$0X�
h��c�e� �{�@���1l���(+B��=� P�8 �P�U��@�Vr��`lqD�v�dL
\W �;�D�1\
�R�1c�n���OF���oO��N�9�r���h��^�G���fsw��h�sp�e��
i~\�����#��������������������9�����|;���Q6��Q- ���D[���������].���z�"�zW������)�d^�w�1Lv���� �L��t��_��� E����=�����H5"j����l��H�|�[��u�p .��l��k'~��t���meWS�jAu���@��9P9��S�������d+q����E:�m�#���_~����)��y#Z-��r��mol��UD���{� � ��{G{�z[�Vw������z�W�q�6G��0L���-�[��a3���,3T8I�Y�/�_�3�2�7��~�f��� � �/���1tGf6�H���w�N=l67C ���a����}M�1��t�����*������� ���>�-Y�9j����N B� �w���� ���&��='xq����<5H�����+����a��v�q�� L{�l��2�>XL�+c���V@�)^���v��.G�|��#���-x #��C����*z���)���(���4�� F�������PY�,�m�mB��� ^~`t:�i�1��|���i6[���� �� � ��O&��H �
;h5Z� �����?yu���������������'���G ��� Js#?����( �[�d>����M�{��M�5�wA�:Q%c���?�u��r�T��o_��1y�6��y'� ��C��
���������������`�������d�`����7� \d AZk5��zr|���M����HxT�G���t 'y�����H L��zu��]�i#6��#��>��Fr��������S��"��1�HQ���P�7w' b> ��^l0���0z����
�BU- ��$hMW.W5���y���T�� ���# Q�!l�� +z�5�����?�}|�6�=w~qn�=t+:k~2 ����o7?�\���6�8K $F7f��~��i��x��Q��%L�+1�|��� � �������h���"� �g�[�v�M���o�9��$�3�&4��$? ��,�j�����o\���V��Yd��y� �1�� _g ��G�����h[#��W�z���.N^$��iz���r�� �o5���zZ�+�}1b��Z�R�����#2��I�!&���H����U���*�Aj�Z|�JU�r{���� j�ex$k���+�.���K�K�����b�b�� l���{��}�8�3 uGC=����88���w6;?�c��<?G��_t��M��'?�prv
�6����_z���g0lN����G>��hw�g�7���� �@~���z��/��Q*�� 17�?�i�[��h���$ �R����rx\��<� |X�pN�� �����a����x�����`���&�V�:���'�hW��� ������3��_���� n�����e'T���FM~�K��aL1r����� ,�:��M�f�\}�~�� ��' �=��CS� ���gs3a���_s���<�E��Z���������?~��uVW5X����r���������� �G���`������Gp��4Var-��A���g�� {��$�%������z�H/�t������A����<W���O�
!� ��|9) �8BB ������&��*r�,P����R9,�z���G�}2�����\^Z� 3��;��t������v����l�`�� � ��i����^�T����Q��%���W��]��w����������0��f��6�} ���
D���9q����7�����s"���K �=��qL��]wuN���������E�����n����_]_�|��Q=�*+������Z�����Z�����x5��������?������~����_�����?������~����_����^�(�' � 
Re: [HACKERS] WIP: Aggregation push-down
Antonin Houska <ah@cybertec.at> writes:
[ agg_pushdown_v8.tgz ]
I spent a few hours looking at this today. It seems to me that at no
point has there been a clear explanation of what the patch is trying to
accomplish, so let me see whether I've got it straight or not. (I suggest
that something like this ought to be included in optimizer/README; the
patch's lack of internal documentation is a serious deficiency.)
--
Conceptually, we'd like to be able to push aggregation down below joins
when that yields a win, which it could do by reducing the number of rows
that have to be processed at the join stage. Thus consider
SELECT agg(a.x) FROM a, b WHERE a.y = b.z;
We can't simply aggregate during the scan of "a", because some values of
"x" might not appear at all in the input of the naively-computed aggregate
(if their rows have no join partner in "b"), or might appear multiple
times (if their rows have multiple join partners). So at first glance,
aggregating before the join is impossible. The key insight of the patch
is that we can make some progress by considering only grouped aggregation:
if we can group "a" into sets of rows that must all have the same join
partners, then we can do preliminary aggregation within each such group,
and take care of the number-of-repetitions problem when we join. If the
groups are large then this reduces the number of rows processed by the
join, at the cost that we might spend time computing the aggregate for
some row sets that will just be discarded by the join. So now we consider
SELECT agg(a.x) FROM a, b WHERE a.y = b.z GROUP BY ?;
and ask what properties the grouping column(s) "?" must have to make this
work. I believe we can say that it's OK to push down through a join if
its join clauses are all of the form "a.y = b.z", where either a.y or b.z
is listed as a GROUP BY column *and* the join operator is equality for
the btree opfamily specified by the SortGroupClause. (Note: actually,
SortGroupClause per se mentions an equality operator, although I think the
planner quickly reduces that to an opfamily specification.) The concerns
Robert had about equality semantics are certainly vital, but I believe
that the logic goes through correctly as long as the grouping equality
operator and the join equality operator belong to the same opfamily.
Case 1: the GROUP BY column is a.y. Two rows of "a" whose y values are
equal per the grouping operator must join to exactly the same set of "b"
rows, else transitivity is failing.
Case 2: the GROUP BY column is b.z. It still works, because the set of
"a" rows that are equal to a given z value is well-defined, and those
rows must join to exactly the "b" rows whose z entries are equal to
the given value, else transitivity is failing.
Robert postulated cases like "citext = text", but that doesn't apply
here because no cross-type citext = text operator could be part of a
well-behaved opfamily. What we'd actually be looking at is either
"citextvar::text texteq textvar" or "citextvar citexteq textvar::citext",
and the casts prevent these expressions from matching GROUP BY entries
that have the wrong semantics.
However: what we have proven here is only that we can aggregate across
a set of rows that must share the same join partners. We still have
to be able to handle the case that the rowset has more than one join
partner, which AFAICS means that we must use partial aggregation and
then apply an "aggmultifn" (or else apply the aggcombinefn N times).
We can avoid that and use plain aggregation when we can prove the "b"
side of the join is unique, so that no sets of rows will have to be merged
post-join; but ISTM that that reduces the set of use cases to be too small
to be worth such a complex patch. So I'm really doubtful that we should
proceed forward with only that case available.
Also, Tomas complained in the earlier thread that he didn't think
grouping on the join column was a very common use-case in the first
place. I share that concern, but I think we could extend the logic
to the case that Tomas posited as being useful:
SELECT agg(a.x) FROM a, b WHERE a.y = b.id GROUP BY b.z;
where the join column b.id is unique. If we group on a.y (using semantics
compatible with the join operator and the uniqueness constraint), then all
"a" rows in a given group will join to exactly one "b" row that
necessarily has exactly one grouping value, so this group can safely be
aggregated together. We might need to combine it post-join with other "b"
rows that have equal "z" values, but we can do that as long as we're okay
with partial aggregation. I think this example shows why the idea is far
more powerful with partial aggregation than without.
--
In short, then, I don't have much use for the patch as presented in this
thread, without "aggmultifn". That might be OK as the first phase in a
multi-step patch, but not as a production feature. I also have no use
for the stuff depending on bitwise equality rather than the user-visible
operators; that's just a hack, and an ugly one.
As far as the patch details go, I've not looked through it in great
detail, but it appears to me that you are still trying to cram the same
stuff into base relations as before; shoving it into a subsidiary struct
doesn't improve that. Multiple people have said that's the wrong thing,
and I agree. Conceptually it's a disaster: a single RelOptInfo should
represent one well-defined set of result rows, not more than one. This
approach also cannot be extended to handle doing aggregations partway up
the join tree, which is at least theoretically interesting (though I'm
fine with not tackling it right away). I think the right representation
is to create UPPERREL_GROUP_AGG RelOptInfos whose relids show the set
of baserels joined before aggregating. Currently there's only one of
those and its relids is equal to all_baserels (or should be, anyway).
This patch would add instances of UPPERREL_GROUP_AGG RelOptInfos with
singleton relids, and later we might put in the ability to consider
aggregation across other relid subsets, and in any case we'd run join
planning on those level-one rels and create new UPPERREL_GROUP_AGG
RelOptInfos that represent the intermediate join steps leading up to
the final join. The correct indexing structure for this collection of
RelOptInfos is certainly not simple_rel_array; it should look more like
the way we index the various regular join rels that we consider. (Maybe
an appropriate preliminary patch is to refactor the support code
associated with join_rel_list + join_rel_hash so that we can treat those
fields as one instance of a structure we'll replicate later.) Note that
I'm envisioning that we'd basically run join planning a second time on
this tree of join rels, rather than try to merge it with the primary
join planning. Since the numbers of rows to be processed will be
completely different in this join tree, merging it with the standard one
seems hopeless.
BTW, I noticed some noise in the v8 patch: what's it doing touching
src/interfaces/libpq/fe-auth.c or src/interfaces/libpq/fe-secure-common.c?
regards, tom lane
Re: [HACKERS] WIP: Aggregation push-down
Tom Lane <tgl@sss.pgh.pa.us> wrote:
Antonin Houska <ah@cybertec.at> writes:
[ agg_pushdown_v8.tgz ]
I spent a few hours looking at this today.
Thanks!
It seems to me that at no point has there been a clear explanation of what
the patch is trying to accomplish, so let me see whether I've got it
straight or not. (I suggest that something like this ought to be included
in optimizer/README; the patch's lack of internal documentation is a serious
deficiency.)
Earlier version of the patch did update optimizer/README but I forgot to merge
it into the current one. I'll fix that in the next version. (Also some more
will be added, especially header comments of some functions.)
Conceptually, we'd like to be able to push aggregation down below joins
when that yields a win, which it could do by reducing the number of rows
that have to be processed at the join stage. Thus considerSELECT agg(a.x) FROM a, b WHERE a.y = b.z;
We can't simply aggregate during the scan of "a", because some values of
"x" might not appear at all in the input of the naively-computed aggregate
(if their rows have no join partner in "b"), or might appear multiple
times (if their rows have multiple join partners). So at first glance,
aggregating before the join is impossible. The key insight of the patch
is that we can make some progress by considering only grouped aggregation:
Yes, the patch does not handle queries with no GROUP BY clause. Primarily
because I don't know how the grouped "a" table in this example could emit the
"a.y" column w/o using it as a grouping key.
if we can group "a" into sets of rows that must all have the same join
partners, then we can do preliminary aggregation within each such group,
and take care of the number-of-repetitions problem when we join. If the
groups are large then this reduces the number of rows processed by the
join, at the cost that we might spend time computing the aggregate for
some row sets that will just be discarded by the join. So now we considerSELECT agg(a.x) FROM a, b WHERE a.y = b.z GROUP BY ?;
and ask what properties the grouping column(s) "?" must have to make this
work. I believe we can say that it's OK to push down through a join if
its join clauses are all of the form "a.y = b.z", where either a.y or b.z
is listed as a GROUP BY column *and* the join operator is equality for
the btree opfamily specified by the SortGroupClause. (Note: actually,
SortGroupClause per se mentions an equality operator, although I think the
planner quickly reduces that to an opfamily specification.)
I suppose you mean make_pathkey_from_sortop().
The concerns Robert had about equality semantics are certainly vital, but I
believe that the logic goes through correctly as long as the grouping
equality operator and the join equality operator belong to the same
opfamily.
ok, generic approach like this is always better. I just could not devise it,
nor can I prove that your theory is wrong.
Robert postulated cases like "citext = text", but that doesn't apply
here because no cross-type citext = text operator could be part of a
well-behaved opfamily.
The reason I can see now is that the GROUP BY operator (opfamily) essentially
has the grouping column on both sides, so it does not match the cross-type
operator.
What we'd actually be looking at is either "citextvar::text texteq textvar"
or "citextvar citexteq textvar::citext", and the casts prevent these
expressions from matching GROUP BY entries that have the wrong semantics.
Can you please explain this? If the expression is "citextvar::text texteq
textvar", then both operands of the operator should be of "text" type (because
the operator receives the output of the cast), so I'd expect a match to
SortGroupClause.eqop of clause like "GROUP BY <text expression>".
However: what we have proven here is only that we can aggregate across
a set of rows that must share the same join partners. We still have
to be able to handle the case that the rowset has more than one join
partner, which AFAICS means that we must use partial aggregation and
then apply an "aggmultifn" (or else apply the aggcombinefn N times).
We can avoid that and use plain aggregation when we can prove the "b"
side of the join is unique, so that no sets of rows will have to be merged
post-join; but ISTM that that reduces the set of use cases to be too small
to be worth such a complex patch. So I'm really doubtful that we should
proceed forward with only that case available.
I tried to follow Heikki's proposal in [1]/messages/by-id/113e9594-7c08-0f1f-ad15-41773b56a86b@iki.fi and separated the functionality
that does not need the partial aggregation. I was not able to foresee that the
patch does not get much smaller this way. Also because I had to introduce
functions that check whether particular join does duplicate aggregated values
or not. (Even if we use partial aggregation, we can use this functionality to
detect that we only need to call aggfinalfn() in some cases, as opposed to
setting up regular Agg node for the final aggregation. However that's an
optimization that can probably be moved to later position in the patch
series.)
I'll re-introduce the parallel aggregation in the next patch version.
Also, Tomas complained in the earlier thread that he didn't think
grouping on the join column was a very common use-case in the first
place. I share that concern, but I think we could extend the logic
to the case that Tomas posited as being useful:SELECT agg(a.x) FROM a, b WHERE a.y = b.id GROUP BY b.z;
where the join column b.id is unique. If we group on a.y (using semantics
compatible with the join operator and the uniqueness constraint), then all
"a" rows in a given group will join to exactly one "b" row that
necessarily has exactly one grouping value, so this group can safely be
aggregated together. We might need to combine it post-join with other "b"
rows that have equal "z" values, but we can do that as long as we're okay
with partial aggregation. I think this example shows why the idea is far
more powerful with partial aggregation than without.
Good idea. I'll try to incorporate it in the next version.
In short, then, I don't have much use for the patch as presented in this
thread, without "aggmultifn". That might be OK as the first phase in a
multi-step patch, but not as a production feature.
The version 8 actually implements only part of the functionality that earlier
versions did. I wanted to have more feedback from the community on the concept
before I rework the whole series again. So for version 9 I'm going to include
the partial aggregation. On the other hand, support of things like parallel
processing, append relation or postgres_fdw doesn't seem to be necessary for
the inital version of the feature.
I also have no use for the stuff depending on bitwise equality rather than
the user-visible operators; that's just a hack, and an ugly one.
The purpose of this was to avoid aggregation push-down for data types like
numeric, see the example
SELECT one.a
FROM one, two
WHERE one.a = two.a AND scale(two.a) > 1
GROUP BY 1
in [2]/messages/by-id/20239.1516971866@localhost. The problem is that aggregation can make some information lost, which
is needed above by WHERE/ON clauses. I appreciate any suggestions how to
address this problem.
As far as the patch details go, I've not looked through it in great
detail, but it appears to me that you are still trying to cram the same
stuff into base relations as before; shoving it into a subsidiary struct
doesn't improve that. Multiple people have said that's the wrong thing,
and I agree. Conceptually it's a disaster: a single RelOptInfo should
represent one well-defined set of result rows, not more than one.
The first version of my patch did what you reject here, but then I started to
use a separate RelOptInfo for the grouped relations. First I introduced
simple_grouped_rel_array, but then (again per Heikki's suggestion) made the
"plain" RelOptInfo point to the grouped one.
I think the right representation is to create UPPERREL_GROUP_AGG RelOptInfos
(Just a detail: since UPPERREL_PARTIAL_GROUP_AGG was added at some point, I
wonder if this is more appropriate than UPPERREL_GROUP_AGG for base relations
and joins. create_grouping_paths() can then add the paths to
UPPERREL_GROUP_AGG.)
whose relids show the set of baserels joined before aggregating. Currently
there's only one of those and its relids is equal to all_baserels (or should
be, anyway). This patch would add instances of UPPERREL_GROUP_AGG
RelOptInfos with singleton relids, and later we might put in the ability to
consider aggregation across other relid subsets, and in any case we'd run
join planning on those level-one rels and create new UPPERREL_GROUP_AGG
RelOptInfos that represent the intermediate join steps leading up to the
final join. The correct indexing structure for this collection of
RelOptInfos is certainly not simple_rel_array; it should look more like the
way we index the various regular join rels that we consider.
Nothing like simple_rel_array (e.g. simple_grouped_rel_array) even for the
base relations? I think the fact that the extra base relations are grouped
should not affect the way they are retrieved.
(Maybe an appropriate preliminary patch is to refactor the support code
associated with join_rel_list + join_rel_hash so that we can treat those
fields as one instance of a structure we'll replicate later.)
Do you mean that, instead of a single RelOptInfo, join_rel_list would contain
a structure that points to two RelOptInfo structures, where one is the "plain"
one and the other is the "grouped" one?
Note that I'm envisioning that we'd basically run join planning a second
time on this tree of join rels, rather than try to merge it with the primary
join planning. Since the numbers of rows to be processed will be completely
different in this join tree, merging it with the standard one seems
hopeless.
A separate run of make_join_rel() for the grouped relations is what I tried to
do so far.
BTW, I noticed some noise in the v8 patch: what's it doing touching
src/interfaces/libpq/fe-auth.c or src/interfaces/libpq/fe-secure-common.c?
It's definitely just a noise. Something went wrong when I was extracting the
diff from my private repository.
[1]: /messages/by-id/113e9594-7c08-0f1f-ad15-41773b56a86b@iki.fi
[2]: /messages/by-id/20239.1516971866@localhost
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com
Re: [HACKERS] WIP: Aggregation push-down
This is the next version. A few more comments below.
Antonin Houska <ah@cybertec.at> wrote:
Tom Lane <tgl@sss.pgh.pa.us> wrote:
Antonin Houska <ah@cybertec.at> writes:
Thanks!
Conceptually, we'd like to be able to push aggregation down below joins
when that yields a win, which it could do by reducing the number of rows
that have to be processed at the join stage. Thus considerSELECT agg(a.x) FROM a, b WHERE a.y = b.z;
We can't simply aggregate during the scan of "a", because some values of
"x" might not appear at all in the input of the naively-computed aggregate
(if their rows have no join partner in "b"), or might appear multiple
times (if their rows have multiple join partners). So at first glance,
aggregating before the join is impossible. The key insight of the patch
is that we can make some progress by considering only grouped aggregation:Yes, the patch does not handle queries with no GROUP BY clause. Primarily
because I don't know how the grouped "a" table in this example could emit the
"a.y" column w/o using it as a grouping key.if we can group "a" into sets of rows that must all have the same join
partners, then we can do preliminary aggregation within each such group,
and take care of the number-of-repetitions problem when we join. If the
groups are large then this reduces the number of rows processed by the
join, at the cost that we might spend time computing the aggregate for
some row sets that will just be discarded by the join. So now we considerSELECT agg(a.x) FROM a, b WHERE a.y = b.z GROUP BY ?;
and ask what properties the grouping column(s) "?" must have to make this
work. I believe we can say that it's OK to push down through a join if
its join clauses are all of the form "a.y = b.z", where either a.y or b.z
is listed as a GROUP BY column *and* the join operator is equality for
the btree opfamily specified by the SortGroupClause.
I think the requirement for the join clauses to be of the form "a.y = b.z"
would only be valid if we wanted to avoid the 2-stage aggregation. For
example, in this query
SELECT agg(a.x) FROM a, b WHERE a.y = b.z GROUP BY b.z
uniqueness of "b.z" ensures that groups resulting from the pushed-down
aggregation (SELECT a.y, agg(a.x) FROM a GROUP BY a.y) do not get duplicated
by the join. However if there's a consensus that the 2-stage aggregation will
be used (because avoidance of it would limit usefulness of the feature too
much), I think we can accept generic join clauses. (Note that the patch does
not try to push aggregate down below nullable side of outer join - handling of
generated NULL values by join clauses would be another problem.)
The concerns Robert had about equality semantics are certainly vital, but I
believe that the logic goes through correctly as long as the grouping
equality operator and the join equality operator belong to the same
opfamily.ok, generic approach like this is always better. I just could not devise it,
nor can I prove that your theory is wrong.Robert postulated cases like "citext = text", but that doesn't apply
here because no cross-type citext = text operator could be part of a
well-behaved opfamily.The reason I can see now is that the GROUP BY operator (opfamily) essentially
has the grouping column on both sides, so it does not match the cross-type
operator.
Even if my statement that the join clause can be of other kind than "<leftarg>
= <rightarg>" was right, it might still be worth insisting on "<leftargt> <op>
<rightarg>" where <op> is of the same operator family as the grouping
operator?
Also, Tomas complained in the earlier thread that he didn't think
grouping on the join column was a very common use-case in the first
place. I share that concern, but I think we could extend the logic
to the case that Tomas posited as being useful:SELECT agg(a.x) FROM a, b WHERE a.y = b.id GROUP BY b.z;
where the join column b.id is unique. If we group on a.y (using semantics
compatible with the join operator and the uniqueness constraint), then all
"a" rows in a given group will join to exactly one "b" row that
necessarily has exactly one grouping value, so this group can safely be
aggregated together. We might need to combine it post-join with other "b"
rows that have equal "z" values, but we can do that as long as we're okay
with partial aggregation. I think this example shows why the idea is far
more powerful with partial aggregation than without.Good idea. I'll try to incorporate it in the next version.
The previous version of the patch (before I removed the partial aggregation)
could actually handle this case, so I restored it. It just does not check the
uniqueness of the b.id column because: the final aggregation will eliminate
the duplicated groups anyway.
The patch views the problem from this perspective: "a.y" is needed by join
clause, and the only way to retrieve it from the partially aggregated relation
"a" is to use it as a grouping column. Such an "artificial" grouping
expression can only be used if the plan contains the final aggregate node,
which uses only the original GROUP BY clause.
I also have no use for the stuff depending on bitwise equality rather than
the user-visible operators; that's just a hack, and an ugly one.The purpose of this was to avoid aggregation push-down for data types like
numeric, see the exampleSELECT one.a
FROM one, two
WHERE one.a = two.a AND scale(two.a) > 1
GROUP BY 1in [2]. The problem is that aggregation can make some information lost, which
is needed above by WHERE/ON clauses. I appreciate any suggestions how to
address this problem.
The new patch version does not contain this hack, but the problem is still
open. Alternatively I think of a boolean flag to be added to pg_opfamily or
pg_opclass catalog, saying whether equality also means "bitwise equality". If
the value is false (the default) for the GROUP BY operator, no aggregate
push-down can take place.
I think the right representation is to create UPPERREL_GROUP_AGG
RelOptInfos whose relids show the set of baserels joined before
aggregating. Currently there's only one of those and its relids is equal
to all_baserels (or should be, anyway). This patch would add instances of
UPPERREL_GROUP_AGG RelOptInfos with singleton relids, and later we might
put in the ability to consider aggregation across other relid subsets, and
in any case we'd run join planning on those level-one rels and create new
UPPERREL_GROUP_AGG RelOptInfos that represent the intermediate join steps
leading up to the final join. The correct indexing structure for this
collection of RelOptInfos is certainly not simple_rel_array; it should
look more like the way we index the various regular join rels that we
consider.
(Maybe an appropriate preliminary patch is to refactor the support code
associated with join_rel_list + join_rel_hash so that we can treat those
fields as one instance of a structure we'll replicate later.)
Do you mean that, instead of a single RelOptInfo, join_rel_list would contain
a structure that points to two RelOptInfo structures, where one is the "plain"
one and the other is the "grouped" one?
Eventually I thought that you propose a separate instance of join_rel_list
(e.g. join_rel_list_grouped) for the 2nd run of the joining process, however
that would mean that some functions need to receive RelOptInfos as arguments,
and *also* search in the the join_rel_list_grouped when joining grouped
relation to non-grouped one. So I decided to wrap both grouped and non-grouped
relation into a new structure (RelOptInfoSet). Thus all the relations to be
joined are passed as function arguments. make_join_rel() shows best how
instances of the new structure is used.
[1] /messages/by-id/113e9594-7c08-0f1f-ad15-41773b56a86b@iki.fi
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com
Attachments:
v09-001-Export_estimate_hashagg_tablesize.patchtext/x-diffDownload
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index b849ae03b8..6f3a4a063c 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -146,9 +146,6 @@ static double get_number_of_groups(PlannerInfo *root,
double path_rows,
grouping_sets_data *gd,
List *target_list);
-static Size estimate_hashagg_tablesize(Path *path,
- const AggClauseCosts *agg_costs,
- double dNumGroups);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *target,
@@ -3634,40 +3631,6 @@ get_number_of_groups(PlannerInfo *root,
}
/*
- * estimate_hashagg_tablesize
- * estimate the number of bytes that a hash aggregate hashtable will
- * require based on the agg_costs, path width and dNumGroups.
- *
- * XXX this may be over-estimating the size now that hashagg knows to omit
- * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
- * grouping columns not in the hashed set are counted here even though hashagg
- * won't store them. Is this a problem?
- */
-static Size
-estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
- double dNumGroups)
-{
- Size hashentrysize;
-
- /* Estimate per-hash-entry space at tuple width... */
- hashentrysize = MAXALIGN(path->pathtarget->width) +
- MAXALIGN(SizeofMinimalTupleHeader);
-
- /* plus space for pass-by-ref transition values... */
- hashentrysize += agg_costs->transitionSpace;
- /* plus the per-hash-entry overhead */
- hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
-
- /*
- * Note that this disregards the effect of fill-factor and growth policy
- * of the hash-table. That's probably ok, given default the default
- * fill-factor is relatively high. It'd be hard to meaningfully factor in
- * "double-in-size" growth policies here.
- */
- return hashentrysize * dNumGroups;
-}
-
-/*
* create_grouping_paths
*
* Build a new upperrel containing Paths for grouping and/or aggregation.
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index e8f51d2d0d..be059f0de0 100644
--- a/src/backend/utils/adt/selfuncs.c
+++ b/src/backend/utils/adt/selfuncs.c
@@ -114,6 +114,7 @@
#include "catalog/pg_statistic_ext.h"
#include "catalog/pg_type.h"
#include "executor/executor.h"
+#include "executor/nodeAgg.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
@@ -3902,6 +3903,39 @@ estimate_hash_bucket_stats(PlannerInfo *root, Node *hashkey, double nbuckets,
ReleaseVariableStats(vardata);
}
+/*
+ * estimate_hashagg_tablesize
+ * estimate the number of bytes that a hash aggregate hashtable will
+ * require based on the agg_costs, path width and dNumGroups.
+ *
+ * XXX this may be over-estimating the size now that hashagg knows to omit
+ * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
+ * grouping columns not in the hashed set are counted here even though hashagg
+ * won't store them. Is this a problem?
+ */
+Size
+estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
+ double dNumGroups)
+{
+ Size hashentrysize;
+
+ /* Estimate per-hash-entry space at tuple width... */
+ hashentrysize = MAXALIGN(path->pathtarget->width) +
+ MAXALIGN(SizeofMinimalTupleHeader);
+
+ /* plus space for pass-by-ref transition values... */
+ hashentrysize += agg_costs->transitionSpace;
+ /* plus the per-hash-entry overhead */
+ hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
+
+ /*
+ * Note that this disregards the effect of fill-factor and growth policy
+ * of the hash-table. That's probably ok, given default the default
+ * fill-factor is relatively high. It'd be hard to meaningfully factor in
+ * "double-in-size" growth policies here.
+ */
+ return hashentrysize * dNumGroups;
+}
/*-------------------------------------------------------------------------
*
diff --git a/src/include/utils/selfuncs.h b/src/include/utils/selfuncs.h
index 5cc4cf15e2..e65ab13b1a 100644
--- a/src/include/utils/selfuncs.h
+++ b/src/include/utils/selfuncs.h
@@ -213,6 +213,9 @@ extern void estimate_hash_bucket_stats(PlannerInfo *root,
Node *hashkey, double nbuckets,
Selectivity *mcv_freq,
Selectivity *bucketsize_frac);
+extern Size estimate_hashagg_tablesize(Path *path,
+ const AggClauseCosts *agg_costs,
+ double dNumGroups);
extern List *deconstruct_indexquals(IndexPath *path);
extern void genericcostestimate(PlannerInfo *root, IndexPath *path,
v09-002-Introduce_make_join_rel_common.patchtext/x-diffDownload
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 38eeb23d81..0caf67a916 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -37,6 +37,7 @@ static bool is_dummy_rel(RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
+static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist);
@@ -651,21 +652,12 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
return true;
}
-
/*
- * make_join_rel
- * Find or create a join RelOptInfo that represents the join of
- * the two given rels, and add to it path information for paths
- * created with the two rels as outer and inner rel.
- * (The join rel may already contain paths generated from other
- * pairs of rels that add up to the same set of base rels.)
- *
- * NB: will return NULL if attempted join is not valid. This can happen
- * when working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
+ * make_join_rel_common
+ * The workhorse of make_join_rel().
*/
-RelOptInfo *
-make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+static RelOptInfo *
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -748,6 +740,24 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
}
/*
+ * make_join_rel
+ * Find or create a join RelOptInfo that represents the join of
+ * the two given rels, and add to it path information for paths
+ * created with the two rels as outer and inner rel.
+ * (The join rel may already contain paths generated from other
+ * pairs of rels that add up to the same set of base rels.)
+ *
+ * NB: will return NULL if attempted join is not valid. This can happen
+ * when working with outer joins, or with IN or EXISTS clauses that have been
+ * turned into joins.
+ */
+RelOptInfo *
+make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+{
+ return make_join_rel_common(root, rel1, rel2);
+}
+
+/*
* populate_joinrel_with_paths
* Add paths to the given joinrel for given pair of joining relations. The
* SpecialJoinInfo provides details about the join and the restrictlist
v09-003-Introduce_estimate_join_rows.patchtext/x-diffDownload
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 99c5ad9b4a..353dc116f4 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -2291,6 +2291,20 @@ cost_group(Path *path, PlannerInfo *root,
}
/*
+ * estimate_join_rows
+ * Set rows of a join path according to its parent relation or according
+ * to parameters.
+ */
+static void
+estimate_join_rows(PlannerInfo *root, Path *path)
+{
+ if (path->param_info)
+ path->rows = path->param_info->ppi_rows;
+ else
+ path->rows = path->parent->rows;
+}
+
+/*
* initial_cost_nestloop
* Preliminary estimate of the cost of a nestloop join path.
*
@@ -2411,10 +2425,7 @@ final_cost_nestloop(PlannerInfo *root, NestPath *path,
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
- if (path->path.param_info)
- path->path.rows = path->path.param_info->ppi_rows;
- else
- path->path.rows = path->path.parent->rows;
+ estimate_join_rows(root, (Path *) path);
/* For partial paths, scale row estimate. */
if (path->path.parallel_workers > 0)
@@ -2857,10 +2868,7 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
- if (path->jpath.path.param_info)
- path->jpath.path.rows = path->jpath.path.param_info->ppi_rows;
- else
- path->jpath.path.rows = path->jpath.path.parent->rows;
+ estimate_join_rows(root, (Path *) path);
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
@@ -3287,10 +3295,7 @@ final_cost_hashjoin(PlannerInfo *root, HashPath *path,
ListCell *hcl;
/* Mark the path with the correct row estimate */
- if (path->jpath.path.param_info)
- path->jpath.path.rows = path->jpath.path.param_info->ppi_rows;
- else
- path->jpath.path.rows = path->jpath.path.parent->rows;
+ estimate_join_rows(root, (Path *) path);
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
v09-004-Agg_pushdown_basic.patchtext/x-diffDownload
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 006a3d1772..d46885de93 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2198,8 +2198,8 @@ _copyOnConflictExpr(const OnConflictExpr *from)
/* ****************************************************************
* relation.h copy functions
*
- * We don't support copying RelOptInfo, IndexOptInfo, or Path nodes.
- * There are some subsidiary structs that are useful to copy, though.
+ * We don't support copying RelOptInfo, IndexOptInfo, RelAggInfo or Path
+ * nodes. There are some subsidiary structs that are useful to copy, though.
* ****************************************************************
*/
@@ -2340,6 +2340,20 @@ _copyPlaceHolderInfo(const PlaceHolderInfo *from)
return newnode;
}
+static GroupedVarInfo *
+_copyGroupedVarInfo(const GroupedVarInfo *from)
+{
+ GroupedVarInfo *newnode = makeNode(GroupedVarInfo);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gv_eval_at);
+ COPY_SCALAR_FIELD(derived);
+
+ return newnode;
+}
+
/* ****************************************************************
* parsenodes.h copy functions
* ****************************************************************
@@ -5107,6 +5121,9 @@ copyObjectImpl(const void *from)
case T_PlaceHolderInfo:
retval = _copyPlaceHolderInfo(from);
break;
+ case T_GroupedVarInfo:
+ retval = _copyGroupedVarInfo(from);
+ break;
/*
* VALUE NODES
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 0fde876c77..8c91355931 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2198,6 +2198,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(append_rel_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
@@ -2205,6 +2206,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(distinct_pathkeys);
WRITE_NODE_FIELD(sort_pathkeys);
WRITE_NODE_FIELD(processed_tlist);
+ WRITE_INT_FIELD(max_sortgroupref);
WRITE_NODE_FIELD(minmax_aggs);
WRITE_FLOAT_FIELD(total_table_pages, "%.0f");
WRITE_FLOAT_FIELD(tuple_fraction, "%.4f");
@@ -2416,6 +2418,22 @@ _outParamPathInfo(StringInfo str, const ParamPathInfo *node)
WRITE_NODE_FIELD(ppi_clauses);
}
+/*
+ * Note we do NOT print plain_rel, else we'd be in infinite recursion.
+ */
+static void
+_outRelAggInfo(StringInfo str, const RelAggInfo *node)
+{
+ WRITE_NODE_TYPE("RELAGGINFO");
+
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(input);
+ WRITE_FLOAT_FIELD(input_rows, "%.0f");
+ WRITE_NODE_FIELD(group_clauses);
+ WRITE_NODE_FIELD(group_exprs);
+ WRITE_NODE_FIELD(agg_exprs);
+}
+
static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
@@ -2504,6 +2522,18 @@ _outPlaceHolderInfo(StringInfo str, const PlaceHolderInfo *node)
}
static void
+_outGroupedVarInfo(StringInfo str, const GroupedVarInfo *node)
+{
+ WRITE_NODE_TYPE("GROUPEDVARINFO");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_BITMAPSET_FIELD(gv_eval_at);
+ WRITE_BOOL_FIELD(derived);
+}
+
+static void
_outMinMaxAggInfo(StringInfo str, const MinMaxAggInfo *node)
{
WRITE_NODE_TYPE("MINMAXAGGINFO");
@@ -4039,6 +4069,9 @@ outNode(StringInfo str, const void *obj)
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_RelAggInfo:
+ _outRelAggInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
@@ -4054,6 +4087,9 @@ outNode(StringInfo str, const void *obj)
case T_PlaceHolderInfo:
_outPlaceHolderInfo(str, obj);
break;
+ case T_GroupedVarInfo:
+ _outGroupedVarInfo(str, obj);
+ break;
case T_MinMaxAggInfo:
_outMinMaxAggInfo(str, obj);
break;
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
index 9c852a15ef..52e31404b0 100644
--- a/src/backend/optimizer/README
+++ b/src/backend/optimizer/README
@@ -1118,3 +1118,90 @@ breaking down aggregation or grouping over a partitioned relation into
aggregation or grouping over its partitions is called partitionwise
aggregation. Especially when the partition keys match the GROUP BY clause,
this can be significantly faster than the regular method.
+
+Aggregate push-down
+-------------------
+
+The obvious way to evaluate aggregates is to evaluate the FROM clause of the
+SQL query (this is what query_planner does) and use the resuing paths as the
+input of Agg node. However, if the groups are large enough, it may be more
+efficient to apply the partial aggregation to the output of base relation
+scan, and finalize it when we have all relations of the query joined:
+
+ EXPLAIN
+ SELECT a.i, avg(b.u)
+ FROM a JOIN b ON b.j = a.i
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+ -> Index Only Scan using a_pkey on a
+ Index Cond: (i = b.j)
+
+Thus the join above the partial aggregate node receives fewer input rows, and
+so the number of outer-to-inner pairs of tuples to be checked can be
+significantly lower, which can in turn lead to considerably lower join cost.
+
+Note that there's often no GROUP BY expression to be used for the partial
+aggregation, so we use equivalence classes to derive grouping expression: in
+the example above, the grouping key "b.j" was derived from "a.i".
+
+Furthermore, extra grouping columns can be added to the partial Agg node if a
+join clause above that node references a column which is not in the query
+GROUP BY clause and which could not be derived using equivalence class.
+
+ EXPLAIN
+ SELECT a.i, avg(b.u)
+ FROM a JOIN b ON b.j = a.x
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (a.x = b.j)
+ -> Seq Scan on a
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+
+Here the partial aggregate uses "b.j" as grouping column although it's not in
+the same equivalence class as "a.i". Note that no column of the {a.x, b.j}
+equivalence class is used as a key for the final aggregation.
+
+Besides base relation, the aggregation can also be pushed down to join:
+
+ EXPLAIN
+ SELECT a.i, avg(b.u + c.v)
+ FROM a JOIN b ON b.j = a.i
+ JOIN c ON c.k = a.i
+ WHERE b.j = c.k GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (b.j = a.i)
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Hash Join
+ Hash Cond: (b.j = c.k)
+ -> Seq Scan on b
+ -> Hash
+ -> Seq Scan on c
+ -> Hash
+ -> Seq Scan on a
+
+Whether the Agg node is created out of base relation or out of join, it's
+added to a separate RelOptInfo that we call "grouped relation". Grouped
+relation can be joined to a non-grouped relation, which results in a grouped
+relation too. Join of two grouped relations does not seem to be very useful
+and is currently not supported.
+
+If query_planner produces a grouped relation that contains valid paths, these
+are simply added to the UPPERREL_PARTIAL_GROUP_AGG relation. Further
+processing of these paths then does not differ from processing of other
+partially grouped paths.
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index e07bab831e..4beac45fdf 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -182,13 +182,15 @@ gimme_tree(PlannerInfo *root, Gene *tour, int num_gene)
for (rel_count = 0; rel_count < num_gene; rel_count++)
{
int cur_rel_index;
+ RelOptInfoSet *cur_relset;
RelOptInfo *cur_rel;
Clump *cur_clump;
/* Get the next input relation */
cur_rel_index = (int) tour[rel_count];
- cur_rel = (RelOptInfo *) list_nth(private->initial_rels,
- cur_rel_index - 1);
+ cur_relset = (RelOptInfoSet *) list_nth(private->initial_rels,
+ cur_rel_index - 1);
+ cur_rel = cur_relset->rel_plain;
/* Make it into a single-rel clump */
cur_clump = (Clump *) palloc(sizeof(Clump));
@@ -251,16 +253,24 @@ merge_clump(PlannerInfo *root, List *clumps, Clump *new_clump, int num_gene,
desirable_join(root, old_clump->joinrel, new_clump->joinrel))
{
RelOptInfo *joinrel;
+ RelOptInfoSet *oldset,
+ *newset;
/*
* Construct a RelOptInfo representing the join of these two input
* relations. Note that we expect the joinrel not to exist in
* root->join_rel_list yet, and so the paths constructed for it
* will only include the ones we want.
+ *
+ * TODO Consider using make_join_rel_common() (possibly renamed)
+ * here instead of wrapping the joinrels into RelOptInfoSet.
*/
- joinrel = make_join_rel(root,
- old_clump->joinrel,
- new_clump->joinrel);
+ oldset = makeNode(RelOptInfoSet);
+ oldset->rel_plain = old_clump->joinrel;
+ newset = makeNode(RelOptInfoSet);
+ newset->rel_plain = new_clump->joinrel;
+
+ joinrel = make_join_rel(root, oldset, newset);
/* Keep searching if join order is not valid */
if (joinrel)
diff --git a/src/backend/optimizer/geqo/geqo_main.c b/src/backend/optimizer/geqo/geqo_main.c
index dc51829dec..b4a4f4ff27 100644
--- a/src/backend/optimizer/geqo/geqo_main.c
+++ b/src/backend/optimizer/geqo/geqo_main.c
@@ -63,7 +63,7 @@ static int gimme_number_generations(int pool_size);
* similar to a constrained Traveling Salesman Problem (TSP)
*/
-RelOptInfo *
+RelOptInfoSet *
geqo(PlannerInfo *root, int number_of_rels, List *initial_rels)
{
GeqoPrivateData private;
@@ -74,6 +74,7 @@ geqo(PlannerInfo *root, int number_of_rels, List *initial_rels)
Pool *pool;
int pool_size,
number_generations;
+ RelOptInfoSet *result;
#ifdef GEQO_DEBUG
int status_interval;
@@ -296,7 +297,9 @@ geqo(PlannerInfo *root, int number_of_rels, List *initial_rels)
/* ... clear root pointer to our private storage */
root->join_search_private = NULL;
- return best_rel;
+ result = makeNode(RelOptInfoSet);
+ result->rel_plain = best_rel;
+ return result;
}
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index bc389b52e4..54bd229daf 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -59,6 +59,7 @@ typedef struct pushdown_safety_info
/* These parameters are set by GUC */
bool enable_geqo = false; /* just in case GUC doesn't set it */
+bool enable_agg_pushdown;
int geqo_threshold;
int min_parallel_table_scan_size;
int min_parallel_index_scan_size;
@@ -74,16 +75,18 @@ static void set_base_rel_consider_startup(PlannerInfo *root);
static void set_base_rel_sizes(PlannerInfo *root);
static void set_base_rel_pathlists(PlannerInfo *root);
static void set_rel_size(PlannerInfo *root, RelOptInfo *rel,
- Index rti, RangeTblEntry *rte);
+ Index rti, RangeTblEntry *rte, RelAggInfo *agg_info);
static void set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
- Index rti, RangeTblEntry *rte);
+ Index rti, RangeTblEntry *rte,
+ RelOptInfo *rel_grouped, RelAggInfo *agg_info);
static void set_plain_rel_size(PlannerInfo *root, RelOptInfo *rel,
- RangeTblEntry *rte);
+ RangeTblEntry *rte, RelAggInfo *agg_info);
static void create_plain_partial_paths(PlannerInfo *root, RelOptInfo *rel);
static void set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
- RangeTblEntry *rte);
+ RangeTblEntry *rte, RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static void set_tablesample_rel_size(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
@@ -119,7 +122,8 @@ static void set_namedtuplestore_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
-static RelOptInfo *make_rel_from_joinlist(PlannerInfo *root, List *joinlist);
+static RelOptInfoSet *make_rel_from_joinlist(PlannerInfo *root,
+ List *joinlist);
static bool subquery_is_pushdown_safe(Query *subquery, Query *topquery,
pushdown_safety_info *safetyInfo);
static bool recurse_pushdown_safe(Node *setOp, Query *topquery,
@@ -143,10 +147,10 @@ static void remove_unused_subquery_outputs(Query *subquery, RelOptInfo *rel);
* Finds all possible access paths for executing a query, returning a
* single rel that represents the join of all base rels in the query.
*/
-RelOptInfo *
+RelOptInfoSet *
make_one_rel(PlannerInfo *root, List *joinlist)
{
- RelOptInfo *rel;
+ RelOptInfoSet *rel;
Index rti;
double total_pages;
@@ -224,7 +228,7 @@ make_one_rel(PlannerInfo *root, List *joinlist)
/*
* The result should join all and only the query's base rels.
*/
- Assert(bms_equal(rel->relids, root->all_baserels));
+ Assert(bms_equal(rel->rel_plain->relids, root->all_baserels));
return rel;
}
@@ -315,7 +319,7 @@ set_base_rel_sizes(PlannerInfo *root)
if (root->glob->parallelModeOK)
set_rel_consider_parallel(root, rel, rte);
- set_rel_size(root, rel, rti, rte);
+ set_rel_size(root, rel, rti, rte, NULL);
}
}
@@ -344,18 +348,29 @@ set_base_rel_pathlists(PlannerInfo *root)
if (rel->reloptkind != RELOPT_BASEREL)
continue;
- set_rel_pathlist(root, rel, rti, root->simple_rte_array[rti]);
+ set_rel_pathlist(root, rel, rti, root->simple_rte_array[rti],
+ NULL, NULL);
}
}
/*
* set_rel_size
* Set size estimates for a base relation
+ *
+ * If "agg_info" is passed, it means that "rel" is a grouped relation for
+ * which "agg_info" provides the information needed for size estimate.
*/
static void
set_rel_size(PlannerInfo *root, RelOptInfo *rel,
- Index rti, RangeTblEntry *rte)
+ Index rti, RangeTblEntry *rte, RelAggInfo *agg_info)
{
+ bool grouped = agg_info != NULL;
+
+ /*
+ * Aggregate push-down is currently used only for relations.
+ */
+ Assert(!grouped || rte->rtekind == RTE_RELATION);
+
if (rel->reloptkind == RELOPT_BASEREL &&
relation_excluded_by_constraints(root, rel, rte))
{
@@ -374,7 +389,13 @@ set_rel_size(PlannerInfo *root, RelOptInfo *rel,
}
else if (rte->inh)
{
- /* It's an "append relation", process accordingly */
+ /*
+ * It's an "append relation", process accordingly.
+ *
+ * The aggregate push-down feature currently does not support grouped
+ * append relation.
+ */
+ Assert(!grouped);
set_append_rel_size(root, rel, rti, rte);
}
else
@@ -384,7 +405,12 @@ set_rel_size(PlannerInfo *root, RelOptInfo *rel,
case RTE_RELATION:
if (rte->relkind == RELKIND_FOREIGN_TABLE)
{
- /* Foreign table */
+ /*
+ * Foreign table
+ *
+ * Grouped foreign table is not supported.
+ */
+ Assert(!grouped);
set_foreign_size(root, rel, rte);
}
else if (rte->relkind == RELKIND_PARTITIONED_TABLE)
@@ -392,18 +418,27 @@ set_rel_size(PlannerInfo *root, RelOptInfo *rel,
/*
* A partitioned table without any partitions is marked as
* a dummy rel.
+ *
+ * Grouped partitioned table is not supported.
*/
+ Assert(!grouped);
+
set_dummy_rel_pathlist(rel);
}
else if (rte->tablesample != NULL)
{
- /* Sampled relation */
+ /*
+ * Sampled relation
+ *
+ * Grouped tablesample rel is not supported.
+ */
+ Assert(!grouped);
set_tablesample_rel_size(root, rel, rte);
}
else
{
/* Plain relation */
- set_plain_rel_size(root, rel, rte);
+ set_plain_rel_size(root, rel, rte, agg_info);
}
break;
case RTE_SUBQUERY:
@@ -454,18 +489,49 @@ set_rel_size(PlannerInfo *root, RelOptInfo *rel,
/*
* set_rel_pathlist
* Build access paths for a base relation
+ *
+ * If "rel_grouped" is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of "rel". "rel_agg_input" describes the
+ * aggregation input. "agg_info" must be passed in such a case too.
*/
static void
set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
- Index rti, RangeTblEntry *rte)
+ Index rti, RangeTblEntry *rte, RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
+ bool grouped = rel_grouped != NULL;
+
+ /*
+ * Aggregate push-down is currently implemented for RTE_RELATION only.
+ */
+ Assert(!grouped || rte->rtekind == RTE_RELATION);
+
+ if (grouped)
+ {
+ /*
+ * Do not apply AggPath if this is the only relation of the query:
+ * create_grouping_paths() will do so anyway.
+ */
+ if (bms_equal(rel->relids, root->all_baserels))
+ {
+ set_dummy_rel_pathlist(rel_grouped);
+ return;
+ }
+ }
+
if (IS_DUMMY_REL(rel))
{
/* We already proved the relation empty, so nothing more to do */
}
else if (rte->inh)
{
- /* It's an "append relation", process accordingly */
+ /*
+ * It's an "append relation", process accordingly.
+ *
+ * The aggregate push-down feature currently does not support append
+ * relation.
+ */
+ Assert(!grouped);
set_append_rel_pathlist(root, rel, rti, rte);
}
else
@@ -475,18 +541,29 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
case RTE_RELATION:
if (rte->relkind == RELKIND_FOREIGN_TABLE)
{
- /* Foreign table */
+ /*
+ * Foreign table
+ *
+ * Aggregate push-down not supported.
+ */
+ Assert(!grouped);
set_foreign_pathlist(root, rel, rte);
}
else if (rte->tablesample != NULL)
{
- /* Sampled relation */
+ /*
+ * Sampled relation.
+ *
+ * Aggregate push-down not supported.
+ */
+ Assert(!grouped);
set_tablesample_rel_pathlist(root, rel, rte);
}
else
{
/* Plain relation */
- set_plain_rel_pathlist(root, rel, rte);
+ set_plain_rel_pathlist(root, rel, rte, rel_grouped,
+ agg_info);
}
break;
case RTE_SUBQUERY:
@@ -528,9 +605,12 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* Also, if this is the topmost scan/join rel (that is, the only baserel),
* we postpone this until the final scan/join targelist is available (see
* grouping_planner).
+ *
+ * Note on aggregation push-down: parallel paths are not supported so far.
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- bms_membership(root->all_baserels) != BMS_SINGLETON)
+ bms_membership(root->all_baserels) != BMS_SINGLETON &&
+ !grouped)
generate_gather_paths(root, rel, false);
/*
@@ -539,10 +619,24 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* add_path(), or delete or modify paths added by the core code.
*/
if (set_rel_pathlist_hook)
- (*set_rel_pathlist_hook) (root, rel, rti, rte);
+ (*set_rel_pathlist_hook) (root, rel, rti, rte, rel_grouped,
+ agg_info);
+
+ /*
+ * The grouped relation is not guaranteed to have any paths. If the
+ * pathlist is empty, there's no point in calling set_cheapest().
+ */
+ if (rel_grouped && rel_grouped->pathlist == NIL)
+ {
+ set_dummy_rel_pathlist(rel_grouped);
+ return;
+ }
/* Now find the cheapest of the paths for this rel */
- set_cheapest(rel);
+ if (!grouped)
+ set_cheapest(rel);
+ else
+ set_cheapest(rel_grouped);
#ifdef OPTIMIZER_DEBUG
debug_print_rel(root, rel);
@@ -552,9 +646,13 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
/*
* set_plain_rel_size
* Set size estimates for a plain relation (no subquery, no inheritance)
+ *
+ * If "agg_info" is passed, it means that "rel" is a grouped relation for
+ * which "agg_info" provides the information needed for size estimate.
*/
static void
-set_plain_rel_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
+set_plain_rel_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte,
+ RelAggInfo *agg_info)
{
/*
* Test any partial indexes of rel for applicability. We must do this
@@ -563,7 +661,7 @@ set_plain_rel_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
check_index_predicates(root, rel);
/* Mark rel with estimated output rows, width, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, agg_info);
}
/*
@@ -740,11 +838,19 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
/*
* set_plain_rel_pathlist
* Build access paths for a plain relation (no subquery, no inheritance)
+ *
+ * If "rel_grouped" is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of "rel". "rel_agg_input" describes the
+ * aggregation input. "agg_info" must be passed in such a case too.
*/
static void
-set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
+set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
+ RangeTblEntry *rte, RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
Relids required_outer;
+ Path *seq_path;
+ bool grouped = rel_grouped != NULL;
/*
* We don't support pushing join clauses into the quals of a seqscan, but
@@ -753,18 +859,38 @@ set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
*/
required_outer = rel->lateral_relids;
- /* Consider sequential scan */
- add_path(rel, create_seqscan_path(root, rel, required_outer, 0));
+ /* Consider sequential scan. */
+ seq_path = create_seqscan_path(root, rel, required_outer, 0, rel_grouped,
+ agg_info);
+
+ if (!grouped)
+ add_path(rel, seq_path);
+
+ /*
+ * It's probably not good idea to repeat hashed aggregation with different
+ * parameters, so check if there are no parameters.
+ */
+ else if (required_outer == NULL)
+ {
+ /*
+ * Only AGG_HASHED is suitable here as it does not expect the input
+ * set to be sorted.
+ */
+ add_grouped_path(root, rel_grouped, seq_path, AGG_HASHED, agg_info);
+ }
/* If appropriate, consider parallel sequential scan */
- if (rel->consider_parallel && required_outer == NULL)
+ if (rel->consider_parallel && required_outer == NULL && !grouped)
create_plain_partial_paths(root, rel);
- /* Consider index scans */
- create_index_paths(root, rel);
+ /*
+ * Consider index scans, possibly including the grouped paths.
+ */
+ create_index_paths(root, rel, rel_grouped, agg_info);
/* Consider TID scans */
- create_tidscan_paths(root, rel);
+ if (!grouped)
+ create_tidscan_paths(root, rel);
}
/*
@@ -776,15 +902,55 @@ create_plain_partial_paths(PlannerInfo *root, RelOptInfo *rel)
{
int parallel_workers;
- parallel_workers = compute_parallel_worker(rel, rel->pages, -1,
- max_parallel_workers_per_gather);
+ parallel_workers = compute_parallel_worker(rel, rel->pages, -1, max_parallel_workers_per_gather);
/* If any limit was set to zero, the user doesn't want a parallel scan. */
if (parallel_workers <= 0)
return;
/* Add an unordered partial path based on a parallel sequential scan. */
- add_partial_path(rel, create_seqscan_path(root, rel, NULL, parallel_workers));
+ add_partial_path(rel, create_seqscan_path(root, rel, NULL,
+ parallel_workers, NULL, NULL));
+}
+
+/*
+ * Apply aggregation to a subpath and add the AggPath to the pathlist.
+ *
+ * The return value tells whether the path was added to the pathlist.
+ *
+ * XXX Pass the plain rel and fetch the grouped from rel->grouped? How about
+ * subroutines then?
+ */
+bool
+add_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ AggStrategy aggstrategy, RelAggInfo *agg_info)
+{
+ Path *agg_path;
+
+ /*
+ * Repeated creation of hash table does not sound like a good idea. Caller
+ * should avoid asking us to do so.
+ */
+ Assert(subpath->param_info == NULL || aggstrategy != AGG_HASHED);
+
+ if (aggstrategy == AGG_HASHED)
+ agg_path = (Path *) create_agg_hashed_path(root, subpath,
+ agg_info);
+ else if (aggstrategy == AGG_SORTED)
+ agg_path = (Path *) create_agg_sorted_path(root, subpath,
+ agg_info);
+ else
+ elog(ERROR, "unexpected strategy %d", aggstrategy);
+
+ /* Add the grouped path to the list of grouped base paths. */
+ if (agg_path != NULL)
+ {
+ add_path(rel, (Path *) agg_path);
+
+ return true;
+ }
+
+ return false;
}
/*
@@ -824,7 +990,7 @@ set_tablesample_rel_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
rel->tuples = tuples;
/* Mark rel with estimated output rows, width, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -1222,7 +1388,7 @@ set_append_rel_size(PlannerInfo *root, RelOptInfo *rel,
/*
* Compute the child's size.
*/
- set_rel_size(root, childrel, childRTindex, childRTE);
+ set_rel_size(root, childrel, childRTindex, childRTE, NULL);
/*
* It is possible that constraint exclusion detected a contradiction
@@ -1371,7 +1537,7 @@ set_append_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
/*
* Compute the child's access paths.
*/
- set_rel_pathlist(root, childrel, childRTindex, childRTE);
+ set_rel_pathlist(root, childrel, childRTindex, childRTE, NULL, NULL);
/*
* If child is dummy, ignore it.
@@ -2631,7 +2797,7 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
* See comments for deconstruct_jointree() for definition of the joinlist
* data structure.
*/
-static RelOptInfo *
+static RelOptInfoSet *
make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
{
int levels_needed;
@@ -2657,13 +2823,35 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
foreach(jl, joinlist)
{
Node *jlnode = (Node *) lfirst(jl);
- RelOptInfo *thisrel;
+ RelOptInfoSet *thisrel;
if (IsA(jlnode, RangeTblRef))
{
- int varno = ((RangeTblRef *) jlnode)->rtindex;
+ RangeTblRef *rtref = castNode(RangeTblRef, jlnode);
+ int varno = rtref->rtindex;
+ RangeTblEntry *rte = root->simple_rte_array[varno];
+ RelOptInfo *rel_grouped;
+
+ thisrel = makeNode(RelOptInfoSet);
+ thisrel->rel_plain = find_base_rel(root, varno);
- thisrel = find_base_rel(root, varno);
+ /*
+ * Create the grouped counterpart of rel_plain if possible.
+ */
+ build_simple_grouped_rel(root, thisrel);
+ rel_grouped = thisrel->rel_grouped;
+
+ /*
+ * If succeeded, do what should already have been done for the
+ * plain relation.
+ */
+ if (rel_grouped)
+ {
+ set_rel_size(root, rel_grouped, varno, rte,
+ thisrel->agg_info);
+ set_rel_pathlist(root, thisrel->rel_plain, varno, rte,
+ rel_grouped, thisrel->agg_info);
+ }
}
else if (IsA(jlnode, List))
{
@@ -2685,7 +2873,7 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
/*
* Single joinlist node, so we're done.
*/
- return (RelOptInfo *) linitial(initial_rels);
+ return (RelOptInfoSet *) linitial(initial_rels);
}
else
{
@@ -2699,11 +2887,19 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
root->initial_rels = initial_rels;
if (join_search_hook)
- return (*join_search_hook) (root, levels_needed, initial_rels);
+ return (*join_search_hook) (root, levels_needed,
+ initial_rels);
else if (enable_geqo && levels_needed >= geqo_threshold)
+ {
+ /*
+ * TODO Teach GEQO about grouped relations. Don't forget that
+ * pathlist can be NIL before set_cheapest() gets called.
+ */
return geqo(root, levels_needed, initial_rels);
+ }
else
- return standard_join_search(root, levels_needed, initial_rels);
+ return standard_join_search(root, levels_needed,
+ initial_rels);
}
}
@@ -2736,11 +2932,11 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
* than one join-order search, you'll probably need to save and restore the
* original states of those data structures. See geqo_eval() for an example.
*/
-RelOptInfo *
+RelOptInfoSet *
standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
{
int lev;
- RelOptInfo *rel;
+ RelOptInfoSet *result;
/*
* This function cannot be invoked recursively within any one planning
@@ -2782,13 +2978,20 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
*
* After that, we're done creating paths for the joinrel, so run
* set_cheapest().
+ *
+ * Neither partitionwise join nor parallel processing is supported for
+ * grouped relations so far.
*/
foreach(lc, root->join_rel_level[lev])
{
- rel = (RelOptInfo *) lfirst(lc);
+ RelOptInfoSet *rel_set;
+ RelOptInfo *rel_plain;
+
+ rel_set = (RelOptInfoSet *) lfirst(lc);
+ rel_plain = rel_set->rel_plain;
/* Create paths for partitionwise joins. */
- generate_partitionwise_join_paths(root, rel);
+ generate_partitionwise_join_paths(root, rel_plain);
/*
* Except for the topmost scan/join rel, consider gathering
@@ -2796,10 +2999,33 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
* once we know the final targetlist (see grouping_planner).
*/
if (lev < levels_needed)
- generate_gather_paths(root, rel, false);
+ generate_gather_paths(root, rel_plain, false);
/* Find and save the cheapest paths for this rel */
- set_cheapest(rel);
+ set_cheapest(rel_plain);
+
+ if (rel_set->rel_grouped)
+ {
+ RelOptInfo *rel_grouped;
+
+ rel_grouped = rel_set->rel_grouped;
+
+ /* Partial paths not supported yet. */
+ Assert(rel_grouped->partial_pathlist == NIL);
+
+ if (rel_grouped->pathlist != NIL)
+ set_cheapest(rel_grouped);
+ else
+ {
+ /*
+ * When checking whether the grouped relation can supply
+ * any input paths to join, test for existence of the
+ * relation will be easier than a test of pathlist.
+ */
+ pfree(rel_grouped);
+ rel_set->rel_grouped = NULL;
+ }
+ }
#ifdef OPTIMIZER_DEBUG
debug_print_rel(root, rel);
@@ -2814,11 +3040,11 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
elog(ERROR, "failed to build any %d-way joins", levels_needed);
Assert(list_length(root->join_rel_level[levels_needed]) == 1);
- rel = (RelOptInfo *) linitial(root->join_rel_level[levels_needed]);
+ result = (RelOptInfoSet *) linitial(root->join_rel_level[levels_needed]);
root->join_rel_level = NULL;
- return rel;
+ return result;
}
/*****************************************************************************
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 353dc116f4..7bca52daad 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -88,6 +88,7 @@
#include "optimizer/plancat.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "optimizer/var.h"
#include "parser/parsetree.h"
#include "utils/lsyscache.h"
#include "utils/selfuncs.h"
@@ -2293,13 +2294,23 @@ cost_group(Path *path, PlannerInfo *root,
/*
* estimate_join_rows
* Set rows of a join path according to its parent relation or according
- * to parameters.
+ * to parameters. If agg_info is passed, the join path is grouped.
*/
static void
-estimate_join_rows(PlannerInfo *root, Path *path)
+estimate_join_rows(PlannerInfo *root, Path *path, RelAggInfo *agg_info)
{
if (path->param_info)
+ {
path->rows = path->param_info->ppi_rows;
+ if (agg_info)
+ {
+ double nrows;
+
+ nrows = estimate_num_groups(root, agg_info->group_exprs,
+ path->rows, NULL);
+ path->rows = clamp_row_est(nrows);
+ }
+ }
else
path->rows = path->parent->rows;
}
@@ -2425,7 +2436,7 @@ final_cost_nestloop(PlannerInfo *root, NestPath *path,
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
- estimate_join_rows(root, (Path *) path);
+ estimate_join_rows(root, (Path *) path, extra->agg_info);
/* For partial paths, scale row estimate. */
if (path->path.parallel_workers > 0)
@@ -2868,7 +2879,7 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
- estimate_join_rows(root, (Path *) path);
+ estimate_join_rows(root, (Path *) path, extra->agg_info);
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
@@ -3295,7 +3306,7 @@ final_cost_hashjoin(PlannerInfo *root, HashPath *path,
ListCell *hcl;
/* Mark the path with the correct row estimate */
- estimate_join_rows(root, (Path *) path);
+ estimate_join_rows(root, (Path *) path, extra->agg_info);
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
@@ -4300,27 +4311,59 @@ approx_tuple_count(PlannerInfo *root, JoinPath *path, List *quals)
* restriction clauses).
* width: the estimated average output tuple width in bytes.
* baserestrictcost: estimated cost of evaluating baserestrictinfo clauses.
+ *
+ * If "agg_info" is passed, it means that "rel" is a grouped relation for
+ * which "agg_info" provides the information needed for size estimate.
*/
void
-set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel)
+set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel,
+ RelAggInfo *agg_info)
{
double nrows;
+ bool grouped = agg_info != NULL;
/* Should only be applied to base relations */
Assert(rel->relid > 0);
- nrows = rel->tuples *
- clauselist_selectivity(root,
- rel->baserestrictinfo,
- 0,
- JOIN_INNER,
- NULL);
+ if (!grouped)
+ {
+ nrows = rel->tuples *
+ clauselist_selectivity(root,
+ rel->baserestrictinfo,
+ 0,
+ JOIN_INNER,
+ NULL);
+ rel->rows = clamp_row_est(nrows);
+ }
- rel->rows = clamp_row_est(nrows);
+ /*
+ * Only set the estimate for grouped base rel if aggregation can take
+ * place. (Aggregation is the only way to build grouped base relation.)
+ */
+ else if (!bms_equal(rel->relids, root->all_baserels))
+ {
+ /*
+ * Grouping essentially changes the number of rows.
+ */
+ nrows = estimate_num_groups(root,
+ agg_info->group_exprs,
+ agg_info->input_rows,
+ NULL);
+ rel->rows = clamp_row_est(nrows);
+ }
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
- set_rel_width(root, rel);
+ /*
+ * The grouped target should have the cost and width set immediately on
+ * creation, see create_rel_agg_info().
+ */
+ if (!grouped)
+ set_rel_width(root, rel);
+#ifdef USE_ASSERT_CHECKING
+ else
+ Assert(rel->reltarget->width > 0);
+#endif
}
/*
@@ -4884,7 +4927,7 @@ set_subquery_size_estimates(PlannerInfo *root, RelOptInfo *rel)
}
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -4922,7 +4965,7 @@ set_function_size_estimates(PlannerInfo *root, RelOptInfo *rel)
}
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -4944,7 +4987,7 @@ set_tablefunc_size_estimates(PlannerInfo *root, RelOptInfo *rel)
rel->tuples = 100;
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -4975,7 +5018,7 @@ set_values_size_estimates(PlannerInfo *root, RelOptInfo *rel)
rel->tuples = list_length(rte->values_lists);
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -5013,7 +5056,7 @@ set_cte_size_estimates(PlannerInfo *root, RelOptInfo *rel, double cte_rows)
}
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -5046,7 +5089,7 @@ set_namedtuplestore_size_estimates(PlannerInfo *root, RelOptInfo *rel)
rel->tuples = 1000;
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -5270,11 +5313,11 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
foreach(lc, target->exprs)
{
Node *node = (Node *) lfirst(lc);
+ int32 item_width;
if (IsA(node, Var))
{
Var *var = (Var *) node;
- int32 item_width;
/* We should not see any upper-level Vars here */
Assert(var->varlevelsup == 0);
@@ -5305,6 +5348,20 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
Assert(item_width > 0);
tuple_width += item_width;
}
+ else if (IsA(node, Aggref))
+ {
+ /*
+ * If the target is evaluated by AggPath, it'll care of cost
+ * estimate. If the target is above AggPath (typically target of a
+ * join relation that contains grouped relation), the cost of
+ * Aggref should not be accounted for again.
+ *
+ * On the other hand, width is always needed.
+ */
+ item_width = get_typavgwidth(exprType(node), exprTypmod(node));
+ Assert(item_width > 0);
+ tuple_width += item_width;
+ }
else
{
/*
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 6e134ae1d2..25f1b7fb44 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -65,7 +65,6 @@ static bool reconsider_outer_join_clause(PlannerInfo *root,
static bool reconsider_full_join_clause(PlannerInfo *root,
RestrictInfo *rinfo);
-
/*
* process_equivalence
* The given clause has a mergejoinable operator and can be applied without
@@ -2511,3 +2510,137 @@ is_redundant_derived_clause(RestrictInfo *rinfo, List *clauselist)
return false;
}
+
+/*
+ * translate_expression_to_rels
+ * If the appropriate equivalence classes exist, replace vars in
+ * gvi->gvexpr with vars whose varno is equal to relid. Return NULL if
+ * translation is not possible or needed.
+ *
+ * Note: Currently we only translate Var expressions. This is subject to
+ * change as the aggregate push-down feature gets enhanced.
+ */
+GroupedVarInfo *
+translate_expression_to_rels(PlannerInfo *root, GroupedVarInfo *gvi,
+ Index relid)
+{
+ Var *var;
+ ListCell *l1;
+ bool found_orig = false;
+ Var *var_translated = NULL;
+ GroupedVarInfo *result;
+
+ /* Can't do anything w/o equivalence classes. */
+ if (root->eq_classes == NIL)
+ return NULL;
+
+ var = castNode(Var, gvi->gvexpr);
+
+ /*
+ * Do we need to translate the var?
+ */
+ if (var->varno == relid)
+ return NULL;
+
+ /*
+ * Find the replacement var.
+ */
+ foreach(l1, root->eq_classes)
+ {
+ EquivalenceClass *ec = lfirst_node(EquivalenceClass, l1);
+ ListCell *l2;
+
+ /* TODO Check if any other EC kind should be ignored. */
+ if (ec->ec_has_volatile || ec->ec_below_outer_join || ec->ec_broken)
+ continue;
+
+ /* Single-element EC can hardly help in translations. */
+ if (list_length(ec->ec_members) == 1)
+ continue;
+
+ /*
+ * Collect all vars of this EC and their varnos.
+ *
+ * ec->ec_relids does not help because we're only interested in a
+ * subset of EC members.
+ */
+ foreach(l2, ec->ec_members)
+ {
+ EquivalenceMember *em = lfirst_node(EquivalenceMember, l2);
+ Var *ec_var;
+
+ /*
+ * The grouping expressions derived here are used to evaluate
+ * possibility to push aggregation down to RELOPT_BASEREL or
+ * RELOPT_JOINREL relations, and to construct reltargets for the
+ * grouped rels. We're not interested at the moment whether the
+ * relations do have children.
+ */
+ if (em->em_is_child)
+ continue;
+
+ if (!IsA(em->em_expr, Var))
+ continue;
+
+ ec_var = castNode(Var, em->em_expr);
+ if (equal(ec_var, var))
+ found_orig = true;
+ else if (ec_var->varno == relid)
+ var_translated = ec_var;
+
+ if (found_orig && var_translated)
+ {
+ /*
+ * The replacement Var must have the same data type, otherwise
+ * the values are not guaranteed to be grouped in the same way
+ * as values of the original Var.
+ */
+ if (ec_var->vartype != var->vartype)
+ return NULL;
+
+ break;
+ }
+ }
+
+ if (found_orig)
+ {
+ /*
+ * The same expression probably does not exist in multiple ECs.
+ */
+ if (var_translated == NULL)
+ {
+ /*
+ * Failed to translate the expression.
+ */
+ return NULL;
+ }
+ else
+ {
+ /* Success. */
+ break;
+ }
+ }
+ else
+ {
+ /*
+ * Vars of the requested relid can be in the next ECs too.
+ */
+ var_translated = NULL;
+ }
+ }
+
+ if (!found_orig)
+ return NULL;
+
+ result = makeNode(GroupedVarInfo);
+ memcpy(result, gvi, sizeof(GroupedVarInfo));
+
+ /*
+ * translate_expression_to_rels_mutator updates gv_eval_at.
+ */
+ result->gv_eval_at = bms_make_singleton(relid);
+ result->gvexpr = (Expr *) var_translated;
+ result->derived = true;
+
+ return result;
+}
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index f8e674c9c4..23e5ecf93e 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -33,6 +33,7 @@
#include "optimizer/predtest.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+#include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "utils/builtins.h"
#include "utils/bytea.h"
@@ -78,13 +79,14 @@ typedef struct
int indexcol; /* index column we want to match to */
} ec_member_matches_arg;
-
static void consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index,
IndexClauseSet *rclauseset,
IndexClauseSet *jclauseset,
IndexClauseSet *eclauseset,
- List **bitindexpaths);
+ List **bitindexpaths,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static void consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index,
IndexClauseSet *rclauseset,
@@ -93,7 +95,9 @@ static void consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
List **bitindexpaths,
List *indexjoinclauses,
int considered_clauses,
- List **considered_relids);
+ List **considered_relids,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static void get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index,
IndexClauseSet *rclauseset,
@@ -101,19 +105,24 @@ static void get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexClauseSet *eclauseset,
List **bitindexpaths,
Relids relids,
- List **considered_relids);
+ List **considered_relids,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static bool eclass_already_used(EquivalenceClass *parent_ec, Relids oldrelids,
List *indexjoinclauses);
static bool bms_equal_any(Relids relids, List *relids_list);
static void get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
- List **bitindexpaths);
+ List **bitindexpaths, RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static List *build_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
- bool *skip_lower_saop);
+ bool *skip_lower_saop,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static List *build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
List *clauses, List *other_clauses);
static List *generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
@@ -217,6 +226,10 @@ static Const *string_to_const(const char *str, Oid datatype);
*
* 'rel' is the relation for which we want to generate index paths
*
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
+ *
* Note: check_index_predicates() must have been run previously for this rel.
*
* Note: in cases involving LATERAL references in the relation's tlist, it's
@@ -229,7 +242,8 @@ static Const *string_to_const(const char *str, Oid datatype);
* as meaning "unparameterized so far as the indexquals are concerned".
*/
void
-create_index_paths(PlannerInfo *root, RelOptInfo *rel)
+create_index_paths(PlannerInfo *root, RelOptInfo *rel, RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
List *indexpaths;
List *bitindexpaths;
@@ -239,6 +253,7 @@ create_index_paths(PlannerInfo *root, RelOptInfo *rel)
IndexClauseSet jclauseset;
IndexClauseSet eclauseset;
ListCell *lc;
+ bool grouped = rel_grouped != NULL;
/* Skip the whole mess if no indexes */
if (rel->indexlist == NIL)
@@ -274,8 +289,8 @@ create_index_paths(PlannerInfo *root, RelOptInfo *rel)
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
- get_index_paths(root, rel, index, &rclauseset,
- &bitindexpaths);
+ get_index_paths(root, rel, index, &rclauseset, &bitindexpaths,
+ rel_grouped, agg_info);
/*
* Identify the join clauses that can match the index. For the moment
@@ -304,15 +319,24 @@ create_index_paths(PlannerInfo *root, RelOptInfo *rel)
&rclauseset,
&jclauseset,
&eclauseset,
- &bitjoinpaths);
+ &bitjoinpaths,
+ rel_grouped,
+ agg_info);
}
/*
+ * It does not seem too efficient to aggregate the individual paths and
+ * then AND them together.
+ */
+ if (grouped)
+ return;
+
+ /*
* Generate BitmapOrPaths for any suitable OR-clauses present in the
* restriction list. Add these to bitindexpaths.
*/
- indexpaths = generate_bitmap_or_paths(root, rel,
- rel->baserestrictinfo, NIL);
+ indexpaths = generate_bitmap_or_paths(root, rel, rel->baserestrictinfo,
+ NIL);
bitindexpaths = list_concat(bitindexpaths, indexpaths);
/*
@@ -434,6 +458,10 @@ create_index_paths(PlannerInfo *root, RelOptInfo *rel)
* 'jclauseset' is the collection of indexable simple join clauses
* 'eclauseset' is the collection of indexable clauses from EquivalenceClasses
* '*bitindexpaths' is the list to add bitmap paths to
+ *
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
*/
static void
consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
@@ -441,7 +469,9 @@ consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
IndexClauseSet *rclauseset,
IndexClauseSet *jclauseset,
IndexClauseSet *eclauseset,
- List **bitindexpaths)
+ List **bitindexpaths,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
int considered_clauses = 0;
List *considered_relids = NIL;
@@ -477,7 +507,9 @@ consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
bitindexpaths,
jclauseset->indexclauses[indexcol],
considered_clauses,
- &considered_relids);
+ &considered_relids,
+ rel_grouped,
+ agg_info);
/* Consider each applicable eclass join clause */
considered_clauses += list_length(eclauseset->indexclauses[indexcol]);
consider_index_join_outer_rels(root, rel, index,
@@ -485,7 +517,9 @@ consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
bitindexpaths,
eclauseset->indexclauses[indexcol],
considered_clauses,
- &considered_relids);
+ &considered_relids,
+ rel_grouped,
+ agg_info);
}
}
@@ -500,6 +534,10 @@ consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
* 'indexjoinclauses' is a list of RestrictInfos for join clauses
* 'considered_clauses' is the total number of clauses considered (so far)
* '*considered_relids' is a list of all relids sets already considered
+ *
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
*/
static void
consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
@@ -510,7 +548,9 @@ consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
List **bitindexpaths,
List *indexjoinclauses,
int considered_clauses,
- List **considered_relids)
+ List **considered_relids,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
ListCell *lc;
@@ -577,7 +617,9 @@ consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
rclauseset, jclauseset, eclauseset,
bitindexpaths,
bms_union(clause_relids, oldrelids),
- considered_relids);
+ considered_relids,
+ rel_grouped,
+ agg_info);
}
/* Also try this set of relids by itself */
@@ -585,7 +627,9 @@ consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
rclauseset, jclauseset, eclauseset,
bitindexpaths,
clause_relids,
- considered_relids);
+ considered_relids,
+ rel_grouped,
+ agg_info);
}
}
@@ -601,6 +645,10 @@ consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
* 'bitindexpaths', 'considered_relids' as above
* 'relids' is the current set of relids to consider (the target rel plus
* one or more outer rels)
+ *
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
*/
static void
get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
@@ -610,7 +658,9 @@ get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexClauseSet *eclauseset,
List **bitindexpaths,
Relids relids,
- List **considered_relids)
+ List **considered_relids,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
IndexClauseSet clauseset;
int indexcol;
@@ -667,7 +717,8 @@ get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
Assert(clauseset.nonempty);
/* Build index path(s) using the collected set of clauses */
- get_index_paths(root, rel, index, &clauseset, bitindexpaths);
+ get_index_paths(root, rel, index, &clauseset, bitindexpaths,
+ rel_grouped, agg_info);
/*
* Remember we considered paths for this set of relids. We use lcons not
@@ -717,7 +768,6 @@ bms_equal_any(Relids relids, List *relids_list)
return false;
}
-
/*
* get_index_paths
* Given an index and a set of index clauses for it, construct IndexPaths.
@@ -732,16 +782,23 @@ bms_equal_any(Relids relids, List *relids_list)
* paths, and then make a separate attempt to include them in bitmap paths.
* Furthermore, we should consider excluding lower-order ScalarArrayOpExpr
* quals so as to create ordered paths.
+ *
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
*/
static void
get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
- List **bitindexpaths)
+ List **bitindexpaths,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
List *indexpaths;
bool skip_nonnative_saop = false;
bool skip_lower_saop = false;
ListCell *lc;
+ bool grouped = rel_grouped != NULL;
/*
* Build simple index paths using the clauses. Allow ScalarArrayOpExpr
@@ -754,7 +811,40 @@ get_index_paths(PlannerInfo *root, RelOptInfo *rel,
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
- &skip_lower_saop);
+ &skip_lower_saop,
+ rel_grouped,
+ agg_info);
+
+ /*
+ * If the relation is grouped, apply the partial aggregation.
+ *
+ * Only AGG_SORTED strategy is used, so we ignore bitmap paths, as well as
+ * indexes that can only produce them.
+ */
+ if (grouped && index->amhasgettuple)
+ {
+ foreach(lc, indexpaths)
+ {
+ IndexPath *ipath = (IndexPath *) lfirst(lc);
+ Path *subpath = &ipath->path;
+
+ if (subpath->pathkeys != NIL)
+ {
+ AggPath *agg_path;
+
+ agg_path = create_agg_sorted_path(root,
+ subpath,
+ agg_info);
+ if (agg_path)
+ add_path(rel_grouped, (Path *) agg_path);
+ }
+ }
+
+ /*
+ * Nothing more to do for grouped relation.
+ */
+ return;
+ }
/*
* If we skipped any lower-order ScalarArrayOpExprs on an index with an AM
@@ -769,7 +859,9 @@ get_index_paths(PlannerInfo *root, RelOptInfo *rel,
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
- NULL));
+ NULL,
+ rel_grouped,
+ agg_info));
}
/*
@@ -809,7 +901,9 @@ get_index_paths(PlannerInfo *root, RelOptInfo *rel,
false,
ST_BITMAPSCAN,
NULL,
- NULL);
+ NULL,
+ rel_grouped,
+ agg_info);
*bitindexpaths = list_concat(*bitindexpaths, indexpaths);
}
}
@@ -853,7 +947,15 @@ get_index_paths(PlannerInfo *root, RelOptInfo *rel,
* 'useful_predicate' indicates whether the index has a useful predicate
* 'scantype' indicates whether we need plain or bitmap scan support
* 'skip_nonnative_saop' indicates whether to accept SAOP if index AM doesn't
- * 'skip_lower_saop' indicates whether to accept non-first-column SAOP
+ * 'skip_lower_saop' indicates whether to accept non-first-column SAOP.
+ *
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
+ *
+ * XXX When enabling the aggregate push-down for partial paths, we'll need the
+ * function to return the aggregation input paths in a separate list instead
+ * of calling add_partial_path() on them immediately.
*/
static List *
build_index_paths(PlannerInfo *root, RelOptInfo *rel,
@@ -861,7 +963,9 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
- bool *skip_lower_saop)
+ bool *skip_lower_saop,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
List *result = NIL;
IndexPath *ipath;
@@ -878,6 +982,12 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
bool index_is_ordered;
bool index_only_scan;
int indexcol;
+ bool include_partial;
+
+ /*
+ * Grouped partial paths are not supported yet.
+ */
+ include_partial = rel_grouped == NULL;
/*
* Check that index supports the desired scan type(s)
@@ -1047,14 +1157,16 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
index_only_scan,
outer_relids,
loop_count,
- false);
+ false,
+ rel_grouped,
+ agg_info);
result = lappend(result, ipath);
/*
* If appropriate, consider parallel index scan. We don't allow
* parallel index scan for bitmap index scans.
*/
- if (index->amcanparallel &&
+ if (include_partial && index->amcanparallel &&
rel->consider_parallel && outer_relids == NULL &&
scantype != ST_BITMAPSCAN)
{
@@ -1070,7 +1182,9 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
index_only_scan,
outer_relids,
loop_count,
- true);
+ true,
+ rel_grouped,
+ agg_info);
/*
* if, after costing the path, we find that it's not worth using
@@ -1104,11 +1218,13 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
index_only_scan,
outer_relids,
loop_count,
- false);
+ false,
+ rel_grouped,
+ agg_info);
result = lappend(result, ipath);
/* If appropriate, consider parallel index scan */
- if (index->amcanparallel &&
+ if (include_partial && index->amcanparallel &&
rel->consider_parallel && outer_relids == NULL &&
scantype != ST_BITMAPSCAN)
{
@@ -1122,7 +1238,9 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
index_only_scan,
outer_relids,
loop_count,
- true);
+ true,
+ rel_grouped,
+ agg_info);
/*
* if, after costing the path, we find that it's not worth
@@ -1244,6 +1362,8 @@ build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
useful_predicate,
ST_BITMAPSCAN,
NULL,
+ NULL,
+ NULL,
NULL);
result = list_concat(result, indexpaths);
}
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index d8ff4bf432..f99d6b98a9 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -120,7 +120,9 @@ add_paths_to_joinrel(PlannerInfo *root,
RelOptInfo *innerrel,
JoinType jointype,
SpecialJoinInfo *sjinfo,
- List *restrictlist)
+ List *restrictlist,
+ RelAggInfo *agg_info,
+ RelOptInfo *rel_agg_input)
{
JoinPathExtraData extra;
bool mergejoin_allowed = true;
@@ -143,6 +145,8 @@ add_paths_to_joinrel(PlannerInfo *root,
extra.mergeclause_list = NIL;
extra.sjinfo = sjinfo;
extra.param_source_rels = NULL;
+ extra.agg_info = agg_info;
+ extra.rel_agg_input = rel_agg_input;
/*
* See if the inner relation is provably unique for this outer rel.
@@ -376,6 +380,8 @@ try_nestloop_path(PlannerInfo *root,
Relids outerrelids;
Relids inner_paramrels = PATH_REQ_OUTER(inner_path);
Relids outer_paramrels = PATH_REQ_OUTER(outer_path);
+ bool do_aggregate = extra->rel_agg_input != NULL;
+ bool success = false;
/*
* Paths are parameterized by top-level parents, so run parameterization
@@ -422,10 +428,37 @@ try_nestloop_path(PlannerInfo *root,
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
- if (add_path_precheck(joinrel,
- workspace.startup_cost, workspace.total_cost,
- pathkeys, required_outer))
+ /*
+ * If the join output should be aggregated, the precheck is skipped
+ * because it makes little sense to compare the new join path to existing,
+ * already aggregated paths. Since we don't have row count estimate yet,
+ * it's hard to involve AggPath in the precheck.
+ */
+ if ((!do_aggregate &&
+ add_path_precheck(joinrel,
+ workspace.startup_cost, workspace.total_cost,
+ pathkeys, required_outer)) ||
+ do_aggregate)
{
+ PathTarget *target;
+ Path *path;
+ RelOptInfo *parent_rel;
+
+ if (!do_aggregate)
+ {
+ target = joinrel->reltarget;
+ parent_rel = joinrel;
+ }
+ else
+ {
+ /*
+ * If the join output is subject to aggregation, the path must
+ * generate aggregation input.
+ */
+ target = extra->agg_info->input;
+ parent_rel = extra->rel_agg_input;
+ }
+
/*
* If the inner path is parameterized, it is parameterized by the
* topmost parent of the outer rel, not the outer rel itself. Fix
@@ -447,21 +480,57 @@ try_nestloop_path(PlannerInfo *root,
}
}
- add_path(joinrel, (Path *)
- create_nestloop_path(root,
- joinrel,
- jointype,
- &workspace,
- extra,
- outer_path,
- inner_path,
- extra->restrictlist,
- pathkeys,
- required_outer));
+ path = (Path *) create_nestloop_path(root,
+ parent_rel,
+ target,
+ jointype,
+ &workspace,
+ extra,
+ outer_path,
+ inner_path,
+ extra->restrictlist,
+ pathkeys,
+ required_outer);
+
+ if (!do_aggregate)
+ {
+ add_path(joinrel, path);
+ success = true;
+ }
+ else
+ {
+ /*
+ * Non-grouped rel had to be passed to create_nestloop_path() so
+ * that row estimate reflects the aggregation input data, however
+ * even the aggregation input path should eventually be owned by
+ * the grouped joinrel.
+ */
+ path->parent = joinrel;
+
+ /*
+ * Try both AGG_HASHED and AGG_SORTED aggregation.
+ *
+ * AGG_HASHED should not be parameterized because we don't want to
+ * create the hashtable again for each set of parameters.
+ */
+ if (required_outer == NULL)
+ success = add_grouped_path(root, joinrel, path,
+ AGG_HASHED, extra->agg_info);
+
+ /*
+ * Don't try AGG_SORTED if add_grouped_path() would reject it
+ * anyway.
+ */
+ if (pathkeys != NIL)
+ success = success ||
+ add_grouped_path(root, joinrel, path, AGG_SORTED,
+ extra->agg_info);
+ }
}
- else
+
+ if (!success)
{
- /* Waste no memory when we reject a path here */
+ /* Waste no memory when we reject path(s) here */
bms_free(required_outer);
}
}
@@ -538,6 +607,7 @@ try_partial_nestloop_path(PlannerInfo *root,
add_partial_path(joinrel, (Path *)
create_nestloop_path(root,
joinrel,
+ joinrel->reltarget,
jointype,
&workspace,
extra,
@@ -568,8 +638,11 @@ try_mergejoin_path(PlannerInfo *root,
{
Relids required_outer;
JoinCostWorkspace workspace;
+ bool grouped = extra->agg_info != NULL;
+ bool do_aggregate = extra->rel_agg_input != NULL;
+ bool success = false;
- if (is_partial)
+ if (!grouped && is_partial)
{
try_partial_mergejoin_path(root,
joinrel,
@@ -617,26 +690,73 @@ try_mergejoin_path(PlannerInfo *root,
outersortkeys, innersortkeys,
extra);
- if (add_path_precheck(joinrel,
- workspace.startup_cost, workspace.total_cost,
- pathkeys, required_outer))
+ /*
+ * See comments in try_nestloop_path().
+ */
+ if ((!do_aggregate &&
+ add_path_precheck(joinrel,
+ workspace.startup_cost, workspace.total_cost,
+ pathkeys, required_outer)) ||
+ do_aggregate)
{
- add_path(joinrel, (Path *)
- create_mergejoin_path(root,
- joinrel,
- jointype,
- &workspace,
- extra,
- outer_path,
- inner_path,
- extra->restrictlist,
- pathkeys,
- required_outer,
- mergeclauses,
- outersortkeys,
- innersortkeys));
+ PathTarget *target;
+ Path *path;
+ RelOptInfo *parent_rel;
+
+ if (!do_aggregate)
+ {
+ target = joinrel->reltarget;
+ parent_rel = joinrel;
+ }
+ else
+ {
+ target = extra->agg_info->input;
+ parent_rel = extra->rel_agg_input;
+ }
+
+ path = (Path *) create_mergejoin_path(root,
+ parent_rel,
+ target,
+ jointype,
+ &workspace,
+ extra,
+ outer_path,
+ inner_path,
+ extra->restrictlist,
+ pathkeys,
+ required_outer,
+ mergeclauses,
+ outersortkeys,
+ innersortkeys);
+
+ if (!do_aggregate)
+ {
+ add_path(joinrel, path);
+ success = true;
+ }
+ else
+ {
+ /* See comment in try_nestloop_path() */
+ path->parent = joinrel;
+
+ if (required_outer == NULL)
+ success = add_grouped_path(root,
+ joinrel,
+ path,
+ AGG_HASHED,
+ extra->agg_info);
+
+ if (pathkeys != NIL)
+ success = success ||
+ add_grouped_path(root,
+ joinrel,
+ path,
+ AGG_SORTED,
+ extra->agg_info);
+ }
}
- else
+
+ if (!success)
{
/* Waste no memory when we reject a path here */
bms_free(required_outer);
@@ -700,6 +820,7 @@ try_partial_mergejoin_path(PlannerInfo *root,
add_partial_path(joinrel, (Path *)
create_mergejoin_path(root,
joinrel,
+ joinrel->reltarget,
jointype,
&workspace,
extra,
@@ -729,6 +850,9 @@ try_hashjoin_path(PlannerInfo *root,
{
Relids required_outer;
JoinCostWorkspace workspace;
+ bool grouped = extra->agg_info != NULL;
+ bool do_aggregate = extra->rel_agg_input != NULL;
+ bool success = false;
/*
* Check to see if proposed path is still parameterized, and reject if the
@@ -745,30 +869,79 @@ try_hashjoin_path(PlannerInfo *root,
}
/*
+ * Parameterized execution of grouped path would mean repeated hashing of
+ * the output of the hashjoin output, so forget about AGG_HASHED if there
+ * are any parameters. And AGG_SORTED makes no sense because the hash join
+ * output is not sorted.
+ */
+ if (required_outer && grouped)
+ return;
+
+ /*
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra, false);
- if (add_path_precheck(joinrel,
- workspace.startup_cost, workspace.total_cost,
- NIL, required_outer))
+
+ /*
+ * See comments in try_nestloop_path().
+ */
+ if ((!do_aggregate &&
+ add_path_precheck(joinrel,
+ workspace.startup_cost, workspace.total_cost,
+ NIL, required_outer)) ||
+ do_aggregate)
{
- add_path(joinrel, (Path *)
- create_hashjoin_path(root,
- joinrel,
- jointype,
- &workspace,
- extra,
- outer_path,
- inner_path,
- false, /* parallel_hash */
- extra->restrictlist,
- required_outer,
- hashclauses));
+ PathTarget *target;
+ Path *path = NULL;
+ RelOptInfo *parent_rel;
+
+ if (!do_aggregate)
+ {
+ target = joinrel->reltarget;
+ parent_rel = joinrel;
+ }
+ else
+ {
+ target = extra->agg_info->input;
+ parent_rel = extra->rel_agg_input;
+ }
+
+ path = (Path *) create_hashjoin_path(root,
+ parent_rel,
+ target,
+ jointype,
+ &workspace,
+ extra,
+ outer_path,
+ inner_path,
+ false, /* parallel_hash */
+ extra->restrictlist,
+ required_outer,
+ hashclauses);
+
+ if (!do_aggregate)
+ {
+ add_path(joinrel, path);
+ success = true;
+ }
+ else
+ {
+ /* See comment in try_nestloop_path() */
+ path->parent = joinrel;
+
+ /*
+ * As the hashjoin path is not sorted, only try AGG_HASHED.
+ */
+ if (add_grouped_path(root, joinrel, path, AGG_HASHED,
+ extra->agg_info))
+ success = true;
+ }
}
- else
+
+ if (!success)
{
/* Waste no memory when we reject a path here */
bms_free(required_outer);
@@ -824,6 +997,7 @@ try_partial_hashjoin_path(PlannerInfo *root,
add_partial_path(joinrel, (Path *)
create_hashjoin_path(root,
joinrel,
+ joinrel->reltarget,
jointype,
&workspace,
extra,
@@ -892,6 +1066,7 @@ sort_inner_and_outer(PlannerInfo *root,
Path *cheapest_safe_inner = NULL;
List *all_pathkeys;
ListCell *l;
+ bool grouped = extra->agg_info != NULL;
/*
* We only consider the cheapest-total-cost input paths, since we are
@@ -1051,7 +1226,7 @@ sort_inner_and_outer(PlannerInfo *root,
* If we have partial outer and parallel safe inner path then try
* partial mergejoin path.
*/
- if (cheapest_partial_outer && cheapest_safe_inner)
+ if (!grouped && cheapest_partial_outer && cheapest_safe_inner)
try_partial_mergejoin_path(root,
joinrel,
cheapest_partial_outer,
@@ -1341,6 +1516,7 @@ match_unsorted_outer(PlannerInfo *root,
Path *inner_cheapest_total = innerrel->cheapest_total_path;
Path *matpath = NULL;
ListCell *lc1;
+ bool grouped = extra->agg_info != NULL;
/*
* Nestloop only supports inner, left, semi, and anti joins. Also, if we
@@ -1516,7 +1692,8 @@ match_unsorted_outer(PlannerInfo *root,
* parameterized. Similarly, we can't handle JOIN_FULL and JOIN_RIGHT,
* because they can produce false null extended rows.
*/
- if (joinrel->consider_parallel &&
+ if (!grouped &&
+ joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
save_jointype != JOIN_FULL &&
save_jointype != JOIN_RIGHT &&
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 0caf67a916..cc2e32bd81 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -17,30 +17,41 @@
#include "miscadmin.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
+#include "optimizer/cost.h"
#include "optimizer/joininfo.h"
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
+#include "optimizer/tlist.h"
#include "partitioning/partbounds.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+#include "utils/selfuncs.h"
static void make_rels_by_clause_joins(PlannerInfo *root,
- RelOptInfo *old_rel,
+ RelOptInfoSet *old_relset,
ListCell *other_rels);
static void make_rels_by_clauseless_joins(PlannerInfo *root,
- RelOptInfo *old_rel,
+ RelOptInfoSet *old_relset,
ListCell *other_rels);
+static void set_grouped_joinrel_target(PlannerInfo *root, RelOptInfo *joinrel,
+ RelAggInfo *agg_info);
static bool has_join_restriction(PlannerInfo *root, RelOptInfo *rel);
static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool is_dummy_rel(RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
-static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2);
+static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ RelAggInfo *agg_info, RelOptInfo *rel_agg_input);
+static void make_join_rel_common_grouped(PlannerInfo *root, RelOptInfoSet *relset1,
+ RelOptInfoSet *relset2,
+ RelAggInfo *agg_info);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
- SpecialJoinInfo *sjinfo, List *restrictlist);
+ SpecialJoinInfo *sjinfo, List *restrictlist,
+ RelAggInfo *agg_info,
+ RelOptInfo *rel_agg_input);
static void try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *parent_sjinfo,
@@ -48,7 +59,6 @@ static void try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1,
static int match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel,
bool strict_op);
-
/*
* join_search_one_level
* Consider ways to produce join relations containing exactly 'level'
@@ -83,7 +93,15 @@ join_search_one_level(PlannerInfo *root, int level)
*/
foreach(r, joinrels[level - 1])
{
- RelOptInfo *old_rel = (RelOptInfo *) lfirst(r);
+ RelOptInfoSet *old_relset = (RelOptInfoSet *) lfirst(r);
+ RelOptInfo *old_rel;
+
+ /*
+ * Both rel_plain and rel_grouped should have the same clauses, so it
+ * does not matter which one we use here. The only difference is that
+ * rel_grouped is not guaranteed to exist, so use rel_plain.
+ */
+ old_rel = old_relset->rel_plain;
if (old_rel->joininfo != NIL || old_rel->has_eclass_joins ||
has_join_restriction(root, old_rel))
@@ -109,7 +127,7 @@ join_search_one_level(PlannerInfo *root, int level)
other_rels = list_head(joinrels[1]);
make_rels_by_clause_joins(root,
- old_rel,
+ old_relset,
other_rels);
}
else
@@ -127,7 +145,7 @@ join_search_one_level(PlannerInfo *root, int level)
* avoid the duplicated effort.
*/
make_rels_by_clauseless_joins(root,
- old_rel,
+ old_relset,
list_head(joinrels[1]));
}
}
@@ -153,7 +171,8 @@ join_search_one_level(PlannerInfo *root, int level)
foreach(r, joinrels[k])
{
- RelOptInfo *old_rel = (RelOptInfo *) lfirst(r);
+ RelOptInfoSet *old_relset = (RelOptInfoSet *) lfirst(r);
+ RelOptInfo *old_rel = old_relset->rel_plain;
ListCell *other_rels;
ListCell *r2;
@@ -173,7 +192,8 @@ join_search_one_level(PlannerInfo *root, int level)
for_each_cell(r2, other_rels)
{
- RelOptInfo *new_rel = (RelOptInfo *) lfirst(r2);
+ RelOptInfoSet *new_relset = (RelOptInfoSet *) lfirst(r2);
+ RelOptInfo *new_rel = new_relset->rel_plain;
if (!bms_overlap(old_rel->relids, new_rel->relids))
{
@@ -185,7 +205,7 @@ join_search_one_level(PlannerInfo *root, int level)
if (have_relevant_joinclause(root, old_rel, new_rel) ||
have_join_order_restriction(root, old_rel, new_rel))
{
- (void) make_join_rel(root, old_rel, new_rel);
+ (void) make_join_rel(root, old_relset, new_relset);
}
}
}
@@ -219,10 +239,10 @@ join_search_one_level(PlannerInfo *root, int level)
*/
foreach(r, joinrels[level - 1])
{
- RelOptInfo *old_rel = (RelOptInfo *) lfirst(r);
+ RelOptInfoSet *old_relset = (RelOptInfoSet *) lfirst(r);
make_rels_by_clauseless_joins(root,
- old_rel,
+ old_relset,
list_head(joinrels[1]));
}
@@ -268,25 +288,31 @@ join_search_one_level(PlannerInfo *root, int level)
* 'other_rels': the first cell in a linked list containing the other
* rels to be considered for joining
*
+ * 'rel_plain' is what we use of RelOptInfoSet because it's guaranteed to
+ * exists. However the test should have the same result if we used the
+ * corresponding 'rel_grouped'.
+ *
* Currently, this is only used with initial rels in other_rels, but it
* will work for joining to joinrels too.
*/
static void
make_rels_by_clause_joins(PlannerInfo *root,
- RelOptInfo *old_rel,
+ RelOptInfoSet *old_relset,
ListCell *other_rels)
{
ListCell *l;
for_each_cell(l, other_rels)
{
- RelOptInfo *other_rel = (RelOptInfo *) lfirst(l);
+ RelOptInfo *old_rel = old_relset->rel_plain;
+ RelOptInfoSet *other_relset = (RelOptInfoSet *) lfirst(l);
+ RelOptInfo *other_rel = other_relset->rel_plain;
if (!bms_overlap(old_rel->relids, other_rel->relids) &&
(have_relevant_joinclause(root, old_rel, other_rel) ||
have_join_order_restriction(root, old_rel, other_rel)))
{
- (void) make_join_rel(root, old_rel, other_rel);
+ (void) make_join_rel(root, old_relset, other_relset);
}
}
}
@@ -302,27 +328,62 @@ make_rels_by_clause_joins(PlannerInfo *root,
* 'other_rels': the first cell of a linked list containing the
* other rels to be considered for joining
*
+ * 'rel_plain' is what we use of RelOptInfoSet because it's guaranteed to
+ * exists. However the test should have the same result if we used the
+ * corresponding 'rel_grouped'.
+ *
* Currently, this is only used with initial rels in other_rels, but it would
* work for joining to joinrels too.
*/
static void
make_rels_by_clauseless_joins(PlannerInfo *root,
- RelOptInfo *old_rel,
+ RelOptInfoSet *old_relset,
ListCell *other_rels)
{
ListCell *l;
for_each_cell(l, other_rels)
{
- RelOptInfo *other_rel = (RelOptInfo *) lfirst(l);
+ RelOptInfo *old_rel = old_relset->rel_plain;
+ RelOptInfoSet *other_relset = (RelOptInfoSet *) lfirst(l);
+ RelOptInfo *other_rel = other_relset->rel_plain;
if (!bms_overlap(other_rel->relids, old_rel->relids))
{
- (void) make_join_rel(root, old_rel, other_rel);
+ (void) make_join_rel(root, old_relset, other_relset);
}
}
}
+/*
+ * Set joinrel's reltarget according to agg_info and estimate the number of
+ * rows.
+ */
+static void
+set_grouped_joinrel_target(PlannerInfo *root, RelOptInfo *joinrel,
+ RelAggInfo *agg_info)
+{
+ Assert(agg_info != NULL);
+
+ /*
+ * build_join_rel() does not create the target for grouped relation.
+ */
+ Assert(joinrel->reltarget == NULL);
+
+ joinrel->reltarget = agg_info->target;
+
+ /*
+ * Grouping essentially changes the number of rows.
+ *
+ * XXX We do not distinguish whether two plain rels are joined and the
+ * result is aggregated, or the aggregation has been already applied to
+ * one of the input rels. Is this worth extra effort, e.g. maintaining a
+ * separate RelOptInfo for each case (one difficulty that would introduce
+ * is construction of AppendPath)?
+ */
+ joinrel->rows = estimate_num_groups(root, agg_info->group_exprs,
+ agg_info->input_rows, NULL);
+}
/*
* join_is_legal
@@ -655,9 +716,18 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
/*
* make_join_rel_common
* The workhorse of make_join_rel().
+ *
+ * 'agg_info' contains the reltarget of grouped relation and everything we
+ * need to aggregate the join result. If NULL, then the join relation
+ * should not be grouped.
+ *
+ * 'rel_agg_input' describes the AggPath input relation if the join output
+ * should be aggregated. If NULL is passed, do not aggregate the join
+ * output.
*/
static RelOptInfo *
-make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ RelAggInfo *agg_info, RelOptInfo *rel_agg_input)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -665,10 +735,15 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
SpecialJoinInfo sjinfo_data;
RelOptInfo *joinrel;
List *restrictlist;
+ bool grouped = agg_info != NULL;
+ bool do_aggregate = rel_agg_input != NULL;
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(rel1->relids, rel2->relids));
+ /* do_aggregate implies the output to be grouped. */
+ Assert(!do_aggregate || grouped);
+
/* Construct Relids set that identifies the joinrel. */
joinrelids = bms_union(rel1->relids, rel2->relids);
@@ -718,7 +793,18 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* goes with this particular joining.
*/
joinrel = build_join_rel(root, joinrelids, rel1, rel2, sjinfo,
- &restrictlist);
+ &restrictlist, agg_info);
+
+ if (grouped)
+ {
+ /*
+ * Make sure the grouped joinrel has reltarget initialized. Caller
+ * should supply the target for grouped relation, so build_join_rel()
+ * should have omitted its creation.
+ */
+ if (joinrel->reltarget == NULL)
+ set_grouped_joinrel_target(root, joinrel, agg_info);
+ }
/*
* If we've already proven this join is empty, we needn't consider any
@@ -732,13 +818,72 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
/* Add paths to the join relation. */
populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
bms_free(joinrelids);
return joinrel;
}
+static void
+make_join_rel_common_grouped(PlannerInfo *root, RelOptInfoSet *relset1,
+ RelOptInfoSet *relset2,
+ RelAggInfo *agg_info)
+{
+ RelOptInfo *rel1_grouped = NULL;
+ RelOptInfo *rel2_grouped = NULL;
+ bool rel1_grouped_useful = false;
+ bool rel2_grouped_useful = false;
+
+ /*
+ * Retrieve the grouped relations.
+ *
+ * Dummy rel may indicates a join relation that is able to generate
+ * grouped paths as such (i.e. it has valid agg_info), but for which the
+ * path actually could not be created (e.g. only AGG_HASHED strategy was
+ * possible but work_mem was not sufficient for hash table).
+ */
+ if (relset1->rel_grouped)
+ rel1_grouped = relset1->rel_grouped;
+ if (relset2->rel_grouped)
+ rel2_grouped = relset2->rel_grouped;
+
+ rel1_grouped_useful = rel1_grouped != NULL && !IS_DUMMY_REL(rel1_grouped);
+ rel2_grouped_useful = rel2_grouped != NULL && !IS_DUMMY_REL(rel2_grouped);
+
+ /*
+ * Nothing to do?
+ */
+ if (!rel1_grouped_useful && !rel2_grouped_useful)
+ return;
+
+ /*
+ * At maximum one input rel can be grouped (here we don't care if any rel
+ * is eventually dummy, the existence of grouped rel indicates that
+ * aggregates can be pushed down to it). If both were grouped, then
+ * grouping of one side would change the occurrence of the other side's
+ * aggregate transient states on the input of the final aggregation. This
+ * can be handled by adjusting the transient states, but it's not worth
+ * the effort because it's hard to find a use case for this kind of join.
+ *
+ * XXX If the join of two grouped rels is implemented someday, note that
+ * both rels can have aggregates, so it'd be hard to join grouped rel to
+ * non-grouped here: 1) such a "mixed join" would require a special
+ * target, 2) both AGGSPLIT_FINAL_DESERIAL and AGGSPLIT_SIMPLE aggregates
+ * could appear in the target of the final aggregation node, originating
+ * from the grouped and the non-grouped input rel respectively.
+ */
+ if (rel1_grouped && rel2_grouped)
+ return;
+
+ if (rel1_grouped_useful)
+ make_join_rel_common(root, rel1_grouped, relset2->rel_plain, agg_info,
+ NULL);
+ else if (rel2_grouped_useful)
+ make_join_rel_common(root, relset1->rel_plain, rel2_grouped, agg_info,
+ NULL);
+}
+
/*
* make_join_rel
* Find or create a join RelOptInfo that represents the join of
@@ -747,14 +892,101 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* (The join rel may already contain paths generated from other
* pairs of rels that add up to the same set of base rels.)
*
- * NB: will return NULL if attempted join is not valid. This can happen
- * when working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
+ * In addition to creating an ordinary join relation, try to create a
+ * grouped one. There are two strategies to achieve that: join a grouped
+ * relation to plain one, or join two plain relations and apply partial
+ * aggregation to the result.
+ *
+ * NB: will return NULL if attempted join is not valid. This can happen when
+ * working with outer joins, or with IN or EXISTS clauses that have been
+ * turned into joins. Besides that, NULL is also returned if caller is
+ * interested in a grouped relation but there's no useful grouped input
+ * relation.
+ *
+ * Only the plain relation is returned.
+ *
+ * TODO geqo is the only caller interested in the result. We'll need to return
+ * RelOptInfoSet if geqo adopts the aggregate pushdown technique.
*/
RelOptInfo *
-make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+make_join_rel(PlannerInfo *root, RelOptInfoSet *relset1,
+ RelOptInfoSet *relset2)
{
- return make_join_rel_common(root, rel1, rel2);
+ Relids joinrelids;
+ RelAggInfo *agg_info;
+ RelOptInfoSet *joinrelset;
+ RelOptInfo *joinrel_plain;
+ RelOptInfo *rel1 = relset1->rel_plain;
+ RelOptInfo *rel2 = relset2->rel_plain;
+
+ /* 1) form the plain join. */
+ joinrel_plain = make_join_rel_common(root, rel1, rel2, NULL, NULL);
+
+ if (joinrel_plain == NULL)
+ return joinrel_plain;
+
+ /*
+ * We're done if there are no grouping expressions nor aggregates.
+ */
+ if (root->grouped_var_list == NIL)
+ return joinrel_plain;
+
+ /*
+ * If the same grouped joinrel was already formed, just with the base rels
+ * divided between rel1 and rel2 in a different way, we should already
+ * have the matching agg_info.
+ */
+ joinrelids = bms_union(rel1->relids, rel2->relids);
+ joinrelset = find_join_rel(root, joinrelids);
+
+ /*
+ * At the moment we know that non-grouped join exists, so the containing
+ * joinrelset should have been fetched.
+ */
+ Assert(joinrelset != NULL);
+
+ if (joinrelset->rel_grouped != NULL)
+ {
+ /*
+ * The same agg_info should be used for all the rels consisting of
+ * exactly joinrelids.
+ */
+ agg_info = joinrelset->agg_info;
+ }
+ else
+ {
+ /*
+ * agg_info must be created from scratch.
+ */
+ agg_info = create_rel_agg_info(root, joinrel_plain);
+
+ /*
+ * The number of aggregate input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ if (agg_info != NULL)
+ agg_info->input_rows = joinrel_plain->rows;
+ }
+
+ /*
+ * Cannot we build grouped join?
+ */
+ if (agg_info == NULL)
+ return joinrel_plain;
+
+ /*
+ * 2) join two plain rels and aggregate the join paths. Aggregate
+ * push-down only makes sense if the join is not the final one.
+ */
+ if (bms_nonempty_difference(root->all_baserels, joinrelids))
+ make_join_rel_common(root, rel1, rel2, agg_info, joinrel_plain);
+
+ /*
+ * 3) combine plain and grouped relations.
+ */
+ make_join_rel_common_grouped(root, relset1, relset2, agg_info);
+
+ return joinrel_plain;
}
/*
@@ -767,8 +999,11 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
static void
populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
- SpecialJoinInfo *sjinfo, List *restrictlist)
+ SpecialJoinInfo *sjinfo, List *restrictlist,
+ RelAggInfo *agg_info, RelOptInfo *rel_agg_input)
{
+ bool grouped = agg_info != NULL;
+
/*
* Consider paths using each rel as both outer and inner. Depending on
* the join type, a provably empty outer or inner rel might mean the join
@@ -798,10 +1033,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_INNER, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_INNER, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
break;
case JOIN_LEFT:
if (is_dummy_rel(rel1) ||
@@ -815,10 +1050,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_LEFT, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_RIGHT, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
break;
case JOIN_FULL:
if ((is_dummy_rel(rel1) && is_dummy_rel(rel2)) ||
@@ -829,10 +1064,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_FULL, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_FULL, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
/*
* If there are join quals that aren't mergeable or hashable, we
@@ -865,7 +1100,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_SEMI, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
}
/*
@@ -888,10 +1123,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_UNIQUE_INNER, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_UNIQUE_OUTER, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
}
break;
case JOIN_ANTI:
@@ -906,7 +1141,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_ANTI, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
break;
default:
/* other values not expected here */
@@ -914,8 +1149,16 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
}
- /* Apply partitionwise join technique, if possible. */
- try_partitionwise_join(root, rel1, rel2, joinrel, sjinfo, restrictlist);
+ /*
+ * The aggregate push-down feature currently does not support
+ * partition-wise aggregation.
+ */
+ if (grouped)
+ return;
+
+ /* Apply partition-wise join technique, if possible. */
+ try_partitionwise_join(root, rel1, rel2, joinrel, sjinfo,
+ restrictlist);
}
@@ -1115,7 +1358,8 @@ has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel)
foreach(lc, root->initial_rels)
{
- RelOptInfo *rel2 = (RelOptInfo *) lfirst(lc);
+ RelOptInfoSet *relset2 = lfirst_node(RelOptInfoSet, lc);
+ RelOptInfo *rel2 = relset2->rel_plain;
/* ignore rels that are already in "rel" */
if (bms_overlap(rel->relids, rel2->relids))
@@ -1423,7 +1667,7 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
populate_joinrel_with_paths(root, child_rel1, child_rel2,
child_joinrel, child_sjinfo,
- child_restrictlist);
+ child_restrictlist, NULL, NULL);
}
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 97d0c28132..1bc73a8110 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -887,6 +887,18 @@ use_physical_tlist(PlannerInfo *root, Path *path, int flags)
}
}
+ /*
+ * If aggregate was pushed down, the target can contain aggregates. The
+ * original target must be preserved then.
+ */
+ foreach(lc, path->pathtarget->exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, Aggref))
+ return false;
+ }
+
return true;
}
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index a66374094f..965d993ba5 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -14,6 +14,7 @@
*/
#include "postgres.h"
+#include "access/sysattr.h"
#include "catalog/pg_type.h"
#include "catalog/pg_class.h"
#include "nodes/nodeFuncs.h"
@@ -27,6 +28,7 @@
#include "optimizer/planner.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
+#include "optimizer/tlist.h"
#include "optimizer/var.h"
#include "parser/analyze.h"
#include "rewrite/rewriteManip.h"
@@ -46,6 +48,8 @@ typedef struct PostponedQual
} PostponedQual;
+static void create_aggregate_grouped_var_infos(PlannerInfo *root);
+static void create_grouping_expr_grouped_var_infos(PlannerInfo *root);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
@@ -96,10 +100,9 @@ static void check_hashjoinable(RestrictInfo *restrictinfo);
* jtnode. Internally, the function recurses through the jointree.
*
* At the end of this process, there should be one baserel RelOptInfo for
- * every non-join RTE that is used in the query. Therefore, this routine
- * is the only place that should call build_simple_rel with reloptkind
- * RELOPT_BASEREL. (Note: build_simple_rel recurses internally to build
- * "other rel" RelOptInfos for the members of any appendrels we find here.)
+ * every non-grouped non-join RTE that is used in the query. (Note:
+ * build_simple_rel recurses internally to build "other rel" RelOptInfos for
+ * the members of any appendrels we find here.)
*/
void
add_base_rels_to_query(PlannerInfo *root, Node *jtnode)
@@ -241,6 +244,261 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
}
}
+/*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate as well as for
+ * each possible grouping expression.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+extern void
+setup_aggregate_pushdown(PlannerInfo *root)
+{
+ ListCell *lc;
+
+ /*
+ * Isn't user interested in the aggregate push-down feature?
+ */
+ if (!enable_agg_pushdown)
+ return;
+
+ /* The feature can only be applied to grouped aggregation. */
+ if (!root->parse->groupClause)
+ return;
+
+ /*
+ * Grouping sets require multiple different groupings but the base
+ * relation can only generate one.
+ */
+ if (root->parse->groupingSets)
+ return;
+
+ /*
+ * SRF is not allowed in the aggregate argument and we don't even want it
+ * in the GROUP BY clause, so forbid it in general. It needs to be
+ * analyzed if evaluation of a GROUP BY clause containing SRF below the
+ * query targetlist would be correct. Currently it does not seem to be an
+ * important use case.
+ */
+ if (root->parse->hasTargetSRFs)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_aggregate_grouped_var_infos(root);
+
+ /* Isn't there any aggregate to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Create GroupedVarInfo per grouping expression. */
+ create_grouping_expr_grouped_var_infos(root);
+
+ /* Isn't there any useful grouping expression for aggregate push-down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /*
+ * Now that we know that grouping can be pushed down, search for the
+ * maximum sortgroupref. The base relations may need it if extra grouping
+ * expressions get added to them.
+ */
+ Assert(root->max_sortgroupref == 0);
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc);
+
+ if (te->ressortgroupref > root->max_sortgroupref)
+ root->max_sortgroupref = te->ressortgroupref;
+ }
+}
+
+/*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ */
+static void
+create_aggregate_grouped_var_infos(PlannerInfo *root)
+{
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES |
+ PVC_INCLUDE_WINDOWFUNCS);
+
+ /*
+ * Although GroupingFunc is related to root->parse->groupingSets, this
+ * field does not necessarily reflect its presence.
+ */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, GroupingFunc))
+ return;
+ }
+
+ /*
+ * Aggregates within the HAVING clause need to be processed in the same
+ * way as those in the main targetlist.
+ *
+ * Note that the contained aggregates will be pushed down, but the
+ * containing HAVING clause must be ignored until the aggregation is
+ * finalized.
+ */
+ if (root->parse->havingQual != NULL)
+ {
+ List *having_exprs;
+
+ having_exprs = pull_var_clause((Node *) root->parse->havingQual,
+ PVC_INCLUDE_AGGREGATES);
+ if (having_exprs != NIL)
+ tlist_exprs = list_concat(tlist_exprs, having_exprs);
+ }
+
+ if (tlist_exprs == NIL)
+ return;
+
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ /*
+ * tlist_exprs may also contain Vars or WindowFuncs, but we only need
+ * Aggrefs.
+ */
+ if (IsA(expr, Var) ||IsA(expr, WindowFunc))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct)
+ {
+ /*
+ * Aggregation push-down is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ gvi = lfirst_node(GroupedVarInfo, lc2);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ gvi = makeNode(GroupedVarInfo);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ gvi->gv_eval_at = NULL;
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+}
+
+/*
+ * Create GroupedVarInfo for each expression usable as grouping key.
+ *
+ * In addition to the expressions of the query targetlist, group_pathkeys is
+ * also considered the source of grouping expressions. That increases the
+ * chance to get the relation output grouped.
+ */
+static void
+create_grouping_expr_grouped_var_infos(PlannerInfo *root)
+{
+ ListCell *l1,
+ *l2;
+ List *exprs = NIL;
+ List *sortgrouprefs = NIL;
+
+ /*
+ * Make sure GroupedVarInfo exists for each expression usable as grouping
+ * key.
+ */
+ foreach(l1, root->parse->groupClause)
+ {
+ SortGroupClause *sgClause;
+ TargetEntry *te;
+ Index sortgroupref;
+
+ sgClause = lfirst_node(SortGroupClause, l1);
+ te = get_sortgroupclause_tle(sgClause, root->processed_tlist);
+ sortgroupref = te->ressortgroupref;
+
+ Assert(sortgroupref > 0);
+
+ /*
+ * Non-zero sortgroupref does not necessarily imply grouping
+ * expression: data can also be sorted by aggregate.
+ */
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ /*
+ * The aggregate push-down feature currently supports only plain Vars
+ * as grouping expressions.
+ */
+ if (!IsA(te->expr, Var))
+ {
+ root->grouped_var_list = NIL;
+ return;
+ }
+
+ exprs = lappend(exprs, te->expr);
+ sortgrouprefs = lappend_int(sortgrouprefs, sortgroupref);
+ }
+
+ /*
+ * Construct GroupedVarInfo for each expression.
+ */
+ forboth(l1, exprs, l2, sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, l1);
+ int sortgroupref = lfirst_int(l2);
+ GroupedVarInfo *gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvexpr = (Expr *) copyObject(var);
+ gvi->sortgroupref = sortgroupref;
+
+ /* Find out where the expression should be evaluated. */
+ gvi->gv_eval_at = bms_make_singleton(var->varno);
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+}
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c
index d98cb89984..6dd3a34a99 100644
--- a/src/backend/optimizer/plan/planagg.c
+++ b/src/backend/optimizer/plan/planagg.c
@@ -348,6 +348,7 @@ build_minmax_path(PlannerInfo *root, MinMaxAggInfo *mminfo,
List *tlist;
NullTest *ntest;
SortGroupClause *sortcl;
+ RelOptInfoSet *final_relset;
RelOptInfo *final_rel;
Path *sorted_path;
Cost path_cost;
@@ -441,7 +442,8 @@ build_minmax_path(PlannerInfo *root, MinMaxAggInfo *mminfo,
subroot->tuple_fraction = 1.0;
subroot->limit_tuples = 1.0;
- final_rel = query_planner(subroot, tlist, minmax_qp_callback, NULL);
+ final_relset = query_planner(subroot, tlist, minmax_qp_callback, NULL);
+ final_rel = final_relset->rel_plain;
/*
* Since we didn't go through subquery_planner() to handle the subquery,
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index fc97a1bb50..7130a05659 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -52,13 +52,15 @@
* qp_callback once we have completed merging the query's equivalence classes.
* (We cannot construct canonical pathkeys until that's done.)
*/
-RelOptInfo *
+RelOptInfoSet *
query_planner(PlannerInfo *root, List *tlist,
query_pathkeys_callback qp_callback, void *qp_extra)
{
Query *parse = root->parse;
List *joinlist;
+ RelOptInfoSet *final_rel_set;
RelOptInfo *final_rel;
+ RelOptInfoSet *final_relset;
/*
* If the query has an empty join tree, then it's something easy like
@@ -66,6 +68,8 @@ query_planner(PlannerInfo *root, List *tlist,
*/
if (parse->jointree->fromlist == NIL)
{
+ RelOptInfo *final_rel;
+
/* We need a dummy joinrel to describe the empty set of baserels */
final_rel = build_empty_join_rel(root);
@@ -95,7 +99,9 @@ query_planner(PlannerInfo *root, List *tlist,
root->canon_pathkeys = NIL;
(*qp_callback) (root, qp_extra);
- return final_rel;
+ final_relset = makeNode(RelOptInfoSet);
+ final_relset->rel_plain = final_rel;
+ return final_relset;
}
/*
@@ -114,6 +120,7 @@ query_planner(PlannerInfo *root, List *tlist,
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
@@ -199,6 +206,11 @@ query_planner(PlannerInfo *root, List *tlist,
joinlist = remove_useless_joins(root, joinlist);
/*
+ * No rels should disappear now, so we can initialize all_baserels.
+ */
+ /* setup_all_baserels(root); */
+
+ /*
* Also, reduce any semijoins with unique inner rels to plain inner joins.
* Likewise, this can't be done until now for lack of needed info.
*/
@@ -232,14 +244,25 @@ query_planner(PlannerInfo *root, List *tlist,
extract_restriction_or_clauses(root);
/*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, setup
+ * root->grouped_var_list.
+ *
+ * The base relations should be fully initialized now, so that we have
+ * enough info to decide whether grouping is possible.
+ */
+ setup_aggregate_pushdown(root);
+
+ /*
* Ready to do the primary planning.
*/
- final_rel = make_one_rel(root, joinlist);
+ final_rel_set = make_one_rel(root, joinlist);
+ final_rel = final_rel_set->rel_plain;
/* Check that we got at least one usable path */
if (!final_rel || !final_rel->cheapest_total_path ||
final_rel->cheapest_total_path->param_info != NULL)
elog(ERROR, "failed to construct the join relation");
- return final_rel;
+ return final_rel_set;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 6f3a4a063c..3ef806fd75 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -147,7 +147,7 @@ static double get_number_of_groups(PlannerInfo *root,
grouping_sets_data *gd,
List *target_list);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
- RelOptInfo *input_rel,
+ RelOptInfoSet *input_relset,
PathTarget *target,
bool target_parallel_safe,
const AggClauseCosts *agg_costs,
@@ -160,7 +160,7 @@ static RelOptInfo *make_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
PathTarget *target, bool target_parallel_safe,
Node *havingQual);
static void create_ordinary_grouping_paths(PlannerInfo *root,
- RelOptInfo *input_rel,
+ RelOptInfoSet *input_relset,
RelOptInfo *grouped_rel,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd,
@@ -627,6 +627,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
memset(root->upper_rels, 0, sizeof(root->upper_rels));
memset(root->upper_targets, 0, sizeof(root->upper_targets));
root->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
root->grouping_map = NULL;
root->minmax_aggs = NIL;
root->qual_security_level = 0;
@@ -1659,6 +1660,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
List *final_targets;
List *final_targets_contain_srfs;
bool final_target_parallel_safe;
+ RelOptInfoSet *current_relset;
RelOptInfo *current_rel;
RelOptInfo *final_rel;
ListCell *lc;
@@ -1878,8 +1880,9 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
* We also generate (in standard_qp_callback) pathkey representations
* of the query's sort clause, distinct clause, etc.
*/
- current_rel = query_planner(root, tlist,
- standard_qp_callback, &qp_extra);
+ current_relset = query_planner(root, tlist,
+ standard_qp_callback, &qp_extra);
+ current_rel = current_relset->rel_plain;
/*
* Convert the query's result tlist into PathTarget format.
@@ -2019,7 +2022,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
if (have_grouping)
{
current_rel = create_grouping_paths(root,
- current_rel,
+ current_relset,
grouping_target,
grouping_target_parallel_safe,
&agg_costs,
@@ -3650,13 +3653,14 @@ get_number_of_groups(PlannerInfo *root,
*/
static RelOptInfo *
create_grouping_paths(PlannerInfo *root,
- RelOptInfo *input_rel,
+ RelOptInfoSet *input_relset,
PathTarget *target,
bool target_parallel_safe,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd)
{
Query *parse = root->parse;
+ RelOptInfo *input_rel = input_relset->rel_plain;
RelOptInfo *grouped_rel;
RelOptInfo *partially_grouped_rel;
@@ -3741,7 +3745,7 @@ create_grouping_paths(PlannerInfo *root,
else
extra.patype = PARTITIONWISE_AGGREGATE_NONE;
- create_ordinary_grouping_paths(root, input_rel, grouped_rel,
+ create_ordinary_grouping_paths(root, input_relset, grouped_rel,
agg_costs, gd, &extra,
&partially_grouped_rel);
}
@@ -3897,13 +3901,14 @@ create_degenerate_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
* function creates, or to NULL if it doesn't create one.
*/
static void
-create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
+create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfoSet *input_relset,
RelOptInfo *grouped_rel,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd,
GroupPathExtraData *extra,
RelOptInfo **partially_grouped_rel_p)
{
+ RelOptInfo *input_rel = input_relset->rel_plain;
Path *cheapest_path = input_rel->cheapest_total_path;
RelOptInfo *partially_grouped_rel = NULL;
double dNumGroups;
@@ -3947,13 +3952,25 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
if ((extra->flags & GROUPING_CAN_PARTIAL_AGG) != 0)
{
bool force_rel_creation;
+ RelOptInfo *input_rel_grouped = input_relset->rel_grouped;
+ bool have_agg_pushdown_paths;
/*
- * If we're doing partitionwise aggregation at this level, force
- * creation of a partially_grouped_rel so we can add partitionwise
- * paths to it.
+ * Check if the aggregate push-down feature succeeded to generate any
+ * paths. Dummy relation can appear here because grouped paths are not
+ * guaranteed to exist for a relation.
*/
- force_rel_creation = (patype == PARTITIONWISE_AGGREGATE_PARTIAL);
+ have_agg_pushdown_paths = input_rel_grouped != NULL &&
+ input_rel_grouped->pathlist != NIL &&
+ !IS_DUMMY_REL(input_rel_grouped);
+
+ /*
+ * If we're doing partitionwise aggregation at this level or if
+ * aggregate push-down succeeded to create some paths, force creation
+ * of a partially_grouped_rel so we can add the related paths to it.
+ */
+ force_rel_creation = patype == PARTITIONWISE_AGGREGATE_PARTIAL ||
+ have_agg_pushdown_paths;
partially_grouped_rel =
create_partial_grouping_paths(root,
@@ -3962,6 +3979,23 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
gd,
extra,
force_rel_creation);
+
+ /*
+ * No further processing is needed for paths provided by the aggregate
+ * push-down feature. Simply add them to the partially grouped
+ * relation.
+ */
+ if (have_agg_pushdown_paths)
+ {
+ ListCell *lc;
+
+ foreach(lc, input_rel_grouped->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ add_path(partially_grouped_rel, path);
+ }
+ }
}
/* Set out parameter. */
@@ -3986,10 +4020,14 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
/* Gather any partially grouped partial paths. */
if (partially_grouped_rel && partially_grouped_rel->partial_pathlist)
- {
gather_grouping_paths(root, partially_grouped_rel);
+
+ /*
+ * The non-partial paths can come either from the Gather above or from
+ * aggregate push-down.
+ */
+ if (partially_grouped_rel && partially_grouped_rel->pathlist)
set_cheapest(partially_grouped_rel);
- }
/*
* Estimate number of groups.
@@ -6068,7 +6106,7 @@ plan_cluster_use_sort(Oid tableOid, Oid indexOid)
comparisonCost = 2.0 * (indexExprCost.startup + indexExprCost.per_tuple);
/* Estimate the cost of seq scan + sort */
- seqScanPath = create_seqscan_path(root, rel, NULL, 0);
+ seqScanPath = create_seqscan_path(root, rel, NULL, 0, NULL, NULL);
cost_sort(&seqScanAndSortPath, root, NIL,
seqScanPath->total_cost, rel->tuples, rel->reltarget->width,
comparisonCost, maintenance_work_mem, -1.0);
@@ -6077,7 +6115,7 @@ plan_cluster_use_sort(Oid tableOid, Oid indexOid)
indexScanPath = create_index_path(root, indexInfo,
NIL, NIL, NIL, NIL, NIL,
ForwardScanDirection, false,
- NULL, 1.0, false);
+ NULL, 1.0, false, NULL, NULL);
return (seqScanAndSortPath.total_cost < indexScanPath->path.total_cost);
}
@@ -7090,6 +7128,7 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
for (cnt_parts = 0; cnt_parts < nparts; cnt_parts++)
{
RelOptInfo *child_input_rel = input_rel->part_rels[cnt_parts];
+ RelOptInfoSet *child_input_relset;
PathTarget *child_target = copy_pathtarget(target);
AppendRelInfo **appinfos;
int nappinfos;
@@ -7148,12 +7187,17 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
continue;
}
+ child_input_relset = makeNode(RelOptInfoSet);
+ child_input_relset->rel_plain = child_input_rel;
+
/* Create grouping paths for this child relation. */
- create_ordinary_grouping_paths(root, child_input_rel,
+ create_ordinary_grouping_paths(root, child_input_relset,
child_grouped_rel,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
+ pfree(child_input_relset);
+
if (child_partially_grouped_rel)
{
partially_grouped_live_children =
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 5d363edab8..52f970ae71 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -2299,6 +2299,39 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, Aggref))
+ {
+ Aggref *aggref = castNode(Aggref, node);
+
+ /*
+ * The upper plan targetlist can contain Aggref whose value has
+ * already been evaluated by the subplan. However this can only happen
+ * with specific value of aggsplit.
+ */
+ if (aggref->aggsplit == AGGSPLIT_INITIAL_SERIAL)
+ {
+ /* See if the Aggref has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ }
+
+ /* No referent found for Aggref */
+ elog(ERROR, "Aggref not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 77dbf4eba3..a07e41169e 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -914,6 +914,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
memset(subroot->upper_rels, 0, sizeof(subroot->upper_rels));
memset(subroot->upper_targets, 0, sizeof(subroot->upper_targets));
subroot->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
subroot->grouping_map = NULL;
subroot->minmax_aggs = NIL;
subroot->qual_security_level = 0;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index b2637d0e89..899d4b4c6b 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -17,6 +17,8 @@
#include <math.h>
#include "miscadmin.h"
+#include "access/sysattr.h"
+#include "catalog/pg_constraint.h"
#include "foreign/fdwapi.h"
#include "nodes/extensible.h"
#include "nodes/nodeFuncs.h"
@@ -58,7 +60,6 @@ static List *reparameterize_pathlist_by_child(PlannerInfo *root,
List *pathlist,
RelOptInfo *child_rel);
-
/*****************************************************************************
* MISC. PATH UTILITIES
*****************************************************************************/
@@ -953,13 +954,15 @@ add_partial_path_precheck(RelOptInfo *parent_rel, Cost total_cost,
*/
Path *
create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
- Relids required_outer, int parallel_workers)
+ Relids required_outer, int parallel_workers,
+ RelOptInfo *rel_grouped, RelAggInfo *agg_info)
{
Path *pathnode = makeNode(Path);
pathnode->pathtype = T_SeqScan;
- pathnode->parent = rel;
- pathnode->pathtarget = rel->reltarget;
+ pathnode->parent = rel_grouped == NULL ? rel : rel_grouped;
+ pathnode->pathtarget = rel_grouped == NULL ? rel->reltarget :
+ agg_info->input;
pathnode->param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->parallel_aware = parallel_workers > 0 ? true : false;
@@ -1019,6 +1022,10 @@ create_samplescan_path(PlannerInfo *root, RelOptInfo *rel, Relids required_outer
* estimates of caching behavior.
* 'partial_path' is true if constructing a parallel index scan path.
*
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
+ *
* Returns the new path node.
*/
IndexPath *
@@ -1033,7 +1040,9 @@ create_index_path(PlannerInfo *root,
bool indexonly,
Relids required_outer,
double loop_count,
- bool partial_path)
+ bool partial_path,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
IndexPath *pathnode = makeNode(IndexPath);
RelOptInfo *rel = index->rel;
@@ -1041,8 +1050,9 @@ create_index_path(PlannerInfo *root,
*indexqualcols;
pathnode->path.pathtype = indexonly ? T_IndexOnlyScan : T_IndexScan;
- pathnode->path.parent = rel;
- pathnode->path.pathtarget = rel->reltarget;
+ pathnode->path.parent = rel_grouped == NULL ? rel : rel_grouped;
+ pathnode->path.pathtarget = rel_grouped == NULL ? rel->reltarget :
+ agg_info->input;
pathnode->path.param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->path.parallel_aware = false;
@@ -1186,8 +1196,8 @@ create_bitmap_or_path(PlannerInfo *root,
* Creates a path corresponding to a scan by TID, returning the pathnode.
*/
TidPath *
-create_tidscan_path(PlannerInfo *root, RelOptInfo *rel, List *tidquals,
- Relids required_outer)
+create_tidscan_path(PlannerInfo *root, RelOptInfo *rel,
+ List *tidquals, Relids required_outer)
{
TidPath *pathnode = makeNode(TidPath);
@@ -1342,7 +1352,8 @@ append_startup_cost_compare(const void *a, const void *b)
/*
* create_merge_append_path
* Creates a path corresponding to a MergeAppend plan, returning the
- * pathnode.
+ * pathnode. target can be supplied by caller. If NULL is passed, the field
+ * is set to rel->reltarget.
*/
MergeAppendPath *
create_merge_append_path(PlannerInfo *root,
@@ -1529,7 +1540,9 @@ create_unique_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
MemoryContext oldcontext;
int numCols;
- /* Caller made a mistake if subpath isn't cheapest_total ... */
+ /*
+ * Caller made a mistake if subpath isn't cheapest_total.
+ */
Assert(subpath == rel->cheapest_total_path);
Assert(subpath->parent == rel);
/* ... or if SpecialJoinInfo is the wrong one */
@@ -2150,6 +2163,7 @@ calc_non_nestloop_required_outer(Path *outer_path, Path *inner_path)
* relations.
*
* 'joinrel' is the join relation.
+ * 'target' is the join path target
* 'jointype' is the type of join required
* 'workspace' is the result from initial_cost_nestloop
* 'extra' contains various information about the join
@@ -2164,6 +2178,7 @@ calc_non_nestloop_required_outer(Path *outer_path, Path *inner_path)
NestPath *
create_nestloop_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -2204,7 +2219,7 @@ create_nestloop_path(PlannerInfo *root,
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
- pathnode->path.pathtarget = joinrel->reltarget;
+ pathnode->path.pathtarget = target;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
@@ -2236,6 +2251,7 @@ create_nestloop_path(PlannerInfo *root,
* two relations
*
* 'joinrel' is the join relation
+ * 'target' is the join path target
* 'jointype' is the type of join required
* 'workspace' is the result from initial_cost_mergejoin
* 'extra' contains various information about the join
@@ -2252,6 +2268,7 @@ create_nestloop_path(PlannerInfo *root,
MergePath *
create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -2268,7 +2285,7 @@ create_mergejoin_path(PlannerInfo *root,
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
- pathnode->jpath.path.pathtarget = joinrel->reltarget;
+ pathnode->jpath.path.pathtarget = target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
@@ -2304,6 +2321,7 @@ create_mergejoin_path(PlannerInfo *root,
* Creates a pathnode corresponding to a hash join between two relations.
*
* 'joinrel' is the join relation
+ * 'target' is the join path target
* 'jointype' is the type of join required
* 'workspace' is the result from initial_cost_hashjoin
* 'extra' contains various information about the join
@@ -2318,6 +2336,7 @@ create_mergejoin_path(PlannerInfo *root,
HashPath *
create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -2332,7 +2351,7 @@ create_hashjoin_path(PlannerInfo *root,
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
- pathnode->jpath.path.pathtarget = joinrel->reltarget;
+ pathnode->jpath.path.pathtarget = target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
@@ -2414,8 +2433,8 @@ create_projection_path(PlannerInfo *root,
* Note: in the latter case, create_projection_plan has to recheck our
* conclusion; see comments therein.
*/
- if (is_projection_capable_path(subpath) ||
- equal(oldtarget->exprs, target->exprs))
+ if ((is_projection_capable_path(subpath) ||
+ equal(oldtarget->exprs, target->exprs)))
{
/* No separate Result node needed */
pathnode->dummypp = true;
@@ -2800,8 +2819,7 @@ create_agg_path(PlannerInfo *root,
pathnode->path.pathtype = T_Agg;
pathnode->path.parent = rel;
pathnode->path.pathtarget = target;
- /* For now, assume we are above any joins, so no parameterization */
- pathnode->path.param_info = NULL;
+ pathnode->path.param_info = subpath->param_info;
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
@@ -2834,6 +2852,152 @@ create_agg_path(PlannerInfo *root,
}
/*
+ * Apply AGG_SORTED aggregation path to subpath if it's suitably sorted.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+AggPath *
+create_agg_sorted_path(PlannerInfo *root, Path *subpath,
+ RelAggInfo *agg_info)
+{
+ RelOptInfo *rel;
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ ListCell *lc1;
+ List *key_subset = NIL;
+ AggPath *result = NULL;
+
+ rel = subpath->parent;
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, (Node *) agg_exprs, aggsplit, &agg_costs);
+
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until the
+ * final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath, target,
+ AGG_SORTED, aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+
+ return result;
+}
+
+/*
+ * Apply AGG_HASHED aggregation to subpath.
+ *
+ * Arguments have the same meaning as those of create_agg_sorted_path.
+ */
+AggPath *
+create_agg_hashed_path(PlannerInfo *root, Path *subpath,
+ RelAggInfo *agg_info)
+{
+ RelOptInfo *rel;
+ bool can_hash;
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ Size hashaggtablesize;
+ Query *parse = root->parse;
+ AggPath *result = NULL;
+
+ rel = subpath->parent;
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, agg_exprs, aggsplit, &agg_costs);
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until
+ * the final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath,
+ target,
+ AGG_HASHED,
+ aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+
+ /*
+ * The agg path should require no fewer parameters than the plain
+ * one.
+ */
+ result->path.param_info = subpath->param_info;
+ }
+ }
+
+ return result;
+}
+
+/*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
*
@@ -3518,7 +3682,8 @@ reparameterize_path(PlannerInfo *root, Path *path,
switch (path->pathtype)
{
case T_SeqScan:
- return create_seqscan_path(root, rel, required_outer, 0);
+ return create_seqscan_path(root, rel, required_outer, 0, NULL,
+ NULL);
case T_SampleScan:
return (Path *) create_samplescan_path(root, rel, required_outer);
case T_IndexScan:
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 8141160c37..5d9c2a09e3 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -17,6 +17,8 @@
#include <limits.h>
#include "miscadmin.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_constraint.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
@@ -24,9 +26,12 @@
#include "optimizer/paths.h"
#include "optimizer/placeholder.h"
#include "optimizer/plancat.h"
+#include "optimizer/planner.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+#include "optimizer/var.h"
+#include "parser/parse_oper.h"
#include "partitioning/partbounds.h"
#include "utils/hsearch.h"
@@ -34,7 +39,7 @@
typedef struct JoinHashEntry
{
Relids join_relids; /* hash key --- MUST BE FIRST */
- RelOptInfo *join_rel;
+ RelOptInfoSet *join_relset;
} JoinHashEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
@@ -54,7 +59,7 @@ static List *subbuild_joinrel_joinlist(RelOptInfo *joinrel,
List *new_joininfo);
static void set_foreign_rel_properties(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel);
-static void add_join_rel(PlannerInfo *root, RelOptInfo *joinrel);
+static void add_join_rel(PlannerInfo *root, RelOptInfoSet *joinrelset);
static void build_joinrel_partition_info(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
List *restrictlist, JoinType jointype);
@@ -63,6 +68,9 @@ static void build_child_join_reltarget(PlannerInfo *root,
RelOptInfo *childrel,
int nappinfos,
AppendRelInfo **appinfos);
+static bool init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p);
/*
@@ -316,6 +324,97 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
}
/*
+ * build_simple_grouped_rel
+ * Construct a new RelOptInfo for a grouped base relation out of an
+ * existing non-grouped relation.
+ */
+void
+build_simple_grouped_rel(PlannerInfo *root, RelOptInfoSet *relset)
+{
+ RangeTblEntry *rte;
+ RelOptInfo *rel_plain,
+ *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /* Isn't there any grouping expression to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ rel_plain = relset->rel_plain;
+
+ /*
+ * Not all RTE kinds are supported when grouping is considered.
+ *
+ * TODO Consider relaxing some of these restrictions.
+ */
+ rte = root->simple_rte_array[rel_plain->relid];
+ if (rte->rtekind != RTE_RELATION ||
+ rte->relkind == RELKIND_FOREIGN_TABLE ||
+ rte->tablesample != NULL)
+ return;;
+
+ /*
+ * Grouped append relation is not supported yet.
+ */
+ if (rte->inh)
+ return;
+
+ /*
+ * Currently we do not support child relations ("other rels").
+ */
+ if (rel_plain->reloptkind != RELOPT_BASEREL)
+ return;
+
+ /*
+ * Prepare the information we need for aggregation of the rel contents.
+ */
+ agg_info = create_rel_agg_info(root, relset->rel_plain);
+ if (agg_info == NULL)
+ return;
+
+ /*
+ * TODO Consider if 1) a flat copy is o.k., 2) it's safer in terms of
+ * adding new fields to RelOptInfo) to copy everything and then reset some
+ * fields, or to zero the structure and copy individual fields.
+ */
+ rel_grouped = makeNode(RelOptInfo);
+ memcpy(rel_grouped, rel_plain, sizeof(RelOptInfo));
+
+ /*
+ * Note on consider_startup: while the AGG_HASHED strategy needs the whole
+ * relation, AGG_SORTED does not. Therefore we do not force
+ * consider_startup to false.
+ */
+
+ /*
+ * Set the appropriate target for grouped paths.
+ *
+ * The aggregation paths will get their input target from agg_info, so
+ * store it too.
+ */
+ rel_grouped->reltarget = agg_info->target;
+
+ /*
+ * Grouped paths must not be mixed with the plain ones.
+ */
+ rel_grouped->pathlist = NIL;
+ rel_grouped->partial_pathlist = NIL;
+ rel_grouped->cheapest_startup_path = NULL;
+ rel_grouped->cheapest_total_path = NULL;
+ rel_grouped->cheapest_unique_path = NULL;
+ rel_grouped->cheapest_parameterized_paths = NIL;
+
+ relset->rel_grouped = rel_grouped;
+
+ /*
+ * The number of aggregation input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ agg_info->input_rows = rel_plain->rows;
+ relset->agg_info = agg_info;
+}
+
+/*
* find_base_rel
* Find a base or other relation entry, which must already exist.
*/
@@ -364,16 +463,16 @@ build_join_rel_hash(PlannerInfo *root)
/* Insert all the already-existing joinrels */
foreach(l, root->join_rel_list)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
+ RelOptInfoSet *relset = (RelOptInfoSet *) lfirst(l);
JoinHashEntry *hentry;
bool found;
hentry = (JoinHashEntry *) hash_search(hashtab,
- &(rel->relids),
+ &(relset->rel_plain->relids),
HASH_ENTER,
&found);
Assert(!found);
- hentry->join_rel = rel;
+ hentry->join_relset = relset;
}
root->join_rel_hash = hashtab;
@@ -384,7 +483,7 @@ build_join_rel_hash(PlannerInfo *root)
* Returns relation entry corresponding to 'relids' (a set of RT indexes),
* or NULL if none exists. This is for join relations.
*/
-RelOptInfo *
+RelOptInfoSet *
find_join_rel(PlannerInfo *root, Relids relids)
{
/*
@@ -412,7 +511,13 @@ find_join_rel(PlannerInfo *root, Relids relids)
HASH_FIND,
NULL);
if (hentry)
- return hentry->join_rel;
+ {
+ RelOptInfoSet *result = hentry->join_relset;;
+
+ /* The plain relation should always be there. */
+ Assert(result->rel_plain != NULL);
+ return result;
+ }
}
else
{
@@ -420,10 +525,10 @@ find_join_rel(PlannerInfo *root, Relids relids)
foreach(l, root->join_rel_list)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
+ RelOptInfoSet *result = (RelOptInfoSet *) lfirst(l);
- if (bms_equal(rel->relids, relids))
- return rel;
+ if (bms_equal(result->rel_plain->relids, relids))
+ return result;
}
}
@@ -486,23 +591,26 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* PlannerInfo. Also add it to the auxiliary hashtable if there is one.
*/
static void
-add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
+add_join_rel(PlannerInfo *root, RelOptInfoSet *joinrelset)
{
- /* GEQO requires us to append the new joinrel to the end of the list! */
- root->join_rel_list = lappend(root->join_rel_list, joinrel);
+ /*
+ * GEQO requires us to append the new joinrel to the end of the list!
+ */
+ root->join_rel_list = lappend(root->join_rel_list, joinrelset);
/* store it into the auxiliary hashtable if there is one. */
if (root->join_rel_hash)
{
JoinHashEntry *hentry;
bool found;
+ Relids relids = joinrelset->rel_plain->relids;
hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
- &(joinrel->relids),
+ &relids,
HASH_ENTER,
&found);
Assert(!found);
- hentry->join_rel = joinrel;
+ hentry->join_relset = joinrelset;
}
}
@@ -518,6 +626,8 @@ add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
* 'restrictlist_ptr': result variable. If not NULL, *restrictlist_ptr
* receives the list of RestrictInfo nodes that apply to this
* particular pair of joinable relations.
+ * 'agg_info' contains information needed for the join to form grouped
+ * paths. If NULL, the join is not grouped.
*
* restrictlist_ptr makes the routine's API a little grotty, but it saves
* duplicated calculation of the restrictlist...
@@ -528,10 +638,20 @@ build_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr)
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info)
{
+ RelOptInfoSet *joinrelset;
+ bool new_set = false;
RelOptInfo *joinrel;
List *restrictlist;
+ bool grouped = agg_info != NULL;
+ bool create_target;
+
+ /*
+ * Target for grouped relation will be supplied by caller.
+ */
+ create_target = !grouped;
/* This function should be used only for join between parents. */
Assert(!IS_OTHER_REL(outer_rel) && !IS_OTHER_REL(inner_rel));
@@ -539,7 +659,19 @@ build_join_rel(PlannerInfo *root,
/*
* See if we already have a joinrel for this set of base rels.
*/
- joinrel = find_join_rel(root, joinrelids);
+ joinrelset = find_join_rel(root, joinrelids);
+ if (joinrelset == NULL)
+ {
+ /*
+ * The plain joinrel should be the first one to be added to the set.
+ */
+ Assert(!grouped);
+
+ joinrelset = makeNode(RelOptInfoSet);
+ new_set = true;
+ }
+
+ joinrel = !grouped ? joinrelset->rel_plain : joinrelset->rel_grouped;
if (joinrel)
{
@@ -566,7 +698,7 @@ build_join_rel(PlannerInfo *root,
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
- joinrel->reltarget = create_empty_pathtarget();
+ joinrel->reltarget = NULL;
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
joinrel->partial_pathlist = NIL;
@@ -631,9 +763,13 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ if (create_target)
+ {
+ joinrel->reltarget = create_empty_pathtarget();
+ build_joinrel_tlist(root, joinrel, outer_rel);
+ build_joinrel_tlist(root, joinrel, inner_rel);
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ }
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -669,45 +805,68 @@ build_join_rel(PlannerInfo *root,
sjinfo->jointype);
/*
- * Set estimates of the joinrel's size.
+ * Assign the joinrel to the set.
*/
- set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
- sjinfo, restrictlist);
+ if (!grouped)
+ joinrelset->rel_plain = joinrel;
+ else
+ {
+ joinrelset->rel_grouped = joinrel;
+ joinrelset->agg_info = agg_info;
+ }
- /*
- * Set the consider_parallel flag if this joinrel could potentially be
- * scanned within a parallel worker. If this flag is false for either
- * inner_rel or outer_rel, then it must be false for the joinrel also.
- * Even if both are true, there might be parallel-restricted expressions
- * in the targetlist or quals.
- *
- * Note that if there are more than two rels in this relation, they could
- * be divided between inner_rel and outer_rel in any arbitrary way. We
- * assume this doesn't matter, because we should hit all the same baserels
- * and joinclauses while building up to this joinrel no matter which we
- * take; therefore, we should make the same decision here however we get
- * here.
- */
- if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
- is_parallel_safe(root, (Node *) restrictlist) &&
- is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
- joinrel->consider_parallel = true;
+ if (new_set)
+ {
+ /* Add the joinrelset to the PlannerInfo. */
+ add_join_rel(root, joinrelset);
- /* Add the joinrel to the PlannerInfo. */
- add_join_rel(root, joinrel);
+ /*
+ * Also, if dynamic-programming join search is active, add the new
+ * joinrelset to the appropriate sublist. Note: you might think the
+ * Assert on number of members should be for equality, but some of the
+ * level 1 rels might have been joinrels already, so we can only
+ * assert <=.
+ */
+ if (root->join_rel_level)
+ {
+ Assert(root->join_cur_level > 0);
+ Assert(root->join_cur_level <= bms_num_members(joinrelids));
+ root->join_rel_level[root->join_cur_level] =
+ lappend(root->join_rel_level[root->join_cur_level],
+ joinrelset);
+ }
+ }
/*
- * Also, if dynamic-programming join search is active, add the new joinrel
- * to the appropriate sublist. Note: you might think the Assert on number
- * of members should be for equality, but some of the level 1 rels might
- * have been joinrels already, so we can only assert <=.
+ * Set estimates of the joinrel's size.
+ *
+ * XXX set_joinrel_size_estimates() claims to need reltarget but it does
+ * not seem to actually use it. Should we call it unconditionally so that
+ * callers of build_join_rel() do not have to care?
*/
- if (root->join_rel_level)
+ if (create_target)
{
- Assert(root->join_cur_level > 0);
- Assert(root->join_cur_level <= bms_num_members(joinrel->relids));
- root->join_rel_level[root->join_cur_level] =
- lappend(root->join_rel_level[root->join_cur_level], joinrel);
+ set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
+ sjinfo, restrictlist);
+
+ /*
+ * Set the consider_parallel flag if this joinrel could potentially be
+ * scanned within a parallel worker. If this flag is false for either
+ * inner_rel or outer_rel, then it must be false for the joinrel also.
+ * Even if both are true, there might be parallel-restricted
+ * expressions in the targetlist or quals.
+ *
+ * Note that if there are more than two rels in this relation, they
+ * could be divided between inner_rel and outer_rel in any arbitrary
+ * way. We assume this doesn't matter, because we should hit all the
+ * same baserels and joinclauses while building up to this joinrel no
+ * matter which we take; therefore, we should make the same decision
+ * here however we get here.
+ */
+ if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
+ is_parallel_safe(root, (Node *) restrictlist) &&
+ is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
+ joinrel->consider_parallel = true;
}
return joinrel;
@@ -734,6 +893,7 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
JoinType jointype)
{
RelOptInfo *joinrel = makeNode(RelOptInfo);
+ RelOptInfoSet *joinrelset = makeNode(RelOptInfoSet);
AppendRelInfo **appinfos;
int nappinfos;
@@ -845,7 +1005,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
Assert(!find_join_rel(root, joinrel->relids));
/* Add the relation to the PlannerInfo. */
- add_join_rel(root, joinrel);
+ joinrelset->rel_plain = joinrel;
+ add_join_rel(root, joinrelset);
return joinrel;
}
@@ -1772,3 +1933,626 @@ build_child_join_reltarget(PlannerInfo *root,
childrel->reltarget->cost.per_tuple = parentrel->reltarget->cost.per_tuple;
childrel->reltarget->width = parentrel->reltarget->width;
}
+
+/*
+ * Check if the relation can produce grouped paths and return the information
+ * it'll need for it. The passed relation is the non-grouped one which has the
+ * reltarget already constructed.
+ */
+RelAggInfo *
+create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel)
+{
+ List *gvis;
+ List *aggregates = NIL;
+ bool found_other_rel_agg;
+ ListCell *lc;
+ RelAggInfo *result;
+ PathTarget *agg_input;
+ PathTarget *target = NULL;
+ List *grp_exprs_extra = NIL;
+ List *group_clauses_final;
+ int i;
+
+ /*
+ * The function shouldn't have been called if there's no opportunity for
+ * aggregation push-down.
+ */
+ Assert(root->grouped_var_list != NIL);
+
+ result = makeNode(RelAggInfo);
+
+ /*
+ * The current implementation of aggregation push-down cannot handle
+ * PlaceHolderVar (PHV).
+ *
+ * If we knew that the PHV should be evaluated in this target (and of
+ * course, if its expression matched some Aggref argument), we'd just let
+ * init_grouping_targets add that Aggref. On the other hand, if we knew
+ * that the PHV is evaluated below the current rel, we could ignore it
+ * because the referencing Aggref would take care of propagation of the
+ * value to upper joins.
+ *
+ * The problem is that the same PHV can be evaluated in the target of the
+ * current rel or in that of lower rel --- depending on the input paths.
+ * For example, consider rel->relids = {A, B, C} and if ph_eval_at = {B,
+ * C}. Path "A JOIN (B JOIN C)" implies that the PHV is evaluated by the
+ * "(B JOIN C)", while path "(A JOIN B) JOIN C" evaluates the PHV itself.
+ */
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *expr = lfirst(lc);
+
+ if (IsA(expr, PlaceHolderVar))
+ return NULL;
+ }
+
+ if (IS_SIMPLE_REL(rel))
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return NULL;
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the Agg plan
+ * would receive different input at the base rel level.
+ *
+ * XXX For RELOPT_JOINREL, do not return if all the joins that can set any
+ * entry of the grouped target (do we need to postpone this check until
+ * the grouped target is available, and init_grouping_targets take care?)
+ * of this rel to NULL are provably below rel. (It's ok if rel is one of
+ * these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return NULL;
+
+ /*
+ * Use equivalence classes to generate additional grouping expressions for
+ * the current rel. Without these we might not be able to apply
+ * aggregation to the relation result set.
+ *
+ * It's important that create_grouping_expr_grouped_var_infos has
+ * processed the explicit grouping columns by now. If the grouping clause
+ * contains multiple expressions belonging to the same EC, the original
+ * (i.e. not derived) one should be preferred when we build grouping
+ * target for a relation. Otherwise we have a problem when trying to match
+ * target entries to grouping clauses during plan creation, see
+ * get_grouping_expression().
+ */
+ gvis = list_copy(root->grouped_var_list);
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+ int relid = -1;
+
+ /* Only interested in grouping expressions. */
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ while ((relid = bms_next_member(rel->relids, relid)) >= 0)
+ {
+ GroupedVarInfo *gvi_trans;
+
+ gvi_trans = translate_expression_to_rels(root, gvi, relid);
+ if (gvi_trans != NULL)
+ gvis = lappend(gvis, gvi_trans);
+ }
+ }
+
+ /*
+ * Check if some aggregates or grouping expressions can be evaluated in
+ * this relation's target, and collect all vars referenced by these
+ * aggregates / grouping expressions;
+ */
+ found_other_rel_agg = false;
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which includes
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * init_grouping_targets will handle plain Var grouping
+ * expressions because it needs to look them up in
+ * grouped_var_list anyway.
+ */
+ if (IsA(gvi->gvexpr, Var))
+ continue;
+
+ /*
+ * Currently, GroupedVarInfo only handles Vars and Aggrefs.
+ */
+ Assert(IsA(gvi->gvexpr, Aggref));
+
+ /* We only derive grouped expressions using ECs, not aggregates */
+ Assert(!gvi->derived);
+
+ gvi->agg_partial = (Aggref *) copyObject(gvi->gvexpr);
+ mark_partial_aggref(gvi->agg_partial, AGGSPLIT_INITIAL_SERIAL);
+
+ /*
+ * Accept the aggregate.
+ */
+ aggregates = lappend(aggregates, gvi);
+ }
+ else if (IsA(gvi->gvexpr, Aggref))
+ {
+ /*
+ * Remember that there is at least one aggregate expression that
+ * needs something else than this rel.
+ */
+ found_other_rel_agg = true;
+
+ /*
+ * This condition effectively terminates creation of the
+ * RelAggInfo, so there's no reason to check the next
+ * GroupedVarInfo.
+ */
+ break;
+ }
+ }
+
+ /*
+ * Grouping makes little sense w/o aggregate function and w/o grouping
+ * expressions.
+ */
+ if (aggregates == NIL)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Give up if some other aggregate(s) need relations other than the
+ * current one.
+ *
+ * If the aggregate needs the current rel plus anything else, then the
+ * problem is that grouping of the current relation could make some input
+ * variables unavailable for the "higher aggregate", and it'd also
+ * decrease the number of input rows the "higher aggregate" receives.
+ *
+ * If the aggregate does not even need the current rel, then neither the
+ * current rel nor anything else should be grouped because we do not
+ * support join of two grouped relations.
+ */
+ if (found_other_rel_agg)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Create target for grouped paths as well as one for the input paths of
+ * the aggregation paths.
+ */
+ target = create_empty_pathtarget();
+ agg_input = create_empty_pathtarget();
+
+ /*
+ * Cannot suitable targets for the aggregation push-down be derived?
+ */
+ if (!init_grouping_targets(root, rel, target, agg_input, gvis,
+ &grp_exprs_extra))
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ list_free(gvis);
+
+ /*
+ * Aggregation push-down makes no sense w/o grouping expressions.
+ */
+ if ((list_length(target->exprs) + list_length(grp_exprs_extra)) == 0)
+ return NULL;
+
+ group_clauses_final = root->parse->groupClause;
+
+ /*
+ * If the aggregation target should have extra grouping expressions (in
+ * order to emit input vars for join conditions), add them now. This step
+ * includes assignment of tleSortGroupRef's which we can generate now.
+ */
+ if (list_length(grp_exprs_extra) > 0)
+ {
+ Index sortgroupref;
+
+ /*
+ * We'll have to add some clauses, but query group clause must be
+ * preserved.
+ */
+ group_clauses_final = list_copy(group_clauses_final);
+
+ /*
+ * Always start at root->max_sortgroupref. The extra grouping
+ * expressions aren't used during the final aggregation, so the
+ * sortgroupref values don't need to be unique across the query. Thus
+ * we don't have to increase root->max_sortgroupref, which makes
+ * recognition of the extra grouping expressions pretty easy.
+ */
+ sortgroupref = root->max_sortgroupref;
+
+ /*
+ * Generate the SortGroupClause's and add the expressions to the
+ * target.
+ */
+ foreach(lc, grp_exprs_extra)
+ {
+ Var *var = lfirst_node(Var, lc);
+ SortGroupClause *cl = makeNode(SortGroupClause);
+
+ /*
+ * Initialize the SortGroupClause.
+ *
+ * As the final aggregation will not use this grouping expression,
+ * we don't care whether sortop is < or >. The value of
+ * nulls_first should not matter for the same reason.
+ */
+ cl->tleSortGroupRef = ++sortgroupref;
+ get_sort_group_operators(var->vartype,
+ false, true, false,
+ &cl->sortop, &cl->eqop, NULL,
+ &cl->hashable);
+ group_clauses_final = lappend(group_clauses_final, cl);
+ add_column_to_pathtarget(target, (Expr *) var,
+ cl->tleSortGroupRef);
+
+ /*
+ * The aggregation input target must emit this var too.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) var,
+ cl->tleSortGroupRef);
+ }
+ }
+
+ /*
+ * Add aggregates to the grouping target.
+ */
+ add_aggregates_to_target(root, target, aggregates);
+
+ /*
+ * Build a list of grouping expressions and a list of the corresponding
+ * SortGroupClauses.
+ */
+ i = 0;
+ foreach(lc, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(lc);
+
+ if (IsA(texpr, Aggref))
+ {
+ /*
+ * Once we see Aggref, no grouping expressions should follow.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides being an aggregate, the target expression should have no
+ * other reason then being a column of a relation functionally
+ * dependent on the GROUP BY clause. So it's not actually a grouping
+ * column.
+ */
+ if (sortgroupref == 0)
+ continue;
+
+ /*
+ * group_clause_final contains the "local" clauses, so this search
+ * should succeed.
+ */
+ cl = get_sortgroupref_clause(sortgroupref, group_clauses_final);
+
+ result->group_clauses = list_append_unique(result->group_clauses,
+ cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ result->group_exprs = list_append_unique(result->group_exprs,
+ texpr);
+ }
+
+ /*
+ * Since neither target nor agg_input is supposed to be identical to the
+ * source reltarget, compute the width and cost again.
+ *
+ * target does not yet contain aggregates, but these will be accounted by
+ * AggPath.
+ */
+ set_pathtarget_cost_width(root, target);
+ set_pathtarget_cost_width(root, agg_input);
+
+ result->target = target;
+ result->input = agg_input;
+
+ /* Finally collect the aggregates. */
+ while (lc != NULL)
+ {
+ Aggref *aggref = lfirst_node(Aggref, lc);
+
+ /*
+ * Partial aggregation is what the grouped paths should do.
+ */
+ result->agg_exprs = lappend(result->agg_exprs, aggref);
+ lc = lnext(lc);
+ }
+
+ /*
+ * The "input_rows" field should be set by caller.
+ */
+ return result;
+}
+
+/*
+ * Initialize target for grouped paths (target) as well as a target for paths
+ * that generate input for aggregation (agg_input).
+ *
+ * group_exprs_extra_p receives a list of Var nodes for which we need to
+ * construct SortGroupClause. Those vars will then be used as additional
+ * grouping expressions, for the sake of join clauses.
+ *
+ * gvis a list of GroupedVarInfo's possibly useful for rel.
+ *
+ * Return true iff the targets could be initialized.
+ */
+static bool
+init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p)
+{
+ ListCell *lc1,
+ *lc2;
+ List *unresolved = NIL;
+ List *unresolved_sortgrouprefs = NIL;
+
+ foreach(lc1, rel->reltarget->exprs)
+ {
+ Var *tvar;
+ bool is_grouping;
+ Index sortgroupref = 0;
+ bool derived = false;
+ bool needed_by_aggregate;
+
+ /*
+ * Given that PlaceHolderVar currently prevents us from doing
+ * aggregation push-down, the source target cannot contain anything
+ * more complex than a Var.
+ */
+ tvar = lfirst_node(Var, lc1);
+
+ is_grouping = is_grouping_expression(gvis, (Expr *) tvar,
+ &sortgroupref, &derived);
+
+ /*
+ * Derived grouping expressions should not be referenced by the query
+ * targetlist, so let them fall into vars_unresolved. It'll be checked
+ * later if the current targetlist needs them. For example, we should
+ * not automatically use Var as a grouping expression if the only
+ * reason for it to be in the plain relation target is that it's
+ * referenced by aggregate argument, and it happens to be in the same
+ * EC as any grouping expression.
+ */
+ if (is_grouping && !derived)
+ {
+ Assert(sortgroupref > 0);
+
+ /*
+ * It's o.k. to use the target expression for grouping.
+ */
+ add_column_to_pathtarget(target, (Expr *) tvar, sortgroupref);
+
+ /*
+ * As for agg_input, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) tvar, sortgroupref);
+
+ /* Process the next expression. */
+ continue;
+ }
+
+ /*
+ * Is this Var needed in the query targetlist for anything else than
+ * aggregate input?
+ */
+ needed_by_aggregate = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc2);
+ ListCell *lc3;
+ List *vars;
+
+ if (!IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (!bms_is_member(tvar->varno, gvi->gv_eval_at))
+ continue;
+
+ /*
+ * XXX Consider some sort of caching.
+ */
+ vars = pull_var_clause((Node *) gvi->gvexpr, PVC_RECURSE_AGGREGATES);
+ foreach(lc3, vars)
+ {
+ Var *var = lfirst_node(Var, lc3);
+
+ if (equal(var, tvar))
+ {
+ needed_by_aggregate = true;
+ break;
+ }
+ }
+ list_free(vars);
+ if (needed_by_aggregate)
+ break;
+ }
+
+ if (needed_by_aggregate)
+ {
+ bool found = false;
+
+ foreach(lc2, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc2);
+
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ if (equal(te->expr, tvar))
+ {
+ found = true;
+ break;
+ }
+ }
+
+ /*
+ * If it's only Aggref input, add it to the aggregation input
+ * target and that's it.
+ */
+ if (!found)
+ {
+ add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
+ continue;
+ }
+ }
+
+ /*
+ * Further investigation involves dependency check, for which we need
+ * to have all the (plain-var) grouping expressions gathered.
+ */
+ unresolved = lappend(unresolved, tvar);
+ unresolved_sortgrouprefs = lappend_int(unresolved_sortgrouprefs,
+ sortgroupref);
+ }
+
+ /*
+ * Check for other possible reasons for the var to be in the plain target.
+ */
+ forboth(lc1, unresolved, lc2, unresolved_sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, lc1);
+ Index sortgroupref = lfirst_int(lc2);
+ RangeTblEntry *rte;
+ List *deps = NIL;
+ Relids relids_subtract;
+ int ndx;
+ RelOptInfo *baserel;
+
+ rte = root->simple_rte_array[var->varno];
+
+ /*
+ * Check if the Var can be in the grouping key even though it's not
+ * mentioned by the GROUP BY clause (and could not be derived using
+ * ECs).
+ */
+ if (sortgroupref == 0 &&
+ check_functional_grouping(rte->relid, var->varno,
+ var->varlevelsup,
+ target->exprs, &deps))
+ {
+ /*
+ * The var shouldn't be actually used as a grouping key (instead,
+ * the one this depends on will be), so sortgroupref should not be
+ * important.
+ */
+ add_new_column_to_pathtarget(target, (Expr *) var);
+ add_new_column_to_pathtarget(agg_input, (Expr *) var);
+
+ /*
+ * The var may or may not be present in generic grouping
+ * expression(s) in addition, but this is handled elsewhere.
+ */
+ continue;
+ }
+
+ /*
+ * Isn't the expression needed by joins above the current rel?
+ *
+ * The relids we're not interested in do include 0, which is the
+ * top-level targetlist. The only reason for relids to contain 0
+ * should be that arg_var is referenced either by aggregate or by
+ * grouping expression, but right now we're interested in the *other*
+ * reasons. (As soon aggregation is pushed down, the aggregates in the
+ * query targetlist no longer need direct reference to arg_var
+ * anyway.)
+ */
+ relids_subtract = bms_copy(rel->relids);
+ bms_add_member(relids_subtract, 0);
+
+ baserel = find_base_rel(root, var->varno);
+ ndx = var->varattno - baserel->min_attr;
+ if (bms_nonempty_difference(baserel->attr_needed[ndx],
+ relids_subtract))
+ {
+ /*
+ * The variable is needed by a join involving this relation. That
+ * case includes variable that is referenced by a generic grouping
+ * expression.
+ *
+ * The only way to bring this var to the aggregation output is to
+ * add it to the grouping expressions too.
+ */
+ if (sortgroupref > 0)
+ {
+ /*
+ * The var could be recognized as a potentially useful
+ * grouping expression at the top of the loop, so we can add
+ * it to the grouping target, as well as to the agg_input.
+ */
+ add_column_to_pathtarget(target, (Expr *) var, sortgroupref);
+ add_column_to_pathtarget(agg_input, (Expr *) var, sortgroupref);
+ }
+ else
+ {
+ /*
+ * Since root->parse->groupClause is not supposed to contain
+ * this expression, we need to construct special
+ * SortGroupClause. Its tleSortGroupRef needs to be unique
+ * within target_agg, so postpone creation of the
+ * SortGroupRefs until we're done with the iteration of
+ * rel->reltarget->exprs.
+ */
+ *group_exprs_extra_p = lappend(*group_exprs_extra_p, var);
+ }
+ }
+ else
+ {
+ /*
+ * As long as the query is semantically correct, arriving here
+ * means that the var is referenced by a generic grouping
+ * expression but not referenced by any join.
+ *
+ * create_rel_agg_info() should add this variable to "agg_input"
+ * target and also add the whole generic expression to "target",
+ * but that's subject to future enhancement of the aggregate
+ * push-down feature.
+ */
+ return false;
+ }
+ }
+
+ return true;
+}
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
index d0cc14f11d..eaa57fea0f 100644
--- a/src/backend/optimizer/util/tlist.c
+++ b/src/backend/optimizer/util/tlist.c
@@ -426,7 +426,6 @@ get_sortgrouplist_exprs(List *sgClauses, List *targetList)
return result;
}
-
/*****************************************************************************
* Functions to extract data from a list of SortGroupClauses
*
@@ -801,6 +800,62 @@ apply_pathtarget_labeling_to_tlist(List *tlist, PathTarget *target)
}
/*
+ * For each aggregate grouping expression or aggregate to the grouped target.
+ *
+ * Caller passes the expressions in the form of GroupedVarInfos so that we
+ * don't have to look for gvid.
+ */
+void
+add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions)
+{
+ ListCell *lc;
+
+ /* Create the vars and add them to the target. */
+ foreach(lc, expressions)
+ {
+ GroupedVarInfo *gvi;
+
+ gvi = lfirst_node(GroupedVarInfo, lc);
+ add_column_to_pathtarget(target, (Expr *) gvi->agg_partial,
+ gvi->sortgroupref);
+ }
+}
+
+/*
+ * Find out if expr can be used as grouping expression in reltarget.
+ *
+ * sortgroupref and is_derived reflect the ->sortgroupref and ->derived fields
+ * of the corresponding GroupedVarInfo.
+ */
+bool
+is_grouping_expression(List *gvis, Expr *expr, Index *sortgroupref,
+ bool *is_derived)
+{
+ ListCell *lc;
+
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (equal(gvi->gvexpr, expr))
+ {
+ Assert(gvi->sortgroupref > 0);
+
+ *sortgroupref = gvi->sortgroupref;
+ *is_derived = gvi->derived;
+ return true;
+ }
+ }
+
+ /* The expression cannot be used as grouping key. */
+ return false;
+}
+
+/*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
*
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index be059f0de0..22f7e1222b 100644
--- a/src/backend/utils/adt/selfuncs.c
+++ b/src/backend/utils/adt/selfuncs.c
@@ -4876,8 +4876,12 @@ examine_variable(PlannerInfo *root, Node *node, int varRelid,
case BMS_MULTIPLE:
if (varRelid == 0)
{
+ RelOptInfoSet *relset;
+
/* treat it as a variable of a join relation */
- vardata->rel = find_join_rel(root, varnos);
+ relset = find_join_rel(root, varnos);
+ if (relset)
+ vardata->rel = relset->rel_plain;
node = basenode; /* strip any relabeling */
}
else if (bms_is_member(varRelid, varnos))
@@ -5735,7 +5739,13 @@ find_join_input_rel(PlannerInfo *root, Relids relids)
rel = find_base_rel(root, bms_singleton_member(relids));
break;
case BMS_MULTIPLE:
- rel = find_join_rel(root, relids);
+ {
+ RelOptInfoSet *relset;
+
+ relset = find_join_rel(root, relids);
+ if (relset)
+ rel = relset->rel_plain;
+ }
break;
}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index ae925c1650..15a0b194c5 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -986,6 +986,15 @@ static struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
{
+ {"enable_agg_pushdown", PGC_USERSET, QUERY_TUNING_METHOD,
+ gettext_noop("Enables aggregation push-down."),
+ NULL
+ },
+ &enable_agg_pushdown,
+ false,
+ NULL, NULL, NULL
+ },
+ {
{"enable_parallel_append", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables the planner's use of parallel append plans."),
NULL
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 10dac60cd3..d88fd6d71d 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -223,6 +223,8 @@ typedef enum NodeTag
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_RelAggInfo,
+ T_RelOptInfoSet,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
@@ -266,6 +268,7 @@ typedef enum NodeTag
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/relation.h b/src/include/nodes/relation.h
index 3430061361..4623d8b761 100644
--- a/src/include/nodes/relation.h
+++ b/src/include/nodes/relation.h
@@ -278,6 +278,8 @@ typedef struct PlannerInfo
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
@@ -290,7 +292,7 @@ typedef struct PlannerInfo
List *part_schemes; /* Canonicalised partition schemes used in the
* query. */
- List *initial_rels; /* RelOptInfos we are now trying to join */
+ List *initial_rels; /* RelOptInfoSets we are now trying to join */
/* Use fetch_upper_rel() to get any particular upper rel */
List *upper_rels[UPPERREL_FINAL + 1]; /* upper-rel RelOptInfos */
@@ -304,6 +306,12 @@ typedef struct PlannerInfo
*/
List *processed_tlist;
+ /*
+ * The maximum ressortgroupref among target entries in processed_list.
+ * Useful when adding extra grouping expressions for partial aggregation.
+ */
+ int max_sortgroupref;
+
/* Fields filled during create_plan() for use in setrefs.c */
AttrNumber *grouping_map; /* for GroupingFunc fixup */
List *minmax_aggs; /* List of MinMaxAggInfos */
@@ -729,6 +737,82 @@ typedef struct RelOptInfo
(rel)->part_rels && (rel)->partexprs && (rel)->nullable_partexprs)
/*
+ * RelAggInfo
+ *
+ * RelOptInfo needs information contained here if its paths should be
+ * aggregated.
+ *
+ * "target" will be used as pathtarget for aggregation if "explicit
+ * aggregation" is applied to base relation or join. The same target will also
+ * --- if the relation is a join --- be used to joinin grouped path to a
+ * non-grouped one.
+ *
+ * These targets contain plain-Var grouping expressions and Aggrefs which.
+ * Once Aggref is evaluated, its value is passed to the upper paths w/o being
+ * evaluated again.
+ *
+ * Note: There's a convention that Aggref expressions are supposed to follow
+ * the other expressions of the target. Iterations of ->exprs may rely on this
+ * arrangement.
+ *
+ * "input" contains Vars used either as grouping expressions or aggregate
+ * arguments. Paths providing the aggregation plan with input data should use
+ * this target.
+ *
+ * "input_rows" is the estimated number of input rows for AggPath. It's
+ * actually just a workspace for users of the structure, i.e. not initialized
+ * when instance of the structure is created.
+ *
+ * "group_clauses" and "group_exprs" are lists of SortGroupClause and the
+ * corresponding grouping expressions respectively.
+ *
+ * "agg_exprs" is a list of Aggref nodes for the aggregation of the relation's
+ * paths.
+ */
+typedef struct RelAggInfo
+{
+ NodeTag type;
+
+ struct PathTarget *target; /* Target for grouped paths.. */
+
+ struct PathTarget *input; /* pathtarget of paths that generate input for
+ * aggregation paths. */
+ double input_rows;
+
+ List *group_clauses;
+ List *group_exprs;
+
+ List *agg_exprs; /* Aggref expressions. */
+} RelAggInfo;
+
+/*
+ * RelOptInfoSet
+ *
+ * Structure to use where RelOptInfo for other UpperRelationKind than
+ * UPPERREL_FINAL may be needed. Currently we only need UPPERREL_FINAL
+ * and UPPERREL_PARTIAL_GROUP_AGG.
+ *
+ * rel_plain should always be initialized. Code paths that do not
+ * distinguish between plain and grouped relation use rel_plain,
+ * e.g. has_legal_joinclause().
+ *
+ * agg_info is initialized iff rel_grouped is.
+ *
+ * XXX Is RelOptInfoPair more appropriate name?
+ */
+typedef struct RelOptInfoSet
+{
+ NodeTag type;
+
+ RelOptInfo *rel_plain; /* UPPERREL_FINAL */
+ RelOptInfo *rel_grouped; /* UPPERREL_PARTIAL_GROUP_AGG */
+
+ RelAggInfo *agg_info; /* Information needed to create rel_grouped
+ * and its paths. It seems just convenient to
+ * store it here. */
+} RelOptInfoSet;
+
+/*
* IndexOptInfo
* Per-index information for planning/optimization
*
@@ -2205,6 +2289,26 @@ typedef struct PlaceHolderInfo
} PlaceHolderInfo;
/*
+ * GroupedVarInfo exists for each expression that can be used as an aggregate
+ * or grouping expression evaluated below a join.
+ */
+typedef struct GroupedVarInfo
+{
+ NodeTag type;
+
+ Expr *gvexpr; /* the represented expression. */
+ Aggref *agg_partial; /* if gvexpr is aggregate, agg_partial is the
+ * corresponding partial aggregate */
+ Index sortgroupref; /* If gvexpr is a grouping expression, this is
+ * the tleSortGroupRef of the corresponding
+ * SortGroupClause. */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+ bool derived; /* derived from another GroupedVarInfo using
+ * equeivalence classes? */
+} GroupedVarInfo;
+
+/*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
* path, the list is stored into root->minmax_aggs for use during setrefs.c.
@@ -2311,6 +2415,11 @@ typedef struct SemiAntiJoinFactors
* sjinfo is extra info about special joins for selectivity estimation
* semifactors is as shown above (only valid for SEMI/ANTI/inner_unique joins)
* param_source_rels are OK targets for parameterization of result paths
+ * agg_info passes information necessary for joins to produce partially
+ * grouped data.
+ * rel_agg_input describes the AggPath input relation if the join output
+ * should be aggregated. If NULL is passed, do not aggregate the join
+ * output.
*/
typedef struct JoinPathExtraData
{
@@ -2320,6 +2429,8 @@ typedef struct JoinPathExtraData
SpecialJoinInfo *sjinfo;
SemiAntiJoinFactors semifactors;
Relids param_source_rels;
+ RelAggInfo *agg_info;
+ RelOptInfo *rel_agg_input;
} JoinPathExtraData;
/*
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
index 5c8580e478..64b5e94048 100644
--- a/src/include/optimizer/clauses.h
+++ b/src/include/optimizer/clauses.h
@@ -88,4 +88,6 @@ extern Query *inline_set_returning_function(PlannerInfo *root,
extern List *expand_function_arguments(List *args, Oid result_type,
HeapTuple func_tuple);
+extern GroupedVarInfo *translate_expression_to_rels(PlannerInfo *root,
+ GroupedVarInfo *gvi, Index relid);
#endif /* CLAUSES_H */
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
index e7005b4a0c..32dffc7cc7 100644
--- a/src/include/optimizer/cost.h
+++ b/src/include/optimizer/cost.h
@@ -72,6 +72,7 @@ extern PGDLLIMPORT bool enable_partitionwise_aggregate;
extern PGDLLIMPORT bool enable_parallel_append;
extern PGDLLIMPORT bool enable_parallel_hash;
extern PGDLLIMPORT bool enable_partition_pruning;
+extern PGDLLIMPORT bool enable_agg_pushdown;
extern PGDLLIMPORT int constraint_exclusion;
extern double clamp_row_est(double nrows);
@@ -174,7 +175,8 @@ extern void compute_semi_anti_join_factors(PlannerInfo *root,
SpecialJoinInfo *sjinfo,
List *restrictlist,
SemiAntiJoinFactors *semifactors);
-extern void set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel);
+extern void set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel,
+ RelAggInfo *agg_info);
extern double get_parameterized_baserel_size(PlannerInfo *root,
RelOptInfo *rel,
List *param_clauses);
diff --git a/src/include/optimizer/geqo.h b/src/include/optimizer/geqo.h
index dc88fbbc1b..701aeac7bb 100644
--- a/src/include/optimizer/geqo.h
+++ b/src/include/optimizer/geqo.h
@@ -78,7 +78,7 @@ typedef struct
/* routines in geqo_main.c */
-extern RelOptInfo *geqo(PlannerInfo *root,
+extern RelOptInfoSet *geqo(PlannerInfo *root,
int number_of_rels, List *initial_rels);
/* routines in geqo_eval.c */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index bd905d3328..f420a2e13a 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -35,7 +35,8 @@ extern bool add_partial_path_precheck(RelOptInfo *parent_rel,
Cost total_cost, List *pathkeys);
extern Path *create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
- Relids required_outer, int parallel_workers);
+ Relids required_outer, int parallel_workers,
+ RelOptInfo *rel_grouped, RelAggInfo *agg_info);
extern Path *create_samplescan_path(PlannerInfo *root, RelOptInfo *rel,
Relids required_outer);
extern IndexPath *create_index_path(PlannerInfo *root,
@@ -49,7 +50,9 @@ extern IndexPath *create_index_path(PlannerInfo *root,
bool indexonly,
Relids required_outer,
double loop_count,
- bool partial_path);
+ bool partial_path,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
extern BitmapHeapPath *create_bitmap_heap_path(PlannerInfo *root,
RelOptInfo *rel,
Path *bitmapqual,
@@ -123,6 +126,7 @@ extern Relids calc_non_nestloop_required_outer(Path *outer_path, Path *inner_pat
extern NestPath *create_nestloop_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -134,6 +138,7 @@ extern NestPath *create_nestloop_path(PlannerInfo *root,
extern MergePath *create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -148,6 +153,7 @@ extern MergePath *create_mergejoin_path(PlannerInfo *root,
extern HashPath *create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -196,6 +202,12 @@ extern AggPath *create_agg_path(PlannerInfo *root,
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+extern AggPath *create_agg_sorted_path(PlannerInfo *root,
+ Path *subpath,
+ RelAggInfo *agg_info);
+extern AggPath *create_agg_hashed_path(PlannerInfo *root,
+ Path *subpath,
+ RelAggInfo *agg_info);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
@@ -263,14 +275,16 @@ extern void setup_simple_rel_arrays(PlannerInfo *root);
extern void setup_append_rel_array(PlannerInfo *root);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
+extern void build_simple_grouped_rel(PlannerInfo *root, RelOptInfoSet *rel);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
-extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
+extern RelOptInfoSet *find_join_rel(PlannerInfo *root, Relids relids);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr);
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info);
extern Relids min_join_parameterization(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
@@ -297,5 +311,5 @@ extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo, JoinType jointype);
-
+extern RelAggInfo *create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index 666217c189..1df5dfebad 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -21,6 +21,7 @@
* allpaths.c
*/
extern PGDLLIMPORT bool enable_geqo;
+extern PGDLLIMPORT bool enable_agg_pushdown;
extern PGDLLIMPORT int geqo_threshold;
extern PGDLLIMPORT int min_parallel_table_scan_size;
extern PGDLLIMPORT int min_parallel_index_scan_size;
@@ -29,7 +30,9 @@ extern PGDLLIMPORT int min_parallel_index_scan_size;
typedef void (*set_rel_pathlist_hook_type) (PlannerInfo *root,
RelOptInfo *rel,
Index rti,
- RangeTblEntry *rte);
+ RangeTblEntry *rte,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
extern PGDLLIMPORT set_rel_pathlist_hook_type set_rel_pathlist_hook;
/* Hook for plugins to get control in add_paths_to_joinrel() */
@@ -42,19 +45,24 @@ typedef void (*set_join_pathlist_hook_type) (PlannerInfo *root,
extern PGDLLIMPORT set_join_pathlist_hook_type set_join_pathlist_hook;
/* Hook for plugins to replace standard_join_search() */
-typedef RelOptInfo *(*join_search_hook_type) (PlannerInfo *root,
- int levels_needed,
- List *initial_rels);
+typedef RelOptInfoSet *(*join_search_hook_type) (PlannerInfo *root,
+ int levels_needed,
+ List *initial_rels);
extern PGDLLIMPORT join_search_hook_type join_search_hook;
-extern RelOptInfo *make_one_rel(PlannerInfo *root, List *joinlist);
+extern RelOptInfoSet *make_one_rel(PlannerInfo *root, List *joinlist);
extern void set_dummy_rel_pathlist(RelOptInfo *rel);
-extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
+extern RelOptInfoSet *standard_join_search(PlannerInfo *root,
+ int levels_needed,
List *initial_rels);
extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
+
+extern bool add_grouped_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, AggStrategy aggstrategy,
+ RelAggInfo *agg_info);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages, int max_workers);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
@@ -70,7 +78,8 @@ extern void debug_print_rel(PlannerInfo *root, RelOptInfo *rel);
* indxpath.c
* routines to generate index paths
*/
-extern void create_index_paths(PlannerInfo *root, RelOptInfo *rel);
+extern void create_index_paths(PlannerInfo *root, RelOptInfo *rel,
+ RelOptInfo *rel_grouped, RelAggInfo *agg_info);
extern bool relation_has_unique_index_for(PlannerInfo *root, RelOptInfo *rel,
List *restrictlist,
List *exprlist, List *oprlist);
@@ -101,7 +110,9 @@ extern void create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel);
extern void add_paths_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, SpecialJoinInfo *sjinfo,
- List *restrictlist);
+ List *restrictlist,
+ RelAggInfo *agg_info,
+ RelOptInfo *rel_agg_input);
/*
* joinrels.c
@@ -109,7 +120,7 @@ extern void add_paths_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
*/
extern void join_search_one_level(PlannerInfo *root, int level);
extern RelOptInfo *make_join_rel(PlannerInfo *root,
- RelOptInfo *rel1, RelOptInfo *rel2);
+ RelOptInfoSet *relset1, RelOptInfoSet *relset2);
extern bool have_join_order_restriction(PlannerInfo *root,
RelOptInfo *rel1, RelOptInfo *rel2);
extern bool have_dangerous_phv(PlannerInfo *root,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index bec0c38617..dc050aec4f 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -37,7 +37,7 @@ typedef void (*query_pathkeys_callback) (PlannerInfo *root, void *extra);
/*
* prototypes for plan/planmain.c
*/
-extern RelOptInfo *query_planner(PlannerInfo *root, List *tlist,
+extern RelOptInfoSet *query_planner(PlannerInfo *root, List *tlist,
query_pathkeys_callback qp_callback, void *qp_extra);
/*
@@ -78,6 +78,7 @@ extern void add_base_rels_to_query(PlannerInfo *root, Node *jtnode);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
Relids where_needed, bool create_new_ph);
+extern void setup_aggregate_pushdown(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
index 8b967f9583..b39ef1ac1c 100644
--- a/src/include/optimizer/tlist.h
+++ b/src/include/optimizer/tlist.h
@@ -16,7 +16,6 @@
#include "nodes/relation.h"
-
extern TargetEntry *tlist_member(Expr *node, List *targetlist);
extern TargetEntry *tlist_member_ignore_relabel(Expr *node, List *targetlist);
@@ -41,7 +40,6 @@ extern Node *get_sortgroupclause_expr(SortGroupClause *sgClause,
List *targetList);
extern List *get_sortgrouplist_exprs(List *sgClauses,
List *targetList);
-
extern SortGroupClause *get_sortgroupref_clause(Index sortref,
List *clauses);
extern SortGroupClause *get_sortgroupref_clause_noerr(Index sortref,
@@ -65,6 +63,13 @@ extern void split_pathtarget_at_srfs(PlannerInfo *root,
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+/* TODO Find the best location (position and in some cases even file) for the
+ * following ones. */
+extern void add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions);
+extern bool is_grouping_expression(List *gvis, Expr *expr,
+ Index *sortgroupref, bool *is_derived);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
new file mode 100644
index 0000000000..b3a97f86d6
--- /dev/null
+++ b/src/test/regress/expected/agg_pushdown.out
@@ -0,0 +1,217 @@
+BEGIN;
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+CREATE INDEX ON agg_pushdown_child1(parent);
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+COMMIT;
+ANALYZE;
+SET enable_agg_pushdown TO on;
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+--------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(10 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(11 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (p.i = c1.parent)
+ -> Sort
+ Sort Key: p.i
+ -> Seq Scan on agg_pushdown_parent p
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(12 rows)
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(8 rows)
+
+SET enable_seqscan TO on;
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.x = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(11 rows)
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: ((k = c1.j) AND (parent = c1.parent))
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(13 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+----------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Hash
+ -> Seq Scan on agg_pushdown_child2 c2
+(15 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: ((c1.j = c2.k) AND (c1.parent = c2.parent))
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(13 rows)
+
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index cc0bbf5db9..74a22fd83b 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -98,6 +98,9 @@ test: rules psql_crosstab amutils
test: select_parallel
test: write_parallel
+# this one runs parallel workers too
+test: agg_pushdown
+
# no relation related tests can be put in this group
test: publication subscription
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 0c10c7100c..af387fd9e3 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -135,6 +135,7 @@ test: rules
test: psql_crosstab
test: select_parallel
test: write_parallel
+test: agg_pushdown
test: publication
test: subscription
test: amutils
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
new file mode 100644
index 0000000000..6cf2ed9c20
--- /dev/null
+++ b/src/test/regress/sql/agg_pushdown.sql
@@ -0,0 +1,117 @@
+BEGIN;
+
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+
+CREATE INDEX ON agg_pushdown_child1(parent);
+
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+COMMIT;
+ANALYZE;
+
+SET enable_agg_pushdown TO on;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_seqscan TO on;
+
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
Re: [HACKERS] WIP: Aggregation push-down
On Tue, Jan 15, 2019 at 03:06:18PM +0100, Antonin Houska wrote:
This is the next version. A few more comments below.
Latest patch set fails to apply, so moved to next CF, waiting on
author.
--
Michael
Re: [HACKERS] WIP: Aggregation push-down
Michael Paquier <michael@paquier.xyz> wrote:
On Tue, Jan 15, 2019 at 03:06:18PM +0100, Antonin Houska wrote:
This is the next version. A few more comments below.
Latest patch set fails to apply, so moved to next CF, waiting on
author.
Rebased.
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com
Attachments:
v10-001-Export_estimate_hashagg_tablesize.patchtext/x-diffDownload
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index b849ae03b8..6f3a4a063c 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -146,9 +146,6 @@ static double get_number_of_groups(PlannerInfo *root,
double path_rows,
grouping_sets_data *gd,
List *target_list);
-static Size estimate_hashagg_tablesize(Path *path,
- const AggClauseCosts *agg_costs,
- double dNumGroups);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
PathTarget *target,
@@ -3634,40 +3631,6 @@ get_number_of_groups(PlannerInfo *root,
}
/*
- * estimate_hashagg_tablesize
- * estimate the number of bytes that a hash aggregate hashtable will
- * require based on the agg_costs, path width and dNumGroups.
- *
- * XXX this may be over-estimating the size now that hashagg knows to omit
- * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
- * grouping columns not in the hashed set are counted here even though hashagg
- * won't store them. Is this a problem?
- */
-static Size
-estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
- double dNumGroups)
-{
- Size hashentrysize;
-
- /* Estimate per-hash-entry space at tuple width... */
- hashentrysize = MAXALIGN(path->pathtarget->width) +
- MAXALIGN(SizeofMinimalTupleHeader);
-
- /* plus space for pass-by-ref transition values... */
- hashentrysize += agg_costs->transitionSpace;
- /* plus the per-hash-entry overhead */
- hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
-
- /*
- * Note that this disregards the effect of fill-factor and growth policy
- * of the hash-table. That's probably ok, given default the default
- * fill-factor is relatively high. It'd be hard to meaningfully factor in
- * "double-in-size" growth policies here.
- */
- return hashentrysize * dNumGroups;
-}
-
-/*
* create_grouping_paths
*
* Build a new upperrel containing Paths for grouping and/or aggregation.
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index e8f51d2d0d..be059f0de0 100644
--- a/src/backend/utils/adt/selfuncs.c
+++ b/src/backend/utils/adt/selfuncs.c
@@ -114,6 +114,7 @@
#include "catalog/pg_statistic_ext.h"
#include "catalog/pg_type.h"
#include "executor/executor.h"
+#include "executor/nodeAgg.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
@@ -3902,6 +3903,39 @@ estimate_hash_bucket_stats(PlannerInfo *root, Node *hashkey, double nbuckets,
ReleaseVariableStats(vardata);
}
+/*
+ * estimate_hashagg_tablesize
+ * estimate the number of bytes that a hash aggregate hashtable will
+ * require based on the agg_costs, path width and dNumGroups.
+ *
+ * XXX this may be over-estimating the size now that hashagg knows to omit
+ * unneeded columns from the hashtable. Also for mixed-mode grouping sets,
+ * grouping columns not in the hashed set are counted here even though hashagg
+ * won't store them. Is this a problem?
+ */
+Size
+estimate_hashagg_tablesize(Path *path, const AggClauseCosts *agg_costs,
+ double dNumGroups)
+{
+ Size hashentrysize;
+
+ /* Estimate per-hash-entry space at tuple width... */
+ hashentrysize = MAXALIGN(path->pathtarget->width) +
+ MAXALIGN(SizeofMinimalTupleHeader);
+
+ /* plus space for pass-by-ref transition values... */
+ hashentrysize += agg_costs->transitionSpace;
+ /* plus the per-hash-entry overhead */
+ hashentrysize += hash_agg_entry_size(agg_costs->numAggs);
+
+ /*
+ * Note that this disregards the effect of fill-factor and growth policy
+ * of the hash-table. That's probably ok, given default the default
+ * fill-factor is relatively high. It'd be hard to meaningfully factor in
+ * "double-in-size" growth policies here.
+ */
+ return hashentrysize * dNumGroups;
+}
/*-------------------------------------------------------------------------
*
diff --git a/src/include/utils/selfuncs.h b/src/include/utils/selfuncs.h
index 5cc4cf15e2..e65ab13b1a 100644
--- a/src/include/utils/selfuncs.h
+++ b/src/include/utils/selfuncs.h
@@ -213,6 +213,9 @@ extern void estimate_hash_bucket_stats(PlannerInfo *root,
Node *hashkey, double nbuckets,
Selectivity *mcv_freq,
Selectivity *bucketsize_frac);
+extern Size estimate_hashagg_tablesize(Path *path,
+ const AggClauseCosts *agg_costs,
+ double dNumGroups);
extern List *deconstruct_indexquals(IndexPath *path);
extern void genericcostestimate(PlannerInfo *root, IndexPath *path,
v10-002-Introduce_make_join_rel_common.patchtext/x-diffDownload
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 38eeb23d81..0caf67a916 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -37,6 +37,7 @@ static bool is_dummy_rel(RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
+static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist);
@@ -651,21 +652,12 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
return true;
}
-
/*
- * make_join_rel
- * Find or create a join RelOptInfo that represents the join of
- * the two given rels, and add to it path information for paths
- * created with the two rels as outer and inner rel.
- * (The join rel may already contain paths generated from other
- * pairs of rels that add up to the same set of base rels.)
- *
- * NB: will return NULL if attempted join is not valid. This can happen
- * when working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
+ * make_join_rel_common
+ * The workhorse of make_join_rel().
*/
-RelOptInfo *
-make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+static RelOptInfo *
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -748,6 +740,24 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
}
/*
+ * make_join_rel
+ * Find or create a join RelOptInfo that represents the join of
+ * the two given rels, and add to it path information for paths
+ * created with the two rels as outer and inner rel.
+ * (The join rel may already contain paths generated from other
+ * pairs of rels that add up to the same set of base rels.)
+ *
+ * NB: will return NULL if attempted join is not valid. This can happen
+ * when working with outer joins, or with IN or EXISTS clauses that have been
+ * turned into joins.
+ */
+RelOptInfo *
+make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+{
+ return make_join_rel_common(root, rel1, rel2);
+}
+
+/*
* populate_joinrel_with_paths
* Add paths to the given joinrel for given pair of joining relations. The
* SpecialJoinInfo provides details about the join and the restrictlist
v10-003-Introduce_estimate_join_rows.patchtext/x-diffDownload
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 99c5ad9b4a..353dc116f4 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -2291,6 +2291,20 @@ cost_group(Path *path, PlannerInfo *root,
}
/*
+ * estimate_join_rows
+ * Set rows of a join path according to its parent relation or according
+ * to parameters.
+ */
+static void
+estimate_join_rows(PlannerInfo *root, Path *path)
+{
+ if (path->param_info)
+ path->rows = path->param_info->ppi_rows;
+ else
+ path->rows = path->parent->rows;
+}
+
+/*
* initial_cost_nestloop
* Preliminary estimate of the cost of a nestloop join path.
*
@@ -2411,10 +2425,7 @@ final_cost_nestloop(PlannerInfo *root, NestPath *path,
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
- if (path->path.param_info)
- path->path.rows = path->path.param_info->ppi_rows;
- else
- path->path.rows = path->path.parent->rows;
+ estimate_join_rows(root, (Path *) path);
/* For partial paths, scale row estimate. */
if (path->path.parallel_workers > 0)
@@ -2857,10 +2868,7 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
- if (path->jpath.path.param_info)
- path->jpath.path.rows = path->jpath.path.param_info->ppi_rows;
- else
- path->jpath.path.rows = path->jpath.path.parent->rows;
+ estimate_join_rows(root, (Path *) path);
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
@@ -3287,10 +3295,7 @@ final_cost_hashjoin(PlannerInfo *root, HashPath *path,
ListCell *hcl;
/* Mark the path with the correct row estimate */
- if (path->jpath.path.param_info)
- path->jpath.path.rows = path->jpath.path.param_info->ppi_rows;
- else
- path->jpath.path.rows = path->jpath.path.parent->rows;
+ estimate_join_rows(root, (Path *) path);
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
v10-004-Agg_pushdown_basic.patchtext/x-diffDownload
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 7fcac81e2e..5752eba563 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -665,7 +665,7 @@ postgresGetForeignRelSize(PlannerInfo *root,
}
/* Estimate baserel size as best we can with local statistics. */
- set_baserel_size_estimates(root, baserel);
+ set_baserel_size_estimates(root, baserel, NULL);
/* Fill in basically-bogus cost estimates for use later. */
estimate_path_cost_size(root, baserel, NIL, NIL,
@@ -2068,7 +2068,10 @@ postgresPlanDirectModify(PlannerInfo *root,
/* Safe to fetch data about the target foreign rel */
if (fscan->scan.scanrelid == 0)
{
- foreignrel = find_join_rel(root, fscan->fs_relids);
+ RelOptInfoSet *foreignrelset;
+
+ foreignrelset = find_join_rel(root, fscan->fs_relids);
+ foreignrel = foreignrelset->rel_plain;
/* We should have a rel for this foreign join. */
Assert(foreignrel);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index b44ead269f..eabea5dce1 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2198,8 +2198,8 @@ _copyOnConflictExpr(const OnConflictExpr *from)
/* ****************************************************************
* pathnodes.h copy functions
*
- * We don't support copying RelOptInfo, IndexOptInfo, or Path nodes.
- * There are some subsidiary structs that are useful to copy, though.
+ * We don't support copying RelOptInfo, IndexOptInfo, RelAggInfo or Path
+ * nodes. There are some subsidiary structs that are useful to copy, though.
* ****************************************************************
*/
@@ -2340,6 +2340,20 @@ _copyPlaceHolderInfo(const PlaceHolderInfo *from)
return newnode;
}
+static GroupedVarInfo *
+_copyGroupedVarInfo(const GroupedVarInfo *from)
+{
+ GroupedVarInfo *newnode = makeNode(GroupedVarInfo);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gv_eval_at);
+ COPY_SCALAR_FIELD(derived);
+
+ return newnode;
+}
+
/* ****************************************************************
* parsenodes.h copy functions
* ****************************************************************
@@ -5108,6 +5122,9 @@ copyObjectImpl(const void *from)
case T_PlaceHolderInfo:
retval = _copyPlaceHolderInfo(from);
break;
+ case T_GroupedVarInfo:
+ retval = _copyGroupedVarInfo(from);
+ break;
/*
* VALUE NODES
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f97cf37f1f..a859d60f43 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2198,6 +2198,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(append_rel_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
@@ -2205,6 +2206,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(distinct_pathkeys);
WRITE_NODE_FIELD(sort_pathkeys);
WRITE_NODE_FIELD(processed_tlist);
+ WRITE_INT_FIELD(max_sortgroupref);
WRITE_NODE_FIELD(minmax_aggs);
WRITE_FLOAT_FIELD(total_table_pages, "%.0f");
WRITE_FLOAT_FIELD(tuple_fraction, "%.4f");
@@ -2415,6 +2417,22 @@ _outParamPathInfo(StringInfo str, const ParamPathInfo *node)
WRITE_NODE_FIELD(ppi_clauses);
}
+/*
+ * Note we do NOT print plain_rel, else we'd be in infinite recursion.
+ */
+static void
+_outRelAggInfo(StringInfo str, const RelAggInfo *node)
+{
+ WRITE_NODE_TYPE("RELAGGINFO");
+
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(input);
+ WRITE_FLOAT_FIELD(input_rows, "%.0f");
+ WRITE_NODE_FIELD(group_clauses);
+ WRITE_NODE_FIELD(group_exprs);
+ WRITE_NODE_FIELD(agg_exprs);
+}
+
static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
@@ -2503,6 +2521,18 @@ _outPlaceHolderInfo(StringInfo str, const PlaceHolderInfo *node)
}
static void
+_outGroupedVarInfo(StringInfo str, const GroupedVarInfo *node)
+{
+ WRITE_NODE_TYPE("GROUPEDVARINFO");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_BITMAPSET_FIELD(gv_eval_at);
+ WRITE_BOOL_FIELD(derived);
+}
+
+static void
_outMinMaxAggInfo(StringInfo str, const MinMaxAggInfo *node)
{
WRITE_NODE_TYPE("MINMAXAGGINFO");
@@ -4041,6 +4071,9 @@ outNode(StringInfo str, const void *obj)
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_RelAggInfo:
+ _outRelAggInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
@@ -4056,6 +4089,9 @@ outNode(StringInfo str, const void *obj)
case T_PlaceHolderInfo:
_outPlaceHolderInfo(str, obj);
break;
+ case T_GroupedVarInfo:
+ _outGroupedVarInfo(str, obj);
+ break;
case T_MinMaxAggInfo:
_outMinMaxAggInfo(str, obj);
break;
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
index 89ce373d5e..4633fb8768 100644
--- a/src/backend/optimizer/README
+++ b/src/backend/optimizer/README
@@ -1127,3 +1127,90 @@ breaking down aggregation or grouping over a partitioned relation into
aggregation or grouping over its partitions is called partitionwise
aggregation. Especially when the partition keys match the GROUP BY clause,
this can be significantly faster than the regular method.
+
+Aggregate push-down
+-------------------
+
+The obvious way to evaluate aggregates is to evaluate the FROM clause of the
+SQL query (this is what query_planner does) and use the resuing paths as the
+input of Agg node. However, if the groups are large enough, it may be more
+efficient to apply the partial aggregation to the output of base relation
+scan, and finalize it when we have all relations of the query joined:
+
+ EXPLAIN
+ SELECT a.i, avg(b.u)
+ FROM a JOIN b ON b.j = a.i
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+ -> Index Only Scan using a_pkey on a
+ Index Cond: (i = b.j)
+
+Thus the join above the partial aggregate node receives fewer input rows, and
+so the number of outer-to-inner pairs of tuples to be checked can be
+significantly lower, which can in turn lead to considerably lower join cost.
+
+Note that there's often no GROUP BY expression to be used for the partial
+aggregation, so we use equivalence classes to derive grouping expression: in
+the example above, the grouping key "b.j" was derived from "a.i".
+
+Furthermore, extra grouping columns can be added to the partial Agg node if a
+join clause above that node references a column which is not in the query
+GROUP BY clause and which could not be derived using equivalence class.
+
+ EXPLAIN
+ SELECT a.i, avg(b.u)
+ FROM a JOIN b ON b.j = a.x
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (a.x = b.j)
+ -> Seq Scan on a
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+
+Here the partial aggregate uses "b.j" as grouping column although it's not in
+the same equivalence class as "a.i". Note that no column of the {a.x, b.j}
+equivalence class is used as a key for the final aggregation.
+
+Besides base relation, the aggregation can also be pushed down to join:
+
+ EXPLAIN
+ SELECT a.i, avg(b.u + c.v)
+ FROM a JOIN b ON b.j = a.i
+ JOIN c ON c.k = a.i
+ WHERE b.j = c.k GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (b.j = a.i)
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Hash Join
+ Hash Cond: (b.j = c.k)
+ -> Seq Scan on b
+ -> Hash
+ -> Seq Scan on c
+ -> Hash
+ -> Seq Scan on a
+
+Whether the Agg node is created out of base relation or out of join, it's
+added to a separate RelOptInfo that we call "grouped relation". Grouped
+relation can be joined to a non-grouped relation, which results in a grouped
+relation too. Join of two grouped relations does not seem to be very useful
+and is currently not supported.
+
+If query_planner produces a grouped relation that contains valid paths, these
+are simply added to the UPPERREL_PARTIAL_GROUP_AGG relation. Further
+processing of these paths then does not differ from processing of other
+partially grouped paths.
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index e07bab831e..4beac45fdf 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -182,13 +182,15 @@ gimme_tree(PlannerInfo *root, Gene *tour, int num_gene)
for (rel_count = 0; rel_count < num_gene; rel_count++)
{
int cur_rel_index;
+ RelOptInfoSet *cur_relset;
RelOptInfo *cur_rel;
Clump *cur_clump;
/* Get the next input relation */
cur_rel_index = (int) tour[rel_count];
- cur_rel = (RelOptInfo *) list_nth(private->initial_rels,
- cur_rel_index - 1);
+ cur_relset = (RelOptInfoSet *) list_nth(private->initial_rels,
+ cur_rel_index - 1);
+ cur_rel = cur_relset->rel_plain;
/* Make it into a single-rel clump */
cur_clump = (Clump *) palloc(sizeof(Clump));
@@ -251,16 +253,24 @@ merge_clump(PlannerInfo *root, List *clumps, Clump *new_clump, int num_gene,
desirable_join(root, old_clump->joinrel, new_clump->joinrel))
{
RelOptInfo *joinrel;
+ RelOptInfoSet *oldset,
+ *newset;
/*
* Construct a RelOptInfo representing the join of these two input
* relations. Note that we expect the joinrel not to exist in
* root->join_rel_list yet, and so the paths constructed for it
* will only include the ones we want.
+ *
+ * TODO Consider using make_join_rel_common() (possibly renamed)
+ * here instead of wrapping the joinrels into RelOptInfoSet.
*/
- joinrel = make_join_rel(root,
- old_clump->joinrel,
- new_clump->joinrel);
+ oldset = makeNode(RelOptInfoSet);
+ oldset->rel_plain = old_clump->joinrel;
+ newset = makeNode(RelOptInfoSet);
+ newset->rel_plain = new_clump->joinrel;
+
+ joinrel = make_join_rel(root, oldset, newset);
/* Keep searching if join order is not valid */
if (joinrel)
diff --git a/src/backend/optimizer/geqo/geqo_main.c b/src/backend/optimizer/geqo/geqo_main.c
index dc51829dec..b4a4f4ff27 100644
--- a/src/backend/optimizer/geqo/geqo_main.c
+++ b/src/backend/optimizer/geqo/geqo_main.c
@@ -63,7 +63,7 @@ static int gimme_number_generations(int pool_size);
* similar to a constrained Traveling Salesman Problem (TSP)
*/
-RelOptInfo *
+RelOptInfoSet *
geqo(PlannerInfo *root, int number_of_rels, List *initial_rels)
{
GeqoPrivateData private;
@@ -74,6 +74,7 @@ geqo(PlannerInfo *root, int number_of_rels, List *initial_rels)
Pool *pool;
int pool_size,
number_generations;
+ RelOptInfoSet *result;
#ifdef GEQO_DEBUG
int status_interval;
@@ -296,7 +297,9 @@ geqo(PlannerInfo *root, int number_of_rels, List *initial_rels)
/* ... clear root pointer to our private storage */
root->join_search_private = NULL;
- return best_rel;
+ result = makeNode(RelOptInfoSet);
+ result->rel_plain = best_rel;
+ return result;
}
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 55b871c02c..7f3ac0a6de 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -59,6 +59,7 @@ typedef struct pushdown_safety_info
/* These parameters are set by GUC */
bool enable_geqo = false; /* just in case GUC doesn't set it */
+bool enable_agg_pushdown;
int geqo_threshold;
int min_parallel_table_scan_size;
int min_parallel_index_scan_size;
@@ -74,16 +75,18 @@ static void set_base_rel_consider_startup(PlannerInfo *root);
static void set_base_rel_sizes(PlannerInfo *root);
static void set_base_rel_pathlists(PlannerInfo *root);
static void set_rel_size(PlannerInfo *root, RelOptInfo *rel,
- Index rti, RangeTblEntry *rte);
+ Index rti, RangeTblEntry *rte, RelAggInfo *agg_info);
static void set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
- Index rti, RangeTblEntry *rte);
+ Index rti, RangeTblEntry *rte,
+ RelOptInfo *rel_grouped, RelAggInfo *agg_info);
static void set_plain_rel_size(PlannerInfo *root, RelOptInfo *rel,
- RangeTblEntry *rte);
+ RangeTblEntry *rte, RelAggInfo *agg_info);
static void create_plain_partial_paths(PlannerInfo *root, RelOptInfo *rel);
static void set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
- RangeTblEntry *rte);
+ RangeTblEntry *rte, RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static void set_tablesample_rel_size(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_tablesample_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
@@ -118,10 +121,11 @@ static void set_cte_pathlist(PlannerInfo *root, RelOptInfo *rel,
static void set_namedtuplestore_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_result_pathlist(PlannerInfo *root, RelOptInfo *rel,
- RangeTblEntry *rte);
+ RangeTblEntry *rte);
static void set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
-static RelOptInfo *make_rel_from_joinlist(PlannerInfo *root, List *joinlist);
+static RelOptInfoSet *make_rel_from_joinlist(PlannerInfo *root,
+ List *joinlist);
static bool subquery_is_pushdown_safe(Query *subquery, Query *topquery,
pushdown_safety_info *safetyInfo);
static bool recurse_pushdown_safe(Node *setOp, Query *topquery,
@@ -148,10 +152,10 @@ static bool apply_child_basequals(PlannerInfo *root, RelOptInfo *rel,
* Finds all possible access paths for executing a query, returning a
* single rel that represents the join of all base rels in the query.
*/
-RelOptInfo *
+RelOptInfoSet *
make_one_rel(PlannerInfo *root, List *joinlist)
{
- RelOptInfo *rel;
+ RelOptInfoSet *rel;
Index rti;
double total_pages;
@@ -229,7 +233,7 @@ make_one_rel(PlannerInfo *root, List *joinlist)
/*
* The result should join all and only the query's base rels.
*/
- Assert(bms_equal(rel->relids, root->all_baserels));
+ Assert(bms_equal(rel->rel_plain->relids, root->all_baserels));
return rel;
}
@@ -320,7 +324,7 @@ set_base_rel_sizes(PlannerInfo *root)
if (root->glob->parallelModeOK)
set_rel_consider_parallel(root, rel, rte);
- set_rel_size(root, rel, rti, rte);
+ set_rel_size(root, rel, rti, rte, NULL);
}
}
@@ -349,18 +353,29 @@ set_base_rel_pathlists(PlannerInfo *root)
if (rel->reloptkind != RELOPT_BASEREL)
continue;
- set_rel_pathlist(root, rel, rti, root->simple_rte_array[rti]);
+ set_rel_pathlist(root, rel, rti, root->simple_rte_array[rti],
+ NULL, NULL);
}
}
/*
* set_rel_size
* Set size estimates for a base relation
+ *
+ * If "agg_info" is passed, it means that "rel" is a grouped relation for
+ * which "agg_info" provides the information needed for size estimate.
*/
static void
set_rel_size(PlannerInfo *root, RelOptInfo *rel,
- Index rti, RangeTblEntry *rte)
+ Index rti, RangeTblEntry *rte, RelAggInfo *agg_info)
{
+ bool grouped = agg_info != NULL;
+
+ /*
+ * Aggregate push-down is currently used only for relations.
+ */
+ Assert(!grouped || rte->rtekind == RTE_RELATION);
+
if (rel->reloptkind == RELOPT_BASEREL &&
relation_excluded_by_constraints(root, rel, rte))
{
@@ -379,7 +394,13 @@ set_rel_size(PlannerInfo *root, RelOptInfo *rel,
}
else if (rte->inh)
{
- /* It's an "append relation", process accordingly */
+ /*
+ * It's an "append relation", process accordingly.
+ *
+ * The aggregate push-down feature currently does not support grouped
+ * append relation.
+ */
+ Assert(!grouped);
set_append_rel_size(root, rel, rti, rte);
}
else
@@ -389,7 +410,12 @@ set_rel_size(PlannerInfo *root, RelOptInfo *rel,
case RTE_RELATION:
if (rte->relkind == RELKIND_FOREIGN_TABLE)
{
- /* Foreign table */
+ /*
+ * Foreign table
+ *
+ * Grouped foreign table is not supported.
+ */
+ Assert(!grouped);
set_foreign_size(root, rel, rte);
}
else if (rte->relkind == RELKIND_PARTITIONED_TABLE)
@@ -397,18 +423,27 @@ set_rel_size(PlannerInfo *root, RelOptInfo *rel,
/*
* A partitioned table without any partitions is marked as
* a dummy rel.
+ *
+ * Grouped partitioned table is not supported.
*/
+ Assert(!grouped);
+
set_dummy_rel_pathlist(rel);
}
else if (rte->tablesample != NULL)
{
- /* Sampled relation */
+ /*
+ * Sampled relation
+ *
+ * Grouped tablesample rel is not supported.
+ */
+ Assert(!grouped);
set_tablesample_rel_size(root, rel, rte);
}
else
{
/* Plain relation */
- set_plain_rel_size(root, rel, rte);
+ set_plain_rel_size(root, rel, rte, agg_info);
}
break;
case RTE_SUBQUERY:
@@ -464,18 +499,49 @@ set_rel_size(PlannerInfo *root, RelOptInfo *rel,
/*
* set_rel_pathlist
* Build access paths for a base relation
+ *
+ * If "rel_grouped" is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of "rel". "rel_agg_input" describes the
+ * aggregation input. "agg_info" must be passed in such a case too.
*/
static void
set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
- Index rti, RangeTblEntry *rte)
+ Index rti, RangeTblEntry *rte, RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
+ bool grouped = rel_grouped != NULL;
+
+ /*
+ * Aggregate push-down is currently implemented for RTE_RELATION only.
+ */
+ Assert(!grouped || rte->rtekind == RTE_RELATION);
+
+ if (grouped)
+ {
+ /*
+ * Do not apply AggPath if this is the only relation of the query:
+ * create_grouping_paths() will do so anyway.
+ */
+ if (bms_equal(rel->relids, root->all_baserels))
+ {
+ set_dummy_rel_pathlist(rel_grouped);
+ return;
+ }
+ }
+
if (IS_DUMMY_REL(rel))
{
/* We already proved the relation empty, so nothing more to do */
}
else if (rte->inh)
{
- /* It's an "append relation", process accordingly */
+ /*
+ * It's an "append relation", process accordingly.
+ *
+ * The aggregate push-down feature currently does not support append
+ * relation.
+ */
+ Assert(!grouped);
set_append_rel_pathlist(root, rel, rti, rte);
}
else
@@ -485,18 +551,29 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
case RTE_RELATION:
if (rte->relkind == RELKIND_FOREIGN_TABLE)
{
- /* Foreign table */
+ /*
+ * Foreign table
+ *
+ * Aggregate push-down not supported.
+ */
+ Assert(!grouped);
set_foreign_pathlist(root, rel, rte);
}
else if (rte->tablesample != NULL)
{
- /* Sampled relation */
+ /*
+ * Sampled relation.
+ *
+ * Aggregate push-down not supported.
+ */
+ Assert(!grouped);
set_tablesample_rel_pathlist(root, rel, rte);
}
else
{
/* Plain relation */
- set_plain_rel_pathlist(root, rel, rte);
+ set_plain_rel_pathlist(root, rel, rte, rel_grouped,
+ agg_info);
}
break;
case RTE_SUBQUERY:
@@ -541,9 +618,12 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* Also, if this is the topmost scan/join rel (that is, the only baserel),
* we postpone this until the final scan/join targelist is available (see
* grouping_planner).
+ *
+ * Note on aggregation push-down: parallel paths are not supported so far.
*/
if (rel->reloptkind == RELOPT_BASEREL &&
- bms_membership(root->all_baserels) != BMS_SINGLETON)
+ bms_membership(root->all_baserels) != BMS_SINGLETON &&
+ !grouped)
generate_gather_paths(root, rel, false);
/*
@@ -552,10 +632,24 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
* add_path(), or delete or modify paths added by the core code.
*/
if (set_rel_pathlist_hook)
- (*set_rel_pathlist_hook) (root, rel, rti, rte);
+ (*set_rel_pathlist_hook) (root, rel, rti, rte, rel_grouped,
+ agg_info);
+
+ /*
+ * The grouped relation is not guaranteed to have any paths. If the
+ * pathlist is empty, there's no point in calling set_cheapest().
+ */
+ if (rel_grouped && rel_grouped->pathlist == NIL)
+ {
+ set_dummy_rel_pathlist(rel_grouped);
+ return;
+ }
/* Now find the cheapest of the paths for this rel */
- set_cheapest(rel);
+ if (!grouped)
+ set_cheapest(rel);
+ else
+ set_cheapest(rel_grouped);
#ifdef OPTIMIZER_DEBUG
debug_print_rel(root, rel);
@@ -565,9 +659,13 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
/*
* set_plain_rel_size
* Set size estimates for a plain relation (no subquery, no inheritance)
+ *
+ * If "agg_info" is passed, it means that "rel" is a grouped relation for
+ * which "agg_info" provides the information needed for size estimate.
*/
static void
-set_plain_rel_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
+set_plain_rel_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte,
+ RelAggInfo *agg_info)
{
/*
* Test any partial indexes of rel for applicability. We must do this
@@ -576,7 +674,7 @@ set_plain_rel_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
check_index_predicates(root, rel);
/* Mark rel with estimated output rows, width, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, agg_info);
}
/*
@@ -757,11 +855,19 @@ set_rel_consider_parallel(PlannerInfo *root, RelOptInfo *rel,
/*
* set_plain_rel_pathlist
* Build access paths for a plain relation (no subquery, no inheritance)
+ *
+ * If "rel_grouped" is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of "rel". "rel_agg_input" describes the
+ * aggregation input. "agg_info" must be passed in such a case too.
*/
static void
-set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
+set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
+ RangeTblEntry *rte, RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
Relids required_outer;
+ Path *seq_path;
+ bool grouped = rel_grouped != NULL;
/*
* We don't support pushing join clauses into the quals of a seqscan, but
@@ -770,18 +876,38 @@ set_plain_rel_pathlist(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
*/
required_outer = rel->lateral_relids;
- /* Consider sequential scan */
- add_path(rel, create_seqscan_path(root, rel, required_outer, 0));
+ /* Consider sequential scan. */
+ seq_path = create_seqscan_path(root, rel, required_outer, 0, rel_grouped,
+ agg_info);
+
+ if (!grouped)
+ add_path(rel, seq_path);
+
+ /*
+ * It's probably not good idea to repeat hashed aggregation with different
+ * parameters, so check if there are no parameters.
+ */
+ else if (required_outer == NULL)
+ {
+ /*
+ * Only AGG_HASHED is suitable here as it does not expect the input
+ * set to be sorted.
+ */
+ add_grouped_path(root, rel_grouped, seq_path, AGG_HASHED, agg_info);
+ }
/* If appropriate, consider parallel sequential scan */
- if (rel->consider_parallel && required_outer == NULL)
+ if (rel->consider_parallel && required_outer == NULL && !grouped)
create_plain_partial_paths(root, rel);
- /* Consider index scans */
- create_index_paths(root, rel);
+ /*
+ * Consider index scans, possibly including the grouped paths.
+ */
+ create_index_paths(root, rel, rel_grouped, agg_info);
/* Consider TID scans */
- create_tidscan_paths(root, rel);
+ if (!grouped)
+ create_tidscan_paths(root, rel);
}
/*
@@ -793,15 +919,55 @@ create_plain_partial_paths(PlannerInfo *root, RelOptInfo *rel)
{
int parallel_workers;
- parallel_workers = compute_parallel_worker(rel, rel->pages, -1,
- max_parallel_workers_per_gather);
+ parallel_workers = compute_parallel_worker(rel, rel->pages, -1, max_parallel_workers_per_gather);
/* If any limit was set to zero, the user doesn't want a parallel scan. */
if (parallel_workers <= 0)
return;
/* Add an unordered partial path based on a parallel sequential scan. */
- add_partial_path(rel, create_seqscan_path(root, rel, NULL, parallel_workers));
+ add_partial_path(rel, create_seqscan_path(root, rel, NULL,
+ parallel_workers, NULL, NULL));
+}
+
+/*
+ * Apply aggregation to a subpath and add the AggPath to the pathlist.
+ *
+ * The return value tells whether the path was added to the pathlist.
+ *
+ * XXX Pass the plain rel and fetch the grouped from rel->grouped? How about
+ * subroutines then?
+ */
+bool
+add_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ AggStrategy aggstrategy, RelAggInfo *agg_info)
+{
+ Path *agg_path;
+
+ /*
+ * Repeated creation of hash table does not sound like a good idea. Caller
+ * should avoid asking us to do so.
+ */
+ Assert(subpath->param_info == NULL || aggstrategy != AGG_HASHED);
+
+ if (aggstrategy == AGG_HASHED)
+ agg_path = (Path *) create_agg_hashed_path(root, subpath,
+ agg_info);
+ else if (aggstrategy == AGG_SORTED)
+ agg_path = (Path *) create_agg_sorted_path(root, subpath,
+ agg_info);
+ else
+ elog(ERROR, "unexpected strategy %d", aggstrategy);
+
+ /* Add the grouped path to the list of grouped base paths. */
+ if (agg_path != NULL)
+ {
+ add_path(rel, (Path *) agg_path);
+
+ return true;
+ }
+
+ return false;
}
/*
@@ -841,7 +1007,7 @@ set_tablesample_rel_size(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte)
rel->tuples = tuples;
/* Mark rel with estimated output rows, width, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -1130,7 +1296,7 @@ set_append_rel_size(PlannerInfo *root, RelOptInfo *rel,
/*
* Compute the child's size.
*/
- set_rel_size(root, childrel, childRTindex, childRTE);
+ set_rel_size(root, childrel, childRTindex, childRTE, NULL);
/*
* It is possible that constraint exclusion detected a contradiction
@@ -1279,7 +1445,7 @@ set_append_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
/*
* Compute the child's access paths.
*/
- set_rel_pathlist(root, childrel, childRTindex, childRTE);
+ set_rel_pathlist(root, childrel, childRTindex, childRTE, NULL, NULL);
/*
* If child is dummy, ignore it.
@@ -2569,7 +2735,7 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
* See comments for deconstruct_jointree() for definition of the joinlist
* data structure.
*/
-static RelOptInfo *
+static RelOptInfoSet *
make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
{
int levels_needed;
@@ -2595,13 +2761,35 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
foreach(jl, joinlist)
{
Node *jlnode = (Node *) lfirst(jl);
- RelOptInfo *thisrel;
+ RelOptInfoSet *thisrel;
if (IsA(jlnode, RangeTblRef))
{
- int varno = ((RangeTblRef *) jlnode)->rtindex;
+ RangeTblRef *rtref = castNode(RangeTblRef, jlnode);
+ int varno = rtref->rtindex;
+ RangeTblEntry *rte = root->simple_rte_array[varno];
+ RelOptInfo *rel_grouped;
+
+ thisrel = makeNode(RelOptInfoSet);
+ thisrel->rel_plain = find_base_rel(root, varno);
- thisrel = find_base_rel(root, varno);
+ /*
+ * Create the grouped counterpart of rel_plain if possible.
+ */
+ build_simple_grouped_rel(root, thisrel);
+ rel_grouped = thisrel->rel_grouped;
+
+ /*
+ * If succeeded, do what should already have been done for the
+ * plain relation.
+ */
+ if (rel_grouped)
+ {
+ set_rel_size(root, rel_grouped, varno, rte,
+ thisrel->agg_info);
+ set_rel_pathlist(root, thisrel->rel_plain, varno, rte,
+ rel_grouped, thisrel->agg_info);
+ }
}
else if (IsA(jlnode, List))
{
@@ -2623,7 +2811,7 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
/*
* Single joinlist node, so we're done.
*/
- return (RelOptInfo *) linitial(initial_rels);
+ return (RelOptInfoSet *) linitial(initial_rels);
}
else
{
@@ -2637,11 +2825,19 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
root->initial_rels = initial_rels;
if (join_search_hook)
- return (*join_search_hook) (root, levels_needed, initial_rels);
+ return (*join_search_hook) (root, levels_needed,
+ initial_rels);
else if (enable_geqo && levels_needed >= geqo_threshold)
+ {
+ /*
+ * TODO Teach GEQO about grouped relations. Don't forget that
+ * pathlist can be NIL before set_cheapest() gets called.
+ */
return geqo(root, levels_needed, initial_rels);
+ }
else
- return standard_join_search(root, levels_needed, initial_rels);
+ return standard_join_search(root, levels_needed,
+ initial_rels);
}
}
@@ -2674,11 +2870,11 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
* than one join-order search, you'll probably need to save and restore the
* original states of those data structures. See geqo_eval() for an example.
*/
-RelOptInfo *
+RelOptInfoSet *
standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
{
int lev;
- RelOptInfo *rel;
+ RelOptInfoSet *result;
/*
* This function cannot be invoked recursively within any one planning
@@ -2720,13 +2916,20 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
*
* After that, we're done creating paths for the joinrel, so run
* set_cheapest().
+ *
+ * Neither partitionwise join nor parallel processing is supported for
+ * grouped relations so far.
*/
foreach(lc, root->join_rel_level[lev])
{
- rel = (RelOptInfo *) lfirst(lc);
+ RelOptInfoSet *rel_set;
+ RelOptInfo *rel_plain;
+
+ rel_set = (RelOptInfoSet *) lfirst(lc);
+ rel_plain = rel_set->rel_plain;
/* Create paths for partitionwise joins. */
- generate_partitionwise_join_paths(root, rel);
+ generate_partitionwise_join_paths(root, rel_plain);
/*
* Except for the topmost scan/join rel, consider gathering
@@ -2734,10 +2937,33 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
* once we know the final targetlist (see grouping_planner).
*/
if (lev < levels_needed)
- generate_gather_paths(root, rel, false);
+ generate_gather_paths(root, rel_plain, false);
/* Find and save the cheapest paths for this rel */
- set_cheapest(rel);
+ set_cheapest(rel_plain);
+
+ if (rel_set->rel_grouped)
+ {
+ RelOptInfo *rel_grouped;
+
+ rel_grouped = rel_set->rel_grouped;
+
+ /* Partial paths not supported yet. */
+ Assert(rel_grouped->partial_pathlist == NIL);
+
+ if (rel_grouped->pathlist != NIL)
+ set_cheapest(rel_grouped);
+ else
+ {
+ /*
+ * When checking whether the grouped relation can supply
+ * any input paths to join, test for existence of the
+ * relation will be easier than a test of pathlist.
+ */
+ pfree(rel_grouped);
+ rel_set->rel_grouped = NULL;
+ }
+ }
#ifdef OPTIMIZER_DEBUG
debug_print_rel(root, rel);
@@ -2752,11 +2978,11 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
elog(ERROR, "failed to build any %d-way joins", levels_needed);
Assert(list_length(root->join_rel_level[levels_needed]) == 1);
- rel = (RelOptInfo *) linitial(root->join_rel_level[levels_needed]);
+ result = (RelOptInfoSet *) linitial(root->join_rel_level[levels_needed]);
root->join_rel_level = NULL;
- return rel;
+ return result;
}
/*****************************************************************************
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index f1baa36886..5c8a009d09 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -2329,13 +2329,23 @@ cost_group(Path *path, PlannerInfo *root,
/*
* estimate_join_rows
* Set rows of a join path according to its parent relation or according
- * to parameters.
+ * to parameters. If agg_info is passed, the join path is grouped.
*/
static void
-estimate_join_rows(PlannerInfo *root, Path *path)
+estimate_join_rows(PlannerInfo *root, Path *path, RelAggInfo *agg_info)
{
if (path->param_info)
+ {
path->rows = path->param_info->ppi_rows;
+ if (agg_info)
+ {
+ double nrows;
+
+ nrows = estimate_num_groups(root, agg_info->group_exprs,
+ path->rows, NULL);
+ path->rows = clamp_row_est(nrows);
+ }
+ }
else
path->rows = path->parent->rows;
}
@@ -2461,7 +2471,7 @@ final_cost_nestloop(PlannerInfo *root, NestPath *path,
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
- estimate_join_rows(root, (Path *) path);
+ estimate_join_rows(root, (Path *) path, extra->agg_info);
/* For partial paths, scale row estimate. */
if (path->path.parallel_workers > 0)
@@ -2904,7 +2914,7 @@ final_cost_mergejoin(PlannerInfo *root, MergePath *path,
inner_path_rows = 1;
/* Mark the path with the correct row estimate */
- estimate_join_rows(root, (Path *) path);
+ estimate_join_rows(root, (Path *) path, extra->agg_info);
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
@@ -3331,7 +3341,7 @@ final_cost_hashjoin(PlannerInfo *root, HashPath *path,
ListCell *hcl;
/* Mark the path with the correct row estimate */
- estimate_join_rows(root, (Path *) path);
+ estimate_join_rows(root, (Path *) path, extra->agg_info);
/* For partial paths, scale row estimate. */
if (path->jpath.path.parallel_workers > 0)
@@ -4336,27 +4346,59 @@ approx_tuple_count(PlannerInfo *root, JoinPath *path, List *quals)
* restriction clauses).
* width: the estimated average output tuple width in bytes.
* baserestrictcost: estimated cost of evaluating baserestrictinfo clauses.
+ *
+ * If "agg_info" is passed, it means that "rel" is a grouped relation for
+ * which "agg_info" provides the information needed for size estimate.
*/
void
-set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel)
+set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel,
+ RelAggInfo *agg_info)
{
double nrows;
+ bool grouped = agg_info != NULL;
/* Should only be applied to base relations */
Assert(rel->relid > 0);
- nrows = rel->tuples *
- clauselist_selectivity(root,
- rel->baserestrictinfo,
- 0,
- JOIN_INNER,
- NULL);
+ if (!grouped)
+ {
+ nrows = rel->tuples *
+ clauselist_selectivity(root,
+ rel->baserestrictinfo,
+ 0,
+ JOIN_INNER,
+ NULL);
+ rel->rows = clamp_row_est(nrows);
+ }
- rel->rows = clamp_row_est(nrows);
+ /*
+ * Only set the estimate for grouped base rel if aggregation can take
+ * place. (Aggregation is the only way to build grouped base relation.)
+ */
+ else if (!bms_equal(rel->relids, root->all_baserels))
+ {
+ /*
+ * Grouping essentially changes the number of rows.
+ */
+ nrows = estimate_num_groups(root,
+ agg_info->group_exprs,
+ agg_info->input_rows,
+ NULL);
+ rel->rows = clamp_row_est(nrows);
+ }
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
- set_rel_width(root, rel);
+ /*
+ * The grouped target should have the cost and width set immediately on
+ * creation, see create_rel_agg_info().
+ */
+ if (!grouped)
+ set_rel_width(root, rel);
+#ifdef USE_ASSERT_CHECKING
+ else
+ Assert(rel->reltarget->width > 0);
+#endif
}
/*
@@ -4920,7 +4962,7 @@ set_subquery_size_estimates(PlannerInfo *root, RelOptInfo *rel)
}
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -4958,7 +5000,7 @@ set_function_size_estimates(PlannerInfo *root, RelOptInfo *rel)
}
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -4980,7 +5022,7 @@ set_tablefunc_size_estimates(PlannerInfo *root, RelOptInfo *rel)
rel->tuples = 100;
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -5011,7 +5053,7 @@ set_values_size_estimates(PlannerInfo *root, RelOptInfo *rel)
rel->tuples = list_length(rte->values_lists);
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -5049,7 +5091,7 @@ set_cte_size_estimates(PlannerInfo *root, RelOptInfo *rel, double cte_rows)
}
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -5082,7 +5124,7 @@ set_namedtuplestore_size_estimates(PlannerInfo *root, RelOptInfo *rel)
rel->tuples = 1000;
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -5105,7 +5147,7 @@ set_result_size_estimates(PlannerInfo *root, RelOptInfo *rel)
rel->tuples = 1;
/* Now estimate number of output rows, etc */
- set_baserel_size_estimates(root, rel);
+ set_baserel_size_estimates(root, rel, NULL);
}
/*
@@ -5329,11 +5371,11 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
foreach(lc, target->exprs)
{
Node *node = (Node *) lfirst(lc);
+ int32 item_width;
if (IsA(node, Var))
{
Var *var = (Var *) node;
- int32 item_width;
/* We should not see any upper-level Vars here */
Assert(var->varlevelsup == 0);
@@ -5364,6 +5406,20 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
Assert(item_width > 0);
tuple_width += item_width;
}
+ else if (IsA(node, Aggref))
+ {
+ /*
+ * If the target is evaluated by AggPath, it'll care of cost
+ * estimate. If the target is above AggPath (typically target of a
+ * join relation that contains grouped relation), the cost of
+ * Aggref should not be accounted for again.
+ *
+ * On the other hand, width is always needed.
+ */
+ item_width = get_typavgwidth(exprType(node), exprTypmod(node));
+ Assert(item_width > 0);
+ tuple_width += item_width;
+ }
else
{
/*
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 3454f12912..0452d415fb 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -65,7 +65,6 @@ static bool reconsider_outer_join_clause(PlannerInfo *root,
static bool reconsider_full_join_clause(PlannerInfo *root,
RestrictInfo *rinfo);
-
/*
* process_equivalence
* The given clause has a mergejoinable operator and can be applied without
@@ -2511,3 +2510,137 @@ is_redundant_derived_clause(RestrictInfo *rinfo, List *clauselist)
return false;
}
+
+/*
+ * translate_expression_to_rels
+ * If the appropriate equivalence classes exist, replace vars in
+ * gvi->gvexpr with vars whose varno is equal to relid. Return NULL if
+ * translation is not possible or needed.
+ *
+ * Note: Currently we only translate Var expressions. This is subject to
+ * change as the aggregate push-down feature gets enhanced.
+ */
+GroupedVarInfo *
+translate_expression_to_rels(PlannerInfo *root, GroupedVarInfo *gvi,
+ Index relid)
+{
+ Var *var;
+ ListCell *l1;
+ bool found_orig = false;
+ Var *var_translated = NULL;
+ GroupedVarInfo *result;
+
+ /* Can't do anything w/o equivalence classes. */
+ if (root->eq_classes == NIL)
+ return NULL;
+
+ var = castNode(Var, gvi->gvexpr);
+
+ /*
+ * Do we need to translate the var?
+ */
+ if (var->varno == relid)
+ return NULL;
+
+ /*
+ * Find the replacement var.
+ */
+ foreach(l1, root->eq_classes)
+ {
+ EquivalenceClass *ec = lfirst_node(EquivalenceClass, l1);
+ ListCell *l2;
+
+ /* TODO Check if any other EC kind should be ignored. */
+ if (ec->ec_has_volatile || ec->ec_below_outer_join || ec->ec_broken)
+ continue;
+
+ /* Single-element EC can hardly help in translations. */
+ if (list_length(ec->ec_members) == 1)
+ continue;
+
+ /*
+ * Collect all vars of this EC and their varnos.
+ *
+ * ec->ec_relids does not help because we're only interested in a
+ * subset of EC members.
+ */
+ foreach(l2, ec->ec_members)
+ {
+ EquivalenceMember *em = lfirst_node(EquivalenceMember, l2);
+ Var *ec_var;
+
+ /*
+ * The grouping expressions derived here are used to evaluate
+ * possibility to push aggregation down to RELOPT_BASEREL or
+ * RELOPT_JOINREL relations, and to construct reltargets for the
+ * grouped rels. We're not interested at the moment whether the
+ * relations do have children.
+ */
+ if (em->em_is_child)
+ continue;
+
+ if (!IsA(em->em_expr, Var))
+ continue;
+
+ ec_var = castNode(Var, em->em_expr);
+ if (equal(ec_var, var))
+ found_orig = true;
+ else if (ec_var->varno == relid)
+ var_translated = ec_var;
+
+ if (found_orig && var_translated)
+ {
+ /*
+ * The replacement Var must have the same data type, otherwise
+ * the values are not guaranteed to be grouped in the same way
+ * as values of the original Var.
+ */
+ if (ec_var->vartype != var->vartype)
+ return NULL;
+
+ break;
+ }
+ }
+
+ if (found_orig)
+ {
+ /*
+ * The same expression probably does not exist in multiple ECs.
+ */
+ if (var_translated == NULL)
+ {
+ /*
+ * Failed to translate the expression.
+ */
+ return NULL;
+ }
+ else
+ {
+ /* Success. */
+ break;
+ }
+ }
+ else
+ {
+ /*
+ * Vars of the requested relid can be in the next ECs too.
+ */
+ var_translated = NULL;
+ }
+ }
+
+ if (!found_orig)
+ return NULL;
+
+ result = makeNode(GroupedVarInfo);
+ memcpy(result, gvi, sizeof(GroupedVarInfo));
+
+ /*
+ * translate_expression_to_rels_mutator updates gv_eval_at.
+ */
+ result->gv_eval_at = bms_make_singleton(relid);
+ result->gvexpr = (Expr *) var_translated;
+ result->derived = true;
+
+ return result;
+}
diff --git a/src/backend/optimizer/path/indxpath.c b/src/backend/optimizer/path/indxpath.c
index 7e1a3908f1..c4080b0587 100644
--- a/src/backend/optimizer/path/indxpath.c
+++ b/src/backend/optimizer/path/indxpath.c
@@ -77,13 +77,14 @@ typedef struct
int indexcol; /* index column we want to match to */
} ec_member_matches_arg;
-
static void consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index,
IndexClauseSet *rclauseset,
IndexClauseSet *jclauseset,
IndexClauseSet *eclauseset,
- List **bitindexpaths);
+ List **bitindexpaths,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static void consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index,
IndexClauseSet *rclauseset,
@@ -92,7 +93,9 @@ static void consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
List **bitindexpaths,
List *indexjoinclauses,
int considered_clauses,
- List **considered_relids);
+ List **considered_relids,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static void get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index,
IndexClauseSet *rclauseset,
@@ -100,19 +103,24 @@ static void get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexClauseSet *eclauseset,
List **bitindexpaths,
Relids relids,
- List **considered_relids);
+ List **considered_relids,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static bool eclass_already_used(EquivalenceClass *parent_ec, Relids oldrelids,
List *indexjoinclauses);
static bool bms_equal_any(Relids relids, List *relids_list);
static void get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
- List **bitindexpaths);
+ List **bitindexpaths, RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static List *build_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
- bool *skip_lower_saop);
+ bool *skip_lower_saop,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
static List *build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
List *clauses, List *other_clauses);
static List *generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
@@ -216,6 +224,10 @@ static Const *string_to_const(const char *str, Oid datatype);
*
* 'rel' is the relation for which we want to generate index paths
*
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
+ *
* Note: check_index_predicates() must have been run previously for this rel.
*
* Note: in cases involving LATERAL references in the relation's tlist, it's
@@ -228,7 +240,8 @@ static Const *string_to_const(const char *str, Oid datatype);
* as meaning "unparameterized so far as the indexquals are concerned".
*/
void
-create_index_paths(PlannerInfo *root, RelOptInfo *rel)
+create_index_paths(PlannerInfo *root, RelOptInfo *rel, RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
List *indexpaths;
List *bitindexpaths;
@@ -238,6 +251,7 @@ create_index_paths(PlannerInfo *root, RelOptInfo *rel)
IndexClauseSet jclauseset;
IndexClauseSet eclauseset;
ListCell *lc;
+ bool grouped = rel_grouped != NULL;
/* Skip the whole mess if no indexes */
if (rel->indexlist == NIL)
@@ -273,8 +287,8 @@ create_index_paths(PlannerInfo *root, RelOptInfo *rel)
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
- get_index_paths(root, rel, index, &rclauseset,
- &bitindexpaths);
+ get_index_paths(root, rel, index, &rclauseset, &bitindexpaths,
+ rel_grouped, agg_info);
/*
* Identify the join clauses that can match the index. For the moment
@@ -303,15 +317,24 @@ create_index_paths(PlannerInfo *root, RelOptInfo *rel)
&rclauseset,
&jclauseset,
&eclauseset,
- &bitjoinpaths);
+ &bitjoinpaths,
+ rel_grouped,
+ agg_info);
}
/*
+ * It does not seem too efficient to aggregate the individual paths and
+ * then AND them together.
+ */
+ if (grouped)
+ return;
+
+ /*
* Generate BitmapOrPaths for any suitable OR-clauses present in the
* restriction list. Add these to bitindexpaths.
*/
- indexpaths = generate_bitmap_or_paths(root, rel,
- rel->baserestrictinfo, NIL);
+ indexpaths = generate_bitmap_or_paths(root, rel, rel->baserestrictinfo,
+ NIL);
bitindexpaths = list_concat(bitindexpaths, indexpaths);
/*
@@ -433,6 +456,10 @@ create_index_paths(PlannerInfo *root, RelOptInfo *rel)
* 'jclauseset' is the collection of indexable simple join clauses
* 'eclauseset' is the collection of indexable clauses from EquivalenceClasses
* '*bitindexpaths' is the list to add bitmap paths to
+ *
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
*/
static void
consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
@@ -440,7 +467,9 @@ consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
IndexClauseSet *rclauseset,
IndexClauseSet *jclauseset,
IndexClauseSet *eclauseset,
- List **bitindexpaths)
+ List **bitindexpaths,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
int considered_clauses = 0;
List *considered_relids = NIL;
@@ -476,7 +505,9 @@ consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
bitindexpaths,
jclauseset->indexclauses[indexcol],
considered_clauses,
- &considered_relids);
+ &considered_relids,
+ rel_grouped,
+ agg_info);
/* Consider each applicable eclass join clause */
considered_clauses += list_length(eclauseset->indexclauses[indexcol]);
consider_index_join_outer_rels(root, rel, index,
@@ -484,7 +515,9 @@ consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
bitindexpaths,
eclauseset->indexclauses[indexcol],
considered_clauses,
- &considered_relids);
+ &considered_relids,
+ rel_grouped,
+ agg_info);
}
}
@@ -499,6 +532,10 @@ consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
* 'indexjoinclauses' is a list of RestrictInfos for join clauses
* 'considered_clauses' is the total number of clauses considered (so far)
* '*considered_relids' is a list of all relids sets already considered
+ *
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
*/
static void
consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
@@ -509,7 +546,9 @@ consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
List **bitindexpaths,
List *indexjoinclauses,
int considered_clauses,
- List **considered_relids)
+ List **considered_relids,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
ListCell *lc;
@@ -576,7 +615,9 @@ consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
rclauseset, jclauseset, eclauseset,
bitindexpaths,
bms_union(clause_relids, oldrelids),
- considered_relids);
+ considered_relids,
+ rel_grouped,
+ agg_info);
}
/* Also try this set of relids by itself */
@@ -584,7 +625,9 @@ consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
rclauseset, jclauseset, eclauseset,
bitindexpaths,
clause_relids,
- considered_relids);
+ considered_relids,
+ rel_grouped,
+ agg_info);
}
}
@@ -600,6 +643,10 @@ consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
* 'bitindexpaths', 'considered_relids' as above
* 'relids' is the current set of relids to consider (the target rel plus
* one or more outer rels)
+ *
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
*/
static void
get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
@@ -609,7 +656,9 @@ get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexClauseSet *eclauseset,
List **bitindexpaths,
Relids relids,
- List **considered_relids)
+ List **considered_relids,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
IndexClauseSet clauseset;
int indexcol;
@@ -666,7 +715,8 @@ get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
Assert(clauseset.nonempty);
/* Build index path(s) using the collected set of clauses */
- get_index_paths(root, rel, index, &clauseset, bitindexpaths);
+ get_index_paths(root, rel, index, &clauseset, bitindexpaths,
+ rel_grouped, agg_info);
/*
* Remember we considered paths for this set of relids. We use lcons not
@@ -716,7 +766,6 @@ bms_equal_any(Relids relids, List *relids_list)
return false;
}
-
/*
* get_index_paths
* Given an index and a set of index clauses for it, construct IndexPaths.
@@ -731,16 +780,23 @@ bms_equal_any(Relids relids, List *relids_list)
* paths, and then make a separate attempt to include them in bitmap paths.
* Furthermore, we should consider excluding lower-order ScalarArrayOpExpr
* quals so as to create ordered paths.
+ *
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
*/
static void
get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
- List **bitindexpaths)
+ List **bitindexpaths,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
List *indexpaths;
bool skip_nonnative_saop = false;
bool skip_lower_saop = false;
ListCell *lc;
+ bool grouped = rel_grouped != NULL;
/*
* Build simple index paths using the clauses. Allow ScalarArrayOpExpr
@@ -753,7 +809,40 @@ get_index_paths(PlannerInfo *root, RelOptInfo *rel,
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
- &skip_lower_saop);
+ &skip_lower_saop,
+ rel_grouped,
+ agg_info);
+
+ /*
+ * If the relation is grouped, apply the partial aggregation.
+ *
+ * Only AGG_SORTED strategy is used, so we ignore bitmap paths, as well as
+ * indexes that can only produce them.
+ */
+ if (grouped && index->amhasgettuple)
+ {
+ foreach(lc, indexpaths)
+ {
+ IndexPath *ipath = (IndexPath *) lfirst(lc);
+ Path *subpath = &ipath->path;
+
+ if (subpath->pathkeys != NIL)
+ {
+ AggPath *agg_path;
+
+ agg_path = create_agg_sorted_path(root,
+ subpath,
+ agg_info);
+ if (agg_path)
+ add_path(rel_grouped, (Path *) agg_path);
+ }
+ }
+
+ /*
+ * Nothing more to do for grouped relation.
+ */
+ return;
+ }
/*
* If we skipped any lower-order ScalarArrayOpExprs on an index with an AM
@@ -768,7 +857,9 @@ get_index_paths(PlannerInfo *root, RelOptInfo *rel,
index->predOK,
ST_ANYSCAN,
&skip_nonnative_saop,
- NULL));
+ NULL,
+ rel_grouped,
+ agg_info));
}
/*
@@ -808,7 +899,9 @@ get_index_paths(PlannerInfo *root, RelOptInfo *rel,
false,
ST_BITMAPSCAN,
NULL,
- NULL);
+ NULL,
+ rel_grouped,
+ agg_info);
*bitindexpaths = list_concat(*bitindexpaths, indexpaths);
}
}
@@ -852,7 +945,15 @@ get_index_paths(PlannerInfo *root, RelOptInfo *rel,
* 'useful_predicate' indicates whether the index has a useful predicate
* 'scantype' indicates whether we need plain or bitmap scan support
* 'skip_nonnative_saop' indicates whether to accept SAOP if index AM doesn't
- * 'skip_lower_saop' indicates whether to accept non-first-column SAOP
+ * 'skip_lower_saop' indicates whether to accept non-first-column SAOP.
+ *
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
+ *
+ * XXX When enabling the aggregate push-down for partial paths, we'll need the
+ * function to return the aggregation input paths in a separate list instead
+ * of calling add_partial_path() on them immediately.
*/
static List *
build_index_paths(PlannerInfo *root, RelOptInfo *rel,
@@ -860,7 +961,9 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
bool useful_predicate,
ScanTypeControl scantype,
bool *skip_nonnative_saop,
- bool *skip_lower_saop)
+ bool *skip_lower_saop,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
List *result = NIL;
IndexPath *ipath;
@@ -877,6 +980,12 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
bool index_is_ordered;
bool index_only_scan;
int indexcol;
+ bool include_partial;
+
+ /*
+ * Grouped partial paths are not supported yet.
+ */
+ include_partial = rel_grouped == NULL;
/*
* Check that index supports the desired scan type(s)
@@ -1046,14 +1155,16 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
index_only_scan,
outer_relids,
loop_count,
- false);
+ false,
+ rel_grouped,
+ agg_info);
result = lappend(result, ipath);
/*
* If appropriate, consider parallel index scan. We don't allow
* parallel index scan for bitmap index scans.
*/
- if (index->amcanparallel &&
+ if (include_partial && index->amcanparallel &&
rel->consider_parallel && outer_relids == NULL &&
scantype != ST_BITMAPSCAN)
{
@@ -1069,7 +1180,9 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
index_only_scan,
outer_relids,
loop_count,
- true);
+ true,
+ rel_grouped,
+ agg_info);
/*
* if, after costing the path, we find that it's not worth using
@@ -1103,11 +1216,13 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
index_only_scan,
outer_relids,
loop_count,
- false);
+ false,
+ rel_grouped,
+ agg_info);
result = lappend(result, ipath);
/* If appropriate, consider parallel index scan */
- if (index->amcanparallel &&
+ if (include_partial && index->amcanparallel &&
rel->consider_parallel && outer_relids == NULL &&
scantype != ST_BITMAPSCAN)
{
@@ -1121,7 +1236,9 @@ build_index_paths(PlannerInfo *root, RelOptInfo *rel,
index_only_scan,
outer_relids,
loop_count,
- true);
+ true,
+ rel_grouped,
+ agg_info);
/*
* if, after costing the path, we find that it's not worth
@@ -1243,6 +1360,8 @@ build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
useful_predicate,
ST_BITMAPSCAN,
NULL,
+ NULL,
+ NULL,
NULL);
result = list_concat(result, indexpaths);
}
diff --git a/src/backend/optimizer/path/joinpath.c b/src/backend/optimizer/path/joinpath.c
index d8ff4bf432..f99d6b98a9 100644
--- a/src/backend/optimizer/path/joinpath.c
+++ b/src/backend/optimizer/path/joinpath.c
@@ -120,7 +120,9 @@ add_paths_to_joinrel(PlannerInfo *root,
RelOptInfo *innerrel,
JoinType jointype,
SpecialJoinInfo *sjinfo,
- List *restrictlist)
+ List *restrictlist,
+ RelAggInfo *agg_info,
+ RelOptInfo *rel_agg_input)
{
JoinPathExtraData extra;
bool mergejoin_allowed = true;
@@ -143,6 +145,8 @@ add_paths_to_joinrel(PlannerInfo *root,
extra.mergeclause_list = NIL;
extra.sjinfo = sjinfo;
extra.param_source_rels = NULL;
+ extra.agg_info = agg_info;
+ extra.rel_agg_input = rel_agg_input;
/*
* See if the inner relation is provably unique for this outer rel.
@@ -376,6 +380,8 @@ try_nestloop_path(PlannerInfo *root,
Relids outerrelids;
Relids inner_paramrels = PATH_REQ_OUTER(inner_path);
Relids outer_paramrels = PATH_REQ_OUTER(outer_path);
+ bool do_aggregate = extra->rel_agg_input != NULL;
+ bool success = false;
/*
* Paths are parameterized by top-level parents, so run parameterization
@@ -422,10 +428,37 @@ try_nestloop_path(PlannerInfo *root,
initial_cost_nestloop(root, &workspace, jointype,
outer_path, inner_path, extra);
- if (add_path_precheck(joinrel,
- workspace.startup_cost, workspace.total_cost,
- pathkeys, required_outer))
+ /*
+ * If the join output should be aggregated, the precheck is skipped
+ * because it makes little sense to compare the new join path to existing,
+ * already aggregated paths. Since we don't have row count estimate yet,
+ * it's hard to involve AggPath in the precheck.
+ */
+ if ((!do_aggregate &&
+ add_path_precheck(joinrel,
+ workspace.startup_cost, workspace.total_cost,
+ pathkeys, required_outer)) ||
+ do_aggregate)
{
+ PathTarget *target;
+ Path *path;
+ RelOptInfo *parent_rel;
+
+ if (!do_aggregate)
+ {
+ target = joinrel->reltarget;
+ parent_rel = joinrel;
+ }
+ else
+ {
+ /*
+ * If the join output is subject to aggregation, the path must
+ * generate aggregation input.
+ */
+ target = extra->agg_info->input;
+ parent_rel = extra->rel_agg_input;
+ }
+
/*
* If the inner path is parameterized, it is parameterized by the
* topmost parent of the outer rel, not the outer rel itself. Fix
@@ -447,21 +480,57 @@ try_nestloop_path(PlannerInfo *root,
}
}
- add_path(joinrel, (Path *)
- create_nestloop_path(root,
- joinrel,
- jointype,
- &workspace,
- extra,
- outer_path,
- inner_path,
- extra->restrictlist,
- pathkeys,
- required_outer));
+ path = (Path *) create_nestloop_path(root,
+ parent_rel,
+ target,
+ jointype,
+ &workspace,
+ extra,
+ outer_path,
+ inner_path,
+ extra->restrictlist,
+ pathkeys,
+ required_outer);
+
+ if (!do_aggregate)
+ {
+ add_path(joinrel, path);
+ success = true;
+ }
+ else
+ {
+ /*
+ * Non-grouped rel had to be passed to create_nestloop_path() so
+ * that row estimate reflects the aggregation input data, however
+ * even the aggregation input path should eventually be owned by
+ * the grouped joinrel.
+ */
+ path->parent = joinrel;
+
+ /*
+ * Try both AGG_HASHED and AGG_SORTED aggregation.
+ *
+ * AGG_HASHED should not be parameterized because we don't want to
+ * create the hashtable again for each set of parameters.
+ */
+ if (required_outer == NULL)
+ success = add_grouped_path(root, joinrel, path,
+ AGG_HASHED, extra->agg_info);
+
+ /*
+ * Don't try AGG_SORTED if add_grouped_path() would reject it
+ * anyway.
+ */
+ if (pathkeys != NIL)
+ success = success ||
+ add_grouped_path(root, joinrel, path, AGG_SORTED,
+ extra->agg_info);
+ }
}
- else
+
+ if (!success)
{
- /* Waste no memory when we reject a path here */
+ /* Waste no memory when we reject path(s) here */
bms_free(required_outer);
}
}
@@ -538,6 +607,7 @@ try_partial_nestloop_path(PlannerInfo *root,
add_partial_path(joinrel, (Path *)
create_nestloop_path(root,
joinrel,
+ joinrel->reltarget,
jointype,
&workspace,
extra,
@@ -568,8 +638,11 @@ try_mergejoin_path(PlannerInfo *root,
{
Relids required_outer;
JoinCostWorkspace workspace;
+ bool grouped = extra->agg_info != NULL;
+ bool do_aggregate = extra->rel_agg_input != NULL;
+ bool success = false;
- if (is_partial)
+ if (!grouped && is_partial)
{
try_partial_mergejoin_path(root,
joinrel,
@@ -617,26 +690,73 @@ try_mergejoin_path(PlannerInfo *root,
outersortkeys, innersortkeys,
extra);
- if (add_path_precheck(joinrel,
- workspace.startup_cost, workspace.total_cost,
- pathkeys, required_outer))
+ /*
+ * See comments in try_nestloop_path().
+ */
+ if ((!do_aggregate &&
+ add_path_precheck(joinrel,
+ workspace.startup_cost, workspace.total_cost,
+ pathkeys, required_outer)) ||
+ do_aggregate)
{
- add_path(joinrel, (Path *)
- create_mergejoin_path(root,
- joinrel,
- jointype,
- &workspace,
- extra,
- outer_path,
- inner_path,
- extra->restrictlist,
- pathkeys,
- required_outer,
- mergeclauses,
- outersortkeys,
- innersortkeys));
+ PathTarget *target;
+ Path *path;
+ RelOptInfo *parent_rel;
+
+ if (!do_aggregate)
+ {
+ target = joinrel->reltarget;
+ parent_rel = joinrel;
+ }
+ else
+ {
+ target = extra->agg_info->input;
+ parent_rel = extra->rel_agg_input;
+ }
+
+ path = (Path *) create_mergejoin_path(root,
+ parent_rel,
+ target,
+ jointype,
+ &workspace,
+ extra,
+ outer_path,
+ inner_path,
+ extra->restrictlist,
+ pathkeys,
+ required_outer,
+ mergeclauses,
+ outersortkeys,
+ innersortkeys);
+
+ if (!do_aggregate)
+ {
+ add_path(joinrel, path);
+ success = true;
+ }
+ else
+ {
+ /* See comment in try_nestloop_path() */
+ path->parent = joinrel;
+
+ if (required_outer == NULL)
+ success = add_grouped_path(root,
+ joinrel,
+ path,
+ AGG_HASHED,
+ extra->agg_info);
+
+ if (pathkeys != NIL)
+ success = success ||
+ add_grouped_path(root,
+ joinrel,
+ path,
+ AGG_SORTED,
+ extra->agg_info);
+ }
}
- else
+
+ if (!success)
{
/* Waste no memory when we reject a path here */
bms_free(required_outer);
@@ -700,6 +820,7 @@ try_partial_mergejoin_path(PlannerInfo *root,
add_partial_path(joinrel, (Path *)
create_mergejoin_path(root,
joinrel,
+ joinrel->reltarget,
jointype,
&workspace,
extra,
@@ -729,6 +850,9 @@ try_hashjoin_path(PlannerInfo *root,
{
Relids required_outer;
JoinCostWorkspace workspace;
+ bool grouped = extra->agg_info != NULL;
+ bool do_aggregate = extra->rel_agg_input != NULL;
+ bool success = false;
/*
* Check to see if proposed path is still parameterized, and reject if the
@@ -745,30 +869,79 @@ try_hashjoin_path(PlannerInfo *root,
}
/*
+ * Parameterized execution of grouped path would mean repeated hashing of
+ * the output of the hashjoin output, so forget about AGG_HASHED if there
+ * are any parameters. And AGG_SORTED makes no sense because the hash join
+ * output is not sorted.
+ */
+ if (required_outer && grouped)
+ return;
+
+ /*
* See comments in try_nestloop_path(). Also note that hashjoin paths
* never have any output pathkeys, per comments in create_hashjoin_path.
*/
initial_cost_hashjoin(root, &workspace, jointype, hashclauses,
outer_path, inner_path, extra, false);
- if (add_path_precheck(joinrel,
- workspace.startup_cost, workspace.total_cost,
- NIL, required_outer))
+
+ /*
+ * See comments in try_nestloop_path().
+ */
+ if ((!do_aggregate &&
+ add_path_precheck(joinrel,
+ workspace.startup_cost, workspace.total_cost,
+ NIL, required_outer)) ||
+ do_aggregate)
{
- add_path(joinrel, (Path *)
- create_hashjoin_path(root,
- joinrel,
- jointype,
- &workspace,
- extra,
- outer_path,
- inner_path,
- false, /* parallel_hash */
- extra->restrictlist,
- required_outer,
- hashclauses));
+ PathTarget *target;
+ Path *path = NULL;
+ RelOptInfo *parent_rel;
+
+ if (!do_aggregate)
+ {
+ target = joinrel->reltarget;
+ parent_rel = joinrel;
+ }
+ else
+ {
+ target = extra->agg_info->input;
+ parent_rel = extra->rel_agg_input;
+ }
+
+ path = (Path *) create_hashjoin_path(root,
+ parent_rel,
+ target,
+ jointype,
+ &workspace,
+ extra,
+ outer_path,
+ inner_path,
+ false, /* parallel_hash */
+ extra->restrictlist,
+ required_outer,
+ hashclauses);
+
+ if (!do_aggregate)
+ {
+ add_path(joinrel, path);
+ success = true;
+ }
+ else
+ {
+ /* See comment in try_nestloop_path() */
+ path->parent = joinrel;
+
+ /*
+ * As the hashjoin path is not sorted, only try AGG_HASHED.
+ */
+ if (add_grouped_path(root, joinrel, path, AGG_HASHED,
+ extra->agg_info))
+ success = true;
+ }
}
- else
+
+ if (!success)
{
/* Waste no memory when we reject a path here */
bms_free(required_outer);
@@ -824,6 +997,7 @@ try_partial_hashjoin_path(PlannerInfo *root,
add_partial_path(joinrel, (Path *)
create_hashjoin_path(root,
joinrel,
+ joinrel->reltarget,
jointype,
&workspace,
extra,
@@ -892,6 +1066,7 @@ sort_inner_and_outer(PlannerInfo *root,
Path *cheapest_safe_inner = NULL;
List *all_pathkeys;
ListCell *l;
+ bool grouped = extra->agg_info != NULL;
/*
* We only consider the cheapest-total-cost input paths, since we are
@@ -1051,7 +1226,7 @@ sort_inner_and_outer(PlannerInfo *root,
* If we have partial outer and parallel safe inner path then try
* partial mergejoin path.
*/
- if (cheapest_partial_outer && cheapest_safe_inner)
+ if (!grouped && cheapest_partial_outer && cheapest_safe_inner)
try_partial_mergejoin_path(root,
joinrel,
cheapest_partial_outer,
@@ -1341,6 +1516,7 @@ match_unsorted_outer(PlannerInfo *root,
Path *inner_cheapest_total = innerrel->cheapest_total_path;
Path *matpath = NULL;
ListCell *lc1;
+ bool grouped = extra->agg_info != NULL;
/*
* Nestloop only supports inner, left, semi, and anti joins. Also, if we
@@ -1516,7 +1692,8 @@ match_unsorted_outer(PlannerInfo *root,
* parameterized. Similarly, we can't handle JOIN_FULL and JOIN_RIGHT,
* because they can produce false null extended rows.
*/
- if (joinrel->consider_parallel &&
+ if (!grouped &&
+ joinrel->consider_parallel &&
save_jointype != JOIN_UNIQUE_OUTER &&
save_jointype != JOIN_FULL &&
save_jointype != JOIN_RIGHT &&
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index fc9eb95f5a..8014f10ac3 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -24,24 +24,33 @@
#include "partitioning/partbounds.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+#include "utils/selfuncs.h"
static void make_rels_by_clause_joins(PlannerInfo *root,
- RelOptInfo *old_rel,
+ RelOptInfoSet *old_relset,
ListCell *other_rels);
static void make_rels_by_clauseless_joins(PlannerInfo *root,
- RelOptInfo *old_rel,
+ RelOptInfoSet *old_relset,
ListCell *other_rels);
+static void set_grouped_joinrel_target(PlannerInfo *root, RelOptInfo *joinrel,
+ RelAggInfo *agg_info);
static bool has_join_restriction(PlannerInfo *root, RelOptInfo *rel);
static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool is_dummy_rel(RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
-static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2);
+static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ RelAggInfo *agg_info, RelOptInfo *rel_agg_input);
+static void make_join_rel_common_grouped(PlannerInfo *root, RelOptInfoSet *relset1,
+ RelOptInfoSet *relset2,
+ RelAggInfo *agg_info);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
- SpecialJoinInfo *sjinfo, List *restrictlist);
+ SpecialJoinInfo *sjinfo, List *restrictlist,
+ RelAggInfo *agg_info,
+ RelOptInfo *rel_agg_input);
static void try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *parent_sjinfo,
@@ -54,7 +63,6 @@ static SpecialJoinInfo *build_child_join_sjinfo(PlannerInfo *root,
static int match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel,
bool strict_op);
-
/*
* join_search_one_level
* Consider ways to produce join relations containing exactly 'level'
@@ -89,7 +97,15 @@ join_search_one_level(PlannerInfo *root, int level)
*/
foreach(r, joinrels[level - 1])
{
- RelOptInfo *old_rel = (RelOptInfo *) lfirst(r);
+ RelOptInfoSet *old_relset = (RelOptInfoSet *) lfirst(r);
+ RelOptInfo *old_rel;
+
+ /*
+ * Both rel_plain and rel_grouped should have the same clauses, so it
+ * does not matter which one we use here. The only difference is that
+ * rel_grouped is not guaranteed to exist, so use rel_plain.
+ */
+ old_rel = old_relset->rel_plain;
if (old_rel->joininfo != NIL || old_rel->has_eclass_joins ||
has_join_restriction(root, old_rel))
@@ -115,7 +131,7 @@ join_search_one_level(PlannerInfo *root, int level)
other_rels = list_head(joinrels[1]);
make_rels_by_clause_joins(root,
- old_rel,
+ old_relset,
other_rels);
}
else
@@ -133,7 +149,7 @@ join_search_one_level(PlannerInfo *root, int level)
* avoid the duplicated effort.
*/
make_rels_by_clauseless_joins(root,
- old_rel,
+ old_relset,
list_head(joinrels[1]));
}
}
@@ -159,7 +175,8 @@ join_search_one_level(PlannerInfo *root, int level)
foreach(r, joinrels[k])
{
- RelOptInfo *old_rel = (RelOptInfo *) lfirst(r);
+ RelOptInfoSet *old_relset = (RelOptInfoSet *) lfirst(r);
+ RelOptInfo *old_rel = old_relset->rel_plain;
ListCell *other_rels;
ListCell *r2;
@@ -179,7 +196,8 @@ join_search_one_level(PlannerInfo *root, int level)
for_each_cell(r2, other_rels)
{
- RelOptInfo *new_rel = (RelOptInfo *) lfirst(r2);
+ RelOptInfoSet *new_relset = (RelOptInfoSet *) lfirst(r2);
+ RelOptInfo *new_rel = new_relset->rel_plain;
if (!bms_overlap(old_rel->relids, new_rel->relids))
{
@@ -191,7 +209,7 @@ join_search_one_level(PlannerInfo *root, int level)
if (have_relevant_joinclause(root, old_rel, new_rel) ||
have_join_order_restriction(root, old_rel, new_rel))
{
- (void) make_join_rel(root, old_rel, new_rel);
+ (void) make_join_rel(root, old_relset, new_relset);
}
}
}
@@ -225,10 +243,10 @@ join_search_one_level(PlannerInfo *root, int level)
*/
foreach(r, joinrels[level - 1])
{
- RelOptInfo *old_rel = (RelOptInfo *) lfirst(r);
+ RelOptInfoSet *old_relset = (RelOptInfoSet *) lfirst(r);
make_rels_by_clauseless_joins(root,
- old_rel,
+ old_relset,
list_head(joinrels[1]));
}
@@ -274,25 +292,31 @@ join_search_one_level(PlannerInfo *root, int level)
* 'other_rels': the first cell in a linked list containing the other
* rels to be considered for joining
*
+ * 'rel_plain' is what we use of RelOptInfoSet because it's guaranteed to
+ * exists. However the test should have the same result if we used the
+ * corresponding 'rel_grouped'.
+ *
* Currently, this is only used with initial rels in other_rels, but it
* will work for joining to joinrels too.
*/
static void
make_rels_by_clause_joins(PlannerInfo *root,
- RelOptInfo *old_rel,
+ RelOptInfoSet *old_relset,
ListCell *other_rels)
{
ListCell *l;
for_each_cell(l, other_rels)
{
- RelOptInfo *other_rel = (RelOptInfo *) lfirst(l);
+ RelOptInfo *old_rel = old_relset->rel_plain;
+ RelOptInfoSet *other_relset = (RelOptInfoSet *) lfirst(l);
+ RelOptInfo *other_rel = other_relset->rel_plain;
if (!bms_overlap(old_rel->relids, other_rel->relids) &&
(have_relevant_joinclause(root, old_rel, other_rel) ||
have_join_order_restriction(root, old_rel, other_rel)))
{
- (void) make_join_rel(root, old_rel, other_rel);
+ (void) make_join_rel(root, old_relset, other_relset);
}
}
}
@@ -308,27 +332,62 @@ make_rels_by_clause_joins(PlannerInfo *root,
* 'other_rels': the first cell of a linked list containing the
* other rels to be considered for joining
*
+ * 'rel_plain' is what we use of RelOptInfoSet because it's guaranteed to
+ * exists. However the test should have the same result if we used the
+ * corresponding 'rel_grouped'.
+ *
* Currently, this is only used with initial rels in other_rels, but it would
* work for joining to joinrels too.
*/
static void
make_rels_by_clauseless_joins(PlannerInfo *root,
- RelOptInfo *old_rel,
+ RelOptInfoSet *old_relset,
ListCell *other_rels)
{
ListCell *l;
for_each_cell(l, other_rels)
{
- RelOptInfo *other_rel = (RelOptInfo *) lfirst(l);
+ RelOptInfo *old_rel = old_relset->rel_plain;
+ RelOptInfoSet *other_relset = (RelOptInfoSet *) lfirst(l);
+ RelOptInfo *other_rel = other_relset->rel_plain;
if (!bms_overlap(other_rel->relids, old_rel->relids))
{
- (void) make_join_rel(root, old_rel, other_rel);
+ (void) make_join_rel(root, old_relset, other_relset);
}
}
}
+/*
+ * Set joinrel's reltarget according to agg_info and estimate the number of
+ * rows.
+ */
+static void
+set_grouped_joinrel_target(PlannerInfo *root, RelOptInfo *joinrel,
+ RelAggInfo *agg_info)
+{
+ Assert(agg_info != NULL);
+
+ /*
+ * build_join_rel() does not create the target for grouped relation.
+ */
+ Assert(joinrel->reltarget == NULL);
+
+ joinrel->reltarget = agg_info->target;
+
+ /*
+ * Grouping essentially changes the number of rows.
+ *
+ * XXX We do not distinguish whether two plain rels are joined and the
+ * result is aggregated, or the aggregation has been already applied to
+ * one of the input rels. Is this worth extra effort, e.g. maintaining a
+ * separate RelOptInfo for each case (one difficulty that would introduce
+ * is construction of AppendPath)?
+ */
+ joinrel->rows = estimate_num_groups(root, agg_info->group_exprs,
+ agg_info->input_rows, NULL);
+}
/*
* join_is_legal
@@ -661,9 +720,18 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
/*
* make_join_rel_common
* The workhorse of make_join_rel().
+ *
+ * 'agg_info' contains the reltarget of grouped relation and everything we
+ * need to aggregate the join result. If NULL, then the join relation
+ * should not be grouped.
+ *
+ * 'rel_agg_input' describes the AggPath input relation if the join output
+ * should be aggregated. If NULL is passed, do not aggregate the join
+ * output.
*/
static RelOptInfo *
-make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ RelAggInfo *agg_info, RelOptInfo *rel_agg_input)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -671,10 +739,15 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
SpecialJoinInfo sjinfo_data;
RelOptInfo *joinrel;
List *restrictlist;
+ bool grouped = agg_info != NULL;
+ bool do_aggregate = rel_agg_input != NULL;
/* We should never try to join two overlapping sets of rels. */
Assert(!bms_overlap(rel1->relids, rel2->relids));
+ /* do_aggregate implies the output to be grouped. */
+ Assert(!do_aggregate || grouped);
+
/* Construct Relids set that identifies the joinrel. */
joinrelids = bms_union(rel1->relids, rel2->relids);
@@ -724,7 +797,18 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* goes with this particular joining.
*/
joinrel = build_join_rel(root, joinrelids, rel1, rel2, sjinfo,
- &restrictlist);
+ &restrictlist, agg_info);
+
+ if (grouped)
+ {
+ /*
+ * Make sure the grouped joinrel has reltarget initialized. Caller
+ * should supply the target for grouped relation, so build_join_rel()
+ * should have omitted its creation.
+ */
+ if (joinrel->reltarget == NULL)
+ set_grouped_joinrel_target(root, joinrel, agg_info);
+ }
/*
* If we've already proven this join is empty, we needn't consider any
@@ -738,13 +822,72 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
/* Add paths to the join relation. */
populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
bms_free(joinrelids);
return joinrel;
}
+static void
+make_join_rel_common_grouped(PlannerInfo *root, RelOptInfoSet *relset1,
+ RelOptInfoSet *relset2,
+ RelAggInfo *agg_info)
+{
+ RelOptInfo *rel1_grouped = NULL;
+ RelOptInfo *rel2_grouped = NULL;
+ bool rel1_grouped_useful = false;
+ bool rel2_grouped_useful = false;
+
+ /*
+ * Retrieve the grouped relations.
+ *
+ * Dummy rel may indicates a join relation that is able to generate
+ * grouped paths as such (i.e. it has valid agg_info), but for which the
+ * path actually could not be created (e.g. only AGG_HASHED strategy was
+ * possible but work_mem was not sufficient for hash table).
+ */
+ if (relset1->rel_grouped)
+ rel1_grouped = relset1->rel_grouped;
+ if (relset2->rel_grouped)
+ rel2_grouped = relset2->rel_grouped;
+
+ rel1_grouped_useful = rel1_grouped != NULL && !IS_DUMMY_REL(rel1_grouped);
+ rel2_grouped_useful = rel2_grouped != NULL && !IS_DUMMY_REL(rel2_grouped);
+
+ /*
+ * Nothing to do?
+ */
+ if (!rel1_grouped_useful && !rel2_grouped_useful)
+ return;
+
+ /*
+ * At maximum one input rel can be grouped (here we don't care if any rel
+ * is eventually dummy, the existence of grouped rel indicates that
+ * aggregates can be pushed down to it). If both were grouped, then
+ * grouping of one side would change the occurrence of the other side's
+ * aggregate transient states on the input of the final aggregation. This
+ * can be handled by adjusting the transient states, but it's not worth
+ * the effort because it's hard to find a use case for this kind of join.
+ *
+ * XXX If the join of two grouped rels is implemented someday, note that
+ * both rels can have aggregates, so it'd be hard to join grouped rel to
+ * non-grouped here: 1) such a "mixed join" would require a special
+ * target, 2) both AGGSPLIT_FINAL_DESERIAL and AGGSPLIT_SIMPLE aggregates
+ * could appear in the target of the final aggregation node, originating
+ * from the grouped and the non-grouped input rel respectively.
+ */
+ if (rel1_grouped && rel2_grouped)
+ return;
+
+ if (rel1_grouped_useful)
+ make_join_rel_common(root, rel1_grouped, relset2->rel_plain, agg_info,
+ NULL);
+ else if (rel2_grouped_useful)
+ make_join_rel_common(root, relset1->rel_plain, rel2_grouped, agg_info,
+ NULL);
+}
+
/*
* make_join_rel
* Find or create a join RelOptInfo that represents the join of
@@ -753,14 +896,101 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* (The join rel may already contain paths generated from other
* pairs of rels that add up to the same set of base rels.)
*
- * NB: will return NULL if attempted join is not valid. This can happen
- * when working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
+ * In addition to creating an ordinary join relation, try to create a
+ * grouped one. There are two strategies to achieve that: join a grouped
+ * relation to plain one, or join two plain relations and apply partial
+ * aggregation to the result.
+ *
+ * NB: will return NULL if attempted join is not valid. This can happen when
+ * working with outer joins, or with IN or EXISTS clauses that have been
+ * turned into joins. Besides that, NULL is also returned if caller is
+ * interested in a grouped relation but there's no useful grouped input
+ * relation.
+ *
+ * Only the plain relation is returned.
+ *
+ * TODO geqo is the only caller interested in the result. We'll need to return
+ * RelOptInfoSet if geqo adopts the aggregate pushdown technique.
*/
RelOptInfo *
-make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+make_join_rel(PlannerInfo *root, RelOptInfoSet *relset1,
+ RelOptInfoSet *relset2)
{
- return make_join_rel_common(root, rel1, rel2);
+ Relids joinrelids;
+ RelAggInfo *agg_info;
+ RelOptInfoSet *joinrelset;
+ RelOptInfo *joinrel_plain;
+ RelOptInfo *rel1 = relset1->rel_plain;
+ RelOptInfo *rel2 = relset2->rel_plain;
+
+ /* 1) form the plain join. */
+ joinrel_plain = make_join_rel_common(root, rel1, rel2, NULL, NULL);
+
+ if (joinrel_plain == NULL)
+ return joinrel_plain;
+
+ /*
+ * We're done if there are no grouping expressions nor aggregates.
+ */
+ if (root->grouped_var_list == NIL)
+ return joinrel_plain;
+
+ /*
+ * If the same grouped joinrel was already formed, just with the base rels
+ * divided between rel1 and rel2 in a different way, we should already
+ * have the matching agg_info.
+ */
+ joinrelids = bms_union(rel1->relids, rel2->relids);
+ joinrelset = find_join_rel(root, joinrelids);
+
+ /*
+ * At the moment we know that non-grouped join exists, so the containing
+ * joinrelset should have been fetched.
+ */
+ Assert(joinrelset != NULL);
+
+ if (joinrelset->rel_grouped != NULL)
+ {
+ /*
+ * The same agg_info should be used for all the rels consisting of
+ * exactly joinrelids.
+ */
+ agg_info = joinrelset->agg_info;
+ }
+ else
+ {
+ /*
+ * agg_info must be created from scratch.
+ */
+ agg_info = create_rel_agg_info(root, joinrel_plain);
+
+ /*
+ * The number of aggregate input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ if (agg_info != NULL)
+ agg_info->input_rows = joinrel_plain->rows;
+ }
+
+ /*
+ * Cannot we build grouped join?
+ */
+ if (agg_info == NULL)
+ return joinrel_plain;
+
+ /*
+ * 2) join two plain rels and aggregate the join paths. Aggregate
+ * push-down only makes sense if the join is not the final one.
+ */
+ if (bms_nonempty_difference(root->all_baserels, joinrelids))
+ make_join_rel_common(root, rel1, rel2, agg_info, joinrel_plain);
+
+ /*
+ * 3) combine plain and grouped relations.
+ */
+ make_join_rel_common_grouped(root, relset1, relset2, agg_info);
+
+ return joinrel_plain;
}
/*
@@ -773,8 +1003,11 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
static void
populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
- SpecialJoinInfo *sjinfo, List *restrictlist)
+ SpecialJoinInfo *sjinfo, List *restrictlist,
+ RelAggInfo *agg_info, RelOptInfo *rel_agg_input)
{
+ bool grouped = agg_info != NULL;
+
/*
* Consider paths using each rel as both outer and inner. Depending on
* the join type, a provably empty outer or inner rel might mean the join
@@ -804,10 +1037,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_INNER, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_INNER, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
break;
case JOIN_LEFT:
if (is_dummy_rel(rel1) ||
@@ -821,10 +1054,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_LEFT, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_RIGHT, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
break;
case JOIN_FULL:
if ((is_dummy_rel(rel1) && is_dummy_rel(rel2)) ||
@@ -835,10 +1068,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_FULL, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_FULL, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
/*
* If there are join quals that aren't mergeable or hashable, we
@@ -871,7 +1104,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_SEMI, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
}
/*
@@ -894,10 +1127,10 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
}
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_UNIQUE_INNER, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
add_paths_to_joinrel(root, joinrel, rel2, rel1,
JOIN_UNIQUE_OUTER, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
}
break;
case JOIN_ANTI:
@@ -912,7 +1145,7 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
mark_dummy_rel(rel2);
add_paths_to_joinrel(root, joinrel, rel1, rel2,
JOIN_ANTI, sjinfo,
- restrictlist);
+ restrictlist, agg_info, rel_agg_input);
break;
default:
/* other values not expected here */
@@ -920,8 +1153,16 @@ populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
break;
}
- /* Apply partitionwise join technique, if possible. */
- try_partitionwise_join(root, rel1, rel2, joinrel, sjinfo, restrictlist);
+ /*
+ * The aggregate push-down feature currently does not support
+ * partition-wise aggregation.
+ */
+ if (grouped)
+ return;
+
+ /* Apply partition-wise join technique, if possible. */
+ try_partitionwise_join(root, rel1, rel2, joinrel, sjinfo,
+ restrictlist);
}
@@ -1121,7 +1362,8 @@ has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel)
foreach(lc, root->initial_rels)
{
- RelOptInfo *rel2 = (RelOptInfo *) lfirst(lc);
+ RelOptInfoSet *relset2 = lfirst_node(RelOptInfoSet, lc);
+ RelOptInfo *rel2 = relset2->rel_plain;
/* ignore rels that are already in "rel" */
if (bms_overlap(rel->relids, rel2->relids))
@@ -1452,7 +1694,7 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
populate_joinrel_with_paths(root, child_rel1, child_rel2,
child_joinrel, child_sjinfo,
- child_restrictlist);
+ child_restrictlist, NULL, NULL);
}
}
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 1b4f7db649..d22de6a909 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -901,6 +901,18 @@ use_physical_tlist(PlannerInfo *root, Path *path, int flags)
}
}
+ /*
+ * If aggregate was pushed down, the target can contain aggregates. The
+ * original target must be preserved then.
+ */
+ foreach(lc, path->pathtarget->exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, Aggref))
+ return false;
+ }
+
return true;
}
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index d6ffa7869d..d7008b33b9 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -47,6 +47,8 @@ typedef struct PostponedQual
} PostponedQual;
+static void create_aggregate_grouped_var_infos(PlannerInfo *root);
+static void create_grouping_expr_grouped_var_infos(PlannerInfo *root);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
@@ -97,10 +99,9 @@ static void check_hashjoinable(RestrictInfo *restrictinfo);
* jtnode. Internally, the function recurses through the jointree.
*
* At the end of this process, there should be one baserel RelOptInfo for
- * every non-join RTE that is used in the query. Therefore, this routine
- * is the only place that should call build_simple_rel with reloptkind
- * RELOPT_BASEREL. (Note: build_simple_rel recurses internally to build
- * "other rel" RelOptInfos for the members of any appendrels we find here.)
+ * every non-grouped non-join RTE that is used in the query. (Note:
+ * build_simple_rel recurses internally to build "other rel" RelOptInfos for
+ * the members of any appendrels we find here.)
*/
void
add_base_rels_to_query(PlannerInfo *root, Node *jtnode)
@@ -242,6 +243,261 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
}
}
+/*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate as well as for
+ * each possible grouping expression.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+extern void
+setup_aggregate_pushdown(PlannerInfo *root)
+{
+ ListCell *lc;
+
+ /*
+ * Isn't user interested in the aggregate push-down feature?
+ */
+ if (!enable_agg_pushdown)
+ return;
+
+ /* The feature can only be applied to grouped aggregation. */
+ if (!root->parse->groupClause)
+ return;
+
+ /*
+ * Grouping sets require multiple different groupings but the base
+ * relation can only generate one.
+ */
+ if (root->parse->groupingSets)
+ return;
+
+ /*
+ * SRF is not allowed in the aggregate argument and we don't even want it
+ * in the GROUP BY clause, so forbid it in general. It needs to be
+ * analyzed if evaluation of a GROUP BY clause containing SRF below the
+ * query targetlist would be correct. Currently it does not seem to be an
+ * important use case.
+ */
+ if (root->parse->hasTargetSRFs)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_aggregate_grouped_var_infos(root);
+
+ /* Isn't there any aggregate to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Create GroupedVarInfo per grouping expression. */
+ create_grouping_expr_grouped_var_infos(root);
+
+ /* Isn't there any useful grouping expression for aggregate push-down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /*
+ * Now that we know that grouping can be pushed down, search for the
+ * maximum sortgroupref. The base relations may need it if extra grouping
+ * expressions get added to them.
+ */
+ Assert(root->max_sortgroupref == 0);
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc);
+
+ if (te->ressortgroupref > root->max_sortgroupref)
+ root->max_sortgroupref = te->ressortgroupref;
+ }
+}
+
+/*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ */
+static void
+create_aggregate_grouped_var_infos(PlannerInfo *root)
+{
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES |
+ PVC_INCLUDE_WINDOWFUNCS);
+
+ /*
+ * Although GroupingFunc is related to root->parse->groupingSets, this
+ * field does not necessarily reflect its presence.
+ */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, GroupingFunc))
+ return;
+ }
+
+ /*
+ * Aggregates within the HAVING clause need to be processed in the same
+ * way as those in the main targetlist.
+ *
+ * Note that the contained aggregates will be pushed down, but the
+ * containing HAVING clause must be ignored until the aggregation is
+ * finalized.
+ */
+ if (root->parse->havingQual != NULL)
+ {
+ List *having_exprs;
+
+ having_exprs = pull_var_clause((Node *) root->parse->havingQual,
+ PVC_INCLUDE_AGGREGATES);
+ if (having_exprs != NIL)
+ tlist_exprs = list_concat(tlist_exprs, having_exprs);
+ }
+
+ if (tlist_exprs == NIL)
+ return;
+
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ /*
+ * tlist_exprs may also contain Vars or WindowFuncs, but we only need
+ * Aggrefs.
+ */
+ if (IsA(expr, Var) ||IsA(expr, WindowFunc))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct)
+ {
+ /*
+ * Aggregation push-down is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ gvi = lfirst_node(GroupedVarInfo, lc2);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ gvi = makeNode(GroupedVarInfo);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ gvi->gv_eval_at = NULL;
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+}
+
+/*
+ * Create GroupedVarInfo for each expression usable as grouping key.
+ *
+ * In addition to the expressions of the query targetlist, group_pathkeys is
+ * also considered the source of grouping expressions. That increases the
+ * chance to get the relation output grouped.
+ */
+static void
+create_grouping_expr_grouped_var_infos(PlannerInfo *root)
+{
+ ListCell *l1,
+ *l2;
+ List *exprs = NIL;
+ List *sortgrouprefs = NIL;
+
+ /*
+ * Make sure GroupedVarInfo exists for each expression usable as grouping
+ * key.
+ */
+ foreach(l1, root->parse->groupClause)
+ {
+ SortGroupClause *sgClause;
+ TargetEntry *te;
+ Index sortgroupref;
+
+ sgClause = lfirst_node(SortGroupClause, l1);
+ te = get_sortgroupclause_tle(sgClause, root->processed_tlist);
+ sortgroupref = te->ressortgroupref;
+
+ Assert(sortgroupref > 0);
+
+ /*
+ * Non-zero sortgroupref does not necessarily imply grouping
+ * expression: data can also be sorted by aggregate.
+ */
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ /*
+ * The aggregate push-down feature currently supports only plain Vars
+ * as grouping expressions.
+ */
+ if (!IsA(te->expr, Var))
+ {
+ root->grouped_var_list = NIL;
+ return;
+ }
+
+ exprs = lappend(exprs, te->expr);
+ sortgrouprefs = lappend_int(sortgrouprefs, sortgroupref);
+ }
+
+ /*
+ * Construct GroupedVarInfo for each expression.
+ */
+ forboth(l1, exprs, l2, sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, l1);
+ int sortgroupref = lfirst_int(l2);
+ GroupedVarInfo *gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvexpr = (Expr *) copyObject(var);
+ gvi->sortgroupref = sortgroupref;
+
+ /* Find out where the expression should be evaluated. */
+ gvi->gv_eval_at = bms_make_singleton(var->varno);
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+}
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c
index 86617099df..1f2f5b0832 100644
--- a/src/backend/optimizer/plan/planagg.c
+++ b/src/backend/optimizer/plan/planagg.c
@@ -349,6 +349,7 @@ build_minmax_path(PlannerInfo *root, MinMaxAggInfo *mminfo,
List *tlist;
NullTest *ntest;
SortGroupClause *sortcl;
+ RelOptInfoSet *final_relset;
RelOptInfo *final_rel;
Path *sorted_path;
Cost path_cost;
@@ -442,7 +443,8 @@ build_minmax_path(PlannerInfo *root, MinMaxAggInfo *mminfo,
subroot->tuple_fraction = 1.0;
subroot->limit_tuples = 1.0;
- final_rel = query_planner(subroot, tlist, minmax_qp_callback, NULL);
+ final_relset = query_planner(subroot, tlist, minmax_qp_callback, NULL);
+ final_rel = final_relset->rel_plain;
/*
* Since we didn't go through subquery_planner() to handle the subquery,
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 3cedd01c98..ad5b18eb61 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -53,13 +53,14 @@
* qp_callback once we have completed merging the query's equivalence classes.
* (We cannot construct canonical pathkeys until that's done.)
*/
-RelOptInfo *
+RelOptInfoSet *
query_planner(PlannerInfo *root, List *tlist,
query_pathkeys_callback qp_callback, void *qp_extra)
{
Query *parse = root->parse;
List *joinlist;
RelOptInfo *final_rel;
+ RelOptInfoSet *final_relset;
/*
* Init planner lists to empty.
@@ -77,6 +78,7 @@ query_planner(PlannerInfo *root, List *tlist,
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
@@ -147,7 +149,10 @@ query_planner(PlannerInfo *root, List *tlist,
*/
(*qp_callback) (root, qp_extra);
- return final_rel;
+ final_relset = makeNode(RelOptInfoSet);
+ final_relset->rel_plain = final_rel;
+
+ return final_relset;
}
}
}
@@ -260,14 +265,25 @@ query_planner(PlannerInfo *root, List *tlist,
extract_restriction_or_clauses(root);
/*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, setup
+ * root->grouped_var_list.
+ *
+ * The base relations should be fully initialized now, so that we have
+ * enough info to decide whether grouping is possible.
+ */
+ setup_aggregate_pushdown(root);
+
+ /*
* Ready to do the primary planning.
*/
- final_rel = make_one_rel(root, joinlist);
+ final_relset = make_one_rel(root, joinlist);
+ final_rel = final_relset->rel_plain;
/* Check that we got at least one usable path */
if (!final_rel || !final_rel->cheapest_total_path ||
final_rel->cheapest_total_path->param_info != NULL)
elog(ERROR, "failed to construct the join relation");
- return final_rel;
+ return final_relset;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 0614f31be4..fcfac56564 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -149,7 +149,7 @@ static double get_number_of_groups(PlannerInfo *root,
grouping_sets_data *gd,
List *target_list);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
- RelOptInfo *input_rel,
+ RelOptInfoSet *input_relset,
PathTarget *target,
bool target_parallel_safe,
const AggClauseCosts *agg_costs,
@@ -162,7 +162,7 @@ static RelOptInfo *make_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
PathTarget *target, bool target_parallel_safe,
Node *havingQual);
static void create_ordinary_grouping_paths(PlannerInfo *root,
- RelOptInfo *input_rel,
+ RelOptInfoSet *input_relset,
RelOptInfo *grouped_rel,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd,
@@ -630,6 +630,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
memset(root->upper_rels, 0, sizeof(root->upper_rels));
memset(root->upper_targets, 0, sizeof(root->upper_targets));
root->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
root->grouping_map = NULL;
root->minmax_aggs = NIL;
root->qual_security_level = 0;
@@ -1683,6 +1684,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
List *final_targets;
List *final_targets_contain_srfs;
bool final_target_parallel_safe;
+ RelOptInfoSet *current_relset;
RelOptInfo *current_rel;
RelOptInfo *final_rel;
ListCell *lc;
@@ -1902,8 +1904,9 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
* We also generate (in standard_qp_callback) pathkey representations
* of the query's sort clause, distinct clause, etc.
*/
- current_rel = query_planner(root, tlist,
- standard_qp_callback, &qp_extra);
+ current_relset = query_planner(root, tlist,
+ standard_qp_callback, &qp_extra);
+ current_rel = current_relset->rel_plain;
/*
* Convert the query's result tlist into PathTarget format.
@@ -2043,7 +2046,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
if (have_grouping)
{
current_rel = create_grouping_paths(root,
- current_rel,
+ current_relset,
grouping_target,
grouping_target_parallel_safe,
&agg_costs,
@@ -3674,13 +3677,14 @@ get_number_of_groups(PlannerInfo *root,
*/
static RelOptInfo *
create_grouping_paths(PlannerInfo *root,
- RelOptInfo *input_rel,
+ RelOptInfoSet *input_relset,
PathTarget *target,
bool target_parallel_safe,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd)
{
Query *parse = root->parse;
+ RelOptInfo *input_rel = input_relset->rel_plain;
RelOptInfo *grouped_rel;
RelOptInfo *partially_grouped_rel;
@@ -3765,7 +3769,7 @@ create_grouping_paths(PlannerInfo *root,
else
extra.patype = PARTITIONWISE_AGGREGATE_NONE;
- create_ordinary_grouping_paths(root, input_rel, grouped_rel,
+ create_ordinary_grouping_paths(root, input_relset, grouped_rel,
agg_costs, gd, &extra,
&partially_grouped_rel);
}
@@ -3921,13 +3925,14 @@ create_degenerate_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
* function creates, or to NULL if it doesn't create one.
*/
static void
-create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
+create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfoSet *input_relset,
RelOptInfo *grouped_rel,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd,
GroupPathExtraData *extra,
RelOptInfo **partially_grouped_rel_p)
{
+ RelOptInfo *input_rel = input_relset->rel_plain;
Path *cheapest_path = input_rel->cheapest_total_path;
RelOptInfo *partially_grouped_rel = NULL;
double dNumGroups;
@@ -3971,13 +3976,25 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
if ((extra->flags & GROUPING_CAN_PARTIAL_AGG) != 0)
{
bool force_rel_creation;
+ RelOptInfo *input_rel_grouped = input_relset->rel_grouped;
+ bool have_agg_pushdown_paths;
/*
- * If we're doing partitionwise aggregation at this level, force
- * creation of a partially_grouped_rel so we can add partitionwise
- * paths to it.
+ * Check if the aggregate push-down feature succeeded to generate any
+ * paths. Dummy relation can appear here because grouped paths are not
+ * guaranteed to exist for a relation.
*/
- force_rel_creation = (patype == PARTITIONWISE_AGGREGATE_PARTIAL);
+ have_agg_pushdown_paths = input_rel_grouped != NULL &&
+ input_rel_grouped->pathlist != NIL &&
+ !IS_DUMMY_REL(input_rel_grouped);
+
+ /*
+ * If we're doing partitionwise aggregation at this level or if
+ * aggregate push-down succeeded to create some paths, force creation
+ * of a partially_grouped_rel so we can add the related paths to it.
+ */
+ force_rel_creation = patype == PARTITIONWISE_AGGREGATE_PARTIAL ||
+ have_agg_pushdown_paths;
partially_grouped_rel =
create_partial_grouping_paths(root,
@@ -3986,6 +4003,23 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
gd,
extra,
force_rel_creation);
+
+ /*
+ * No further processing is needed for paths provided by the aggregate
+ * push-down feature. Simply add them to the partially grouped
+ * relation.
+ */
+ if (have_agg_pushdown_paths)
+ {
+ ListCell *lc;
+
+ foreach(lc, input_rel_grouped->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ add_path(partially_grouped_rel, path);
+ }
+ }
}
/* Set out parameter. */
@@ -4010,10 +4044,14 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
/* Gather any partially grouped partial paths. */
if (partially_grouped_rel && partially_grouped_rel->partial_pathlist)
- {
gather_grouping_paths(root, partially_grouped_rel);
+
+ /*
+ * The non-partial paths can come either from the Gather above or from
+ * aggregate push-down.
+ */
+ if (partially_grouped_rel && partially_grouped_rel->pathlist)
set_cheapest(partially_grouped_rel);
- }
/*
* Estimate number of groups.
@@ -6092,7 +6130,7 @@ plan_cluster_use_sort(Oid tableOid, Oid indexOid)
comparisonCost = 2.0 * (indexExprCost.startup + indexExprCost.per_tuple);
/* Estimate the cost of seq scan + sort */
- seqScanPath = create_seqscan_path(root, rel, NULL, 0);
+ seqScanPath = create_seqscan_path(root, rel, NULL, 0, NULL, NULL);
cost_sort(&seqScanAndSortPath, root, NIL,
seqScanPath->total_cost, rel->tuples, rel->reltarget->width,
comparisonCost, maintenance_work_mem, -1.0);
@@ -6101,7 +6139,7 @@ plan_cluster_use_sort(Oid tableOid, Oid indexOid)
indexScanPath = create_index_path(root, indexInfo,
NIL, NIL, NIL, NIL, NIL,
ForwardScanDirection, false,
- NULL, 1.0, false);
+ NULL, 1.0, false, NULL, NULL);
return (seqScanAndSortPath.total_cost < indexScanPath->path.total_cost);
}
@@ -7114,6 +7152,7 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
for (cnt_parts = 0; cnt_parts < nparts; cnt_parts++)
{
RelOptInfo *child_input_rel = input_rel->part_rels[cnt_parts];
+ RelOptInfoSet *child_input_relset;
PathTarget *child_target = copy_pathtarget(target);
AppendRelInfo **appinfos;
int nappinfos;
@@ -7172,12 +7211,17 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
continue;
}
+ child_input_relset = makeNode(RelOptInfoSet);
+ child_input_relset->rel_plain = child_input_rel;
+
/* Create grouping paths for this child relation. */
- create_ordinary_grouping_paths(root, child_input_rel,
+ create_ordinary_grouping_paths(root, child_input_relset,
child_grouped_rel,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
+ pfree(child_input_relset);
+
if (child_partially_grouped_rel)
{
partially_grouped_live_children =
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 0213a37670..d618007929 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -2300,6 +2300,39 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, Aggref))
+ {
+ Aggref *aggref = castNode(Aggref, node);
+
+ /*
+ * The upper plan targetlist can contain Aggref whose value has
+ * already been evaluated by the subplan. However this can only happen
+ * with specific value of aggsplit.
+ */
+ if (aggref->aggsplit == AGGSPLIT_INITIAL_SERIAL)
+ {
+ /* See if the Aggref has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ }
+
+ /* No referent found for Aggref */
+ elog(ERROR, "Aggref not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index aebe162713..9886c18bbd 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -891,6 +891,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
memset(subroot->upper_rels, 0, sizeof(subroot->upper_rels));
memset(subroot->upper_targets, 0, sizeof(subroot->upper_targets));
subroot->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
subroot->grouping_map = NULL;
subroot->minmax_aggs = NIL;
subroot->qual_security_level = 0;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index b57de6b4c6..50bdf69b2e 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -17,6 +17,8 @@
#include <math.h>
#include "miscadmin.h"
+#include "access/sysattr.h"
+#include "catalog/pg_constraint.h"
#include "foreign/fdwapi.h"
#include "nodes/extensible.h"
#include "nodes/nodeFuncs.h"
@@ -58,7 +60,6 @@ static List *reparameterize_pathlist_by_child(PlannerInfo *root,
List *pathlist,
RelOptInfo *child_rel);
-
/*****************************************************************************
* MISC. PATH UTILITIES
*****************************************************************************/
@@ -953,13 +954,15 @@ add_partial_path_precheck(RelOptInfo *parent_rel, Cost total_cost,
*/
Path *
create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
- Relids required_outer, int parallel_workers)
+ Relids required_outer, int parallel_workers,
+ RelOptInfo *rel_grouped, RelAggInfo *agg_info)
{
Path *pathnode = makeNode(Path);
pathnode->pathtype = T_SeqScan;
- pathnode->parent = rel;
- pathnode->pathtarget = rel->reltarget;
+ pathnode->parent = rel_grouped == NULL ? rel : rel_grouped;
+ pathnode->pathtarget = rel_grouped == NULL ? rel->reltarget :
+ agg_info->input;
pathnode->param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->parallel_aware = parallel_workers > 0 ? true : false;
@@ -1019,6 +1022,10 @@ create_samplescan_path(PlannerInfo *root, RelOptInfo *rel, Relids required_outer
* estimates of caching behavior.
* 'partial_path' is true if constructing a parallel index scan path.
*
+ * If 'rel_grouped' is passed, apply partial aggregation to each path and add
+ * the path to this relation instead of 'rel'. 'rel_agg_input' describes the
+ * aggregation input. 'agg_info' must be passed in such a case too.
+ *
* Returns the new path node.
*/
IndexPath *
@@ -1033,7 +1040,9 @@ create_index_path(PlannerInfo *root,
bool indexonly,
Relids required_outer,
double loop_count,
- bool partial_path)
+ bool partial_path,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info)
{
IndexPath *pathnode = makeNode(IndexPath);
RelOptInfo *rel = index->rel;
@@ -1041,8 +1050,9 @@ create_index_path(PlannerInfo *root,
*indexqualcols;
pathnode->path.pathtype = indexonly ? T_IndexOnlyScan : T_IndexScan;
- pathnode->path.parent = rel;
- pathnode->path.pathtarget = rel->reltarget;
+ pathnode->path.parent = rel_grouped == NULL ? rel : rel_grouped;
+ pathnode->path.pathtarget = rel_grouped == NULL ? rel->reltarget :
+ agg_info->input;
pathnode->path.param_info = get_baserel_parampathinfo(root, rel,
required_outer);
pathnode->path.parallel_aware = false;
@@ -1186,8 +1196,8 @@ create_bitmap_or_path(PlannerInfo *root,
* Creates a path corresponding to a scan by TID, returning the pathnode.
*/
TidPath *
-create_tidscan_path(PlannerInfo *root, RelOptInfo *rel, List *tidquals,
- Relids required_outer)
+create_tidscan_path(PlannerInfo *root, RelOptInfo *rel,
+ List *tidquals, Relids required_outer)
{
TidPath *pathnode = makeNode(TidPath);
@@ -1342,7 +1352,8 @@ append_startup_cost_compare(const void *a, const void *b)
/*
* create_merge_append_path
* Creates a path corresponding to a MergeAppend plan, returning the
- * pathnode.
+ * pathnode. target can be supplied by caller. If NULL is passed, the field
+ * is set to rel->reltarget.
*/
MergeAppendPath *
create_merge_append_path(PlannerInfo *root,
@@ -1533,7 +1544,9 @@ create_unique_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
MemoryContext oldcontext;
int numCols;
- /* Caller made a mistake if subpath isn't cheapest_total ... */
+ /*
+ * Caller made a mistake if subpath isn't cheapest_total.
+ */
Assert(subpath == rel->cheapest_total_path);
Assert(subpath->parent == rel);
/* ... or if SpecialJoinInfo is the wrong one */
@@ -2180,6 +2193,7 @@ calc_non_nestloop_required_outer(Path *outer_path, Path *inner_path)
* relations.
*
* 'joinrel' is the join relation.
+ * 'target' is the join path target
* 'jointype' is the type of join required
* 'workspace' is the result from initial_cost_nestloop
* 'extra' contains various information about the join
@@ -2194,6 +2208,7 @@ calc_non_nestloop_required_outer(Path *outer_path, Path *inner_path)
NestPath *
create_nestloop_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -2234,7 +2249,7 @@ create_nestloop_path(PlannerInfo *root,
pathnode->path.pathtype = T_NestLoop;
pathnode->path.parent = joinrel;
- pathnode->path.pathtarget = joinrel->reltarget;
+ pathnode->path.pathtarget = target;
pathnode->path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
@@ -2266,6 +2281,7 @@ create_nestloop_path(PlannerInfo *root,
* two relations
*
* 'joinrel' is the join relation
+ * 'target' is the join path target
* 'jointype' is the type of join required
* 'workspace' is the result from initial_cost_mergejoin
* 'extra' contains various information about the join
@@ -2282,6 +2298,7 @@ create_nestloop_path(PlannerInfo *root,
MergePath *
create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -2298,7 +2315,7 @@ create_mergejoin_path(PlannerInfo *root,
pathnode->jpath.path.pathtype = T_MergeJoin;
pathnode->jpath.path.parent = joinrel;
- pathnode->jpath.path.pathtarget = joinrel->reltarget;
+ pathnode->jpath.path.pathtarget = target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
@@ -2334,6 +2351,7 @@ create_mergejoin_path(PlannerInfo *root,
* Creates a pathnode corresponding to a hash join between two relations.
*
* 'joinrel' is the join relation
+ * 'target' is the join path target
* 'jointype' is the type of join required
* 'workspace' is the result from initial_cost_hashjoin
* 'extra' contains various information about the join
@@ -2348,6 +2366,7 @@ create_mergejoin_path(PlannerInfo *root,
HashPath *
create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -2362,7 +2381,7 @@ create_hashjoin_path(PlannerInfo *root,
pathnode->jpath.path.pathtype = T_HashJoin;
pathnode->jpath.path.parent = joinrel;
- pathnode->jpath.path.pathtarget = joinrel->reltarget;
+ pathnode->jpath.path.pathtarget = target;
pathnode->jpath.path.param_info =
get_joinrel_parampathinfo(root,
joinrel,
@@ -2444,8 +2463,8 @@ create_projection_path(PlannerInfo *root,
* Note: in the latter case, create_projection_plan has to recheck our
* conclusion; see comments therein.
*/
- if (is_projection_capable_path(subpath) ||
- equal(oldtarget->exprs, target->exprs))
+ if ((is_projection_capable_path(subpath) ||
+ equal(oldtarget->exprs, target->exprs)))
{
/* No separate Result node needed */
pathnode->dummypp = true;
@@ -2830,8 +2849,7 @@ create_agg_path(PlannerInfo *root,
pathnode->path.pathtype = T_Agg;
pathnode->path.parent = rel;
pathnode->path.pathtarget = target;
- /* For now, assume we are above any joins, so no parameterization */
- pathnode->path.param_info = NULL;
+ pathnode->path.param_info = subpath->param_info;
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe;
@@ -2864,6 +2882,152 @@ create_agg_path(PlannerInfo *root,
}
/*
+ * Apply AGG_SORTED aggregation path to subpath if it's suitably sorted.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+AggPath *
+create_agg_sorted_path(PlannerInfo *root, Path *subpath,
+ RelAggInfo *agg_info)
+{
+ RelOptInfo *rel;
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ ListCell *lc1;
+ List *key_subset = NIL;
+ AggPath *result = NULL;
+
+ rel = subpath->parent;
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, (Node *) agg_exprs, aggsplit, &agg_costs);
+
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until the
+ * final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath, target,
+ AGG_SORTED, aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+
+ return result;
+}
+
+/*
+ * Apply AGG_HASHED aggregation to subpath.
+ *
+ * Arguments have the same meaning as those of create_agg_sorted_path.
+ */
+AggPath *
+create_agg_hashed_path(PlannerInfo *root, Path *subpath,
+ RelAggInfo *agg_info)
+{
+ RelOptInfo *rel;
+ bool can_hash;
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ Size hashaggtablesize;
+ Query *parse = root->parse;
+ AggPath *result = NULL;
+
+ rel = subpath->parent;
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, agg_exprs, aggsplit, &agg_costs);
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until
+ * the final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath,
+ target,
+ AGG_HASHED,
+ aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+
+ /*
+ * The agg path should require no fewer parameters than the plain
+ * one.
+ */
+ result->path.param_info = subpath->param_info;
+ }
+ }
+
+ return result;
+}
+
+/*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
*
@@ -3548,7 +3712,8 @@ reparameterize_path(PlannerInfo *root, Path *path,
switch (path->pathtype)
{
case T_SeqScan:
- return create_seqscan_path(root, rel, required_outer, 0);
+ return create_seqscan_path(root, rel, required_outer, 0, NULL,
+ NULL);
case T_SampleScan:
return (Path *) create_samplescan_path(root, rel, required_outer);
case T_IndexScan:
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index f04c6b76f4..c54964ce64 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -17,16 +17,21 @@
#include <limits.h>
#include "miscadmin.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_constraint.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
+#include "optimizer/optimizer.h"
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/placeholder.h"
#include "optimizer/plancat.h"
+#include "optimizer/planner.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+#include "parser/parse_oper.h"
#include "partitioning/partbounds.h"
#include "utils/hsearch.h"
@@ -34,7 +39,7 @@
typedef struct JoinHashEntry
{
Relids join_relids; /* hash key --- MUST BE FIRST */
- RelOptInfo *join_rel;
+ RelOptInfoSet *join_relset;
} JoinHashEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
@@ -54,7 +59,7 @@ static List *subbuild_joinrel_joinlist(RelOptInfo *joinrel,
List *new_joininfo);
static void set_foreign_rel_properties(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel);
-static void add_join_rel(PlannerInfo *root, RelOptInfo *joinrel);
+static void add_join_rel(PlannerInfo *root, RelOptInfoSet *joinrelset);
static void build_joinrel_partition_info(RelOptInfo *joinrel,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
List *restrictlist, JoinType jointype);
@@ -63,6 +68,9 @@ static void build_child_join_reltarget(PlannerInfo *root,
RelOptInfo *childrel,
int nappinfos,
AppendRelInfo **appinfos);
+static bool init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p);
/*
@@ -323,6 +331,97 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
}
/*
+ * build_simple_grouped_rel
+ * Construct a new RelOptInfo for a grouped base relation out of an
+ * existing non-grouped relation.
+ */
+void
+build_simple_grouped_rel(PlannerInfo *root, RelOptInfoSet *relset)
+{
+ RangeTblEntry *rte;
+ RelOptInfo *rel_plain,
+ *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /* Isn't there any grouping expression to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ rel_plain = relset->rel_plain;
+
+ /*
+ * Not all RTE kinds are supported when grouping is considered.
+ *
+ * TODO Consider relaxing some of these restrictions.
+ */
+ rte = root->simple_rte_array[rel_plain->relid];
+ if (rte->rtekind != RTE_RELATION ||
+ rte->relkind == RELKIND_FOREIGN_TABLE ||
+ rte->tablesample != NULL)
+ return;;
+
+ /*
+ * Grouped append relation is not supported yet.
+ */
+ if (rte->inh)
+ return;
+
+ /*
+ * Currently we do not support child relations ("other rels").
+ */
+ if (rel_plain->reloptkind != RELOPT_BASEREL)
+ return;
+
+ /*
+ * Prepare the information we need for aggregation of the rel contents.
+ */
+ agg_info = create_rel_agg_info(root, relset->rel_plain);
+ if (agg_info == NULL)
+ return;
+
+ /*
+ * TODO Consider if 1) a flat copy is o.k., 2) it's safer in terms of
+ * adding new fields to RelOptInfo) to copy everything and then reset some
+ * fields, or to zero the structure and copy individual fields.
+ */
+ rel_grouped = makeNode(RelOptInfo);
+ memcpy(rel_grouped, rel_plain, sizeof(RelOptInfo));
+
+ /*
+ * Note on consider_startup: while the AGG_HASHED strategy needs the whole
+ * relation, AGG_SORTED does not. Therefore we do not force
+ * consider_startup to false.
+ */
+
+ /*
+ * Set the appropriate target for grouped paths.
+ *
+ * The aggregation paths will get their input target from agg_info, so
+ * store it too.
+ */
+ rel_grouped->reltarget = agg_info->target;
+
+ /*
+ * Grouped paths must not be mixed with the plain ones.
+ */
+ rel_grouped->pathlist = NIL;
+ rel_grouped->partial_pathlist = NIL;
+ rel_grouped->cheapest_startup_path = NULL;
+ rel_grouped->cheapest_total_path = NULL;
+ rel_grouped->cheapest_unique_path = NULL;
+ rel_grouped->cheapest_parameterized_paths = NIL;
+
+ relset->rel_grouped = rel_grouped;
+
+ /*
+ * The number of aggregation input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ agg_info->input_rows = rel_plain->rows;
+ relset->agg_info = agg_info;
+}
+
+/*
* find_base_rel
* Find a base or other relation entry, which must already exist.
*/
@@ -371,16 +470,16 @@ build_join_rel_hash(PlannerInfo *root)
/* Insert all the already-existing joinrels */
foreach(l, root->join_rel_list)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
+ RelOptInfoSet *relset = (RelOptInfoSet *) lfirst(l);
JoinHashEntry *hentry;
bool found;
hentry = (JoinHashEntry *) hash_search(hashtab,
- &(rel->relids),
+ &(relset->rel_plain->relids),
HASH_ENTER,
&found);
Assert(!found);
- hentry->join_rel = rel;
+ hentry->join_relset = relset;
}
root->join_rel_hash = hashtab;
@@ -391,7 +490,7 @@ build_join_rel_hash(PlannerInfo *root)
* Returns relation entry corresponding to 'relids' (a set of RT indexes),
* or NULL if none exists. This is for join relations.
*/
-RelOptInfo *
+RelOptInfoSet *
find_join_rel(PlannerInfo *root, Relids relids)
{
/*
@@ -419,7 +518,13 @@ find_join_rel(PlannerInfo *root, Relids relids)
HASH_FIND,
NULL);
if (hentry)
- return hentry->join_rel;
+ {
+ RelOptInfoSet *result = hentry->join_relset;;
+
+ /* The plain relation should always be there. */
+ Assert(result->rel_plain != NULL);
+ return result;
+ }
}
else
{
@@ -427,10 +532,10 @@ find_join_rel(PlannerInfo *root, Relids relids)
foreach(l, root->join_rel_list)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
+ RelOptInfoSet *result = (RelOptInfoSet *) lfirst(l);
- if (bms_equal(rel->relids, relids))
- return rel;
+ if (bms_equal(result->rel_plain->relids, relids))
+ return result;
}
}
@@ -493,23 +598,26 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* PlannerInfo. Also add it to the auxiliary hashtable if there is one.
*/
static void
-add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
+add_join_rel(PlannerInfo *root, RelOptInfoSet *joinrelset)
{
- /* GEQO requires us to append the new joinrel to the end of the list! */
- root->join_rel_list = lappend(root->join_rel_list, joinrel);
+ /*
+ * GEQO requires us to append the new joinrel to the end of the list!
+ */
+ root->join_rel_list = lappend(root->join_rel_list, joinrelset);
/* store it into the auxiliary hashtable if there is one. */
if (root->join_rel_hash)
{
JoinHashEntry *hentry;
bool found;
+ Relids relids = joinrelset->rel_plain->relids;
hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
- &(joinrel->relids),
+ &relids,
HASH_ENTER,
&found);
Assert(!found);
- hentry->join_rel = joinrel;
+ hentry->join_relset = joinrelset;
}
}
@@ -525,6 +633,8 @@ add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
* 'restrictlist_ptr': result variable. If not NULL, *restrictlist_ptr
* receives the list of RestrictInfo nodes that apply to this
* particular pair of joinable relations.
+ * 'agg_info' contains information needed for the join to form grouped
+ * paths. If NULL, the join is not grouped.
*
* restrictlist_ptr makes the routine's API a little grotty, but it saves
* duplicated calculation of the restrictlist...
@@ -535,10 +645,20 @@ build_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr)
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info)
{
+ RelOptInfoSet *joinrelset;
+ bool new_set = false;
RelOptInfo *joinrel;
List *restrictlist;
+ bool grouped = agg_info != NULL;
+ bool create_target;
+
+ /*
+ * Target for grouped relation will be supplied by caller.
+ */
+ create_target = !grouped;
/* This function should be used only for join between parents. */
Assert(!IS_OTHER_REL(outer_rel) && !IS_OTHER_REL(inner_rel));
@@ -546,7 +666,19 @@ build_join_rel(PlannerInfo *root,
/*
* See if we already have a joinrel for this set of base rels.
*/
- joinrel = find_join_rel(root, joinrelids);
+ joinrelset = find_join_rel(root, joinrelids);
+ if (joinrelset == NULL)
+ {
+ /*
+ * The plain joinrel should be the first one to be added to the set.
+ */
+ Assert(!grouped);
+
+ joinrelset = makeNode(RelOptInfoSet);
+ new_set = true;
+ }
+
+ joinrel = !grouped ? joinrelset->rel_plain : joinrelset->rel_grouped;
if (joinrel)
{
@@ -573,7 +705,7 @@ build_join_rel(PlannerInfo *root,
joinrel->consider_startup = (root->tuple_fraction > 0);
joinrel->consider_param_startup = false;
joinrel->consider_parallel = false;
- joinrel->reltarget = create_empty_pathtarget();
+ joinrel->reltarget = NULL;
joinrel->pathlist = NIL;
joinrel->ppilist = NIL;
joinrel->partial_pathlist = NIL;
@@ -638,9 +770,13 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ if (create_target)
+ {
+ joinrel->reltarget = create_empty_pathtarget();
+ build_joinrel_tlist(root, joinrel, outer_rel);
+ build_joinrel_tlist(root, joinrel, inner_rel);
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ }
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -676,45 +812,68 @@ build_join_rel(PlannerInfo *root,
sjinfo->jointype);
/*
- * Set estimates of the joinrel's size.
+ * Assign the joinrel to the set.
*/
- set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
- sjinfo, restrictlist);
+ if (!grouped)
+ joinrelset->rel_plain = joinrel;
+ else
+ {
+ joinrelset->rel_grouped = joinrel;
+ joinrelset->agg_info = agg_info;
+ }
- /*
- * Set the consider_parallel flag if this joinrel could potentially be
- * scanned within a parallel worker. If this flag is false for either
- * inner_rel or outer_rel, then it must be false for the joinrel also.
- * Even if both are true, there might be parallel-restricted expressions
- * in the targetlist or quals.
- *
- * Note that if there are more than two rels in this relation, they could
- * be divided between inner_rel and outer_rel in any arbitrary way. We
- * assume this doesn't matter, because we should hit all the same baserels
- * and joinclauses while building up to this joinrel no matter which we
- * take; therefore, we should make the same decision here however we get
- * here.
- */
- if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
- is_parallel_safe(root, (Node *) restrictlist) &&
- is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
- joinrel->consider_parallel = true;
+ if (new_set)
+ {
+ /* Add the joinrelset to the PlannerInfo. */
+ add_join_rel(root, joinrelset);
- /* Add the joinrel to the PlannerInfo. */
- add_join_rel(root, joinrel);
+ /*
+ * Also, if dynamic-programming join search is active, add the new
+ * joinrelset to the appropriate sublist. Note: you might think the
+ * Assert on number of members should be for equality, but some of the
+ * level 1 rels might have been joinrels already, so we can only
+ * assert <=.
+ */
+ if (root->join_rel_level)
+ {
+ Assert(root->join_cur_level > 0);
+ Assert(root->join_cur_level <= bms_num_members(joinrelids));
+ root->join_rel_level[root->join_cur_level] =
+ lappend(root->join_rel_level[root->join_cur_level],
+ joinrelset);
+ }
+ }
/*
- * Also, if dynamic-programming join search is active, add the new joinrel
- * to the appropriate sublist. Note: you might think the Assert on number
- * of members should be for equality, but some of the level 1 rels might
- * have been joinrels already, so we can only assert <=.
+ * Set estimates of the joinrel's size.
+ *
+ * XXX set_joinrel_size_estimates() claims to need reltarget but it does
+ * not seem to actually use it. Should we call it unconditionally so that
+ * callers of build_join_rel() do not have to care?
*/
- if (root->join_rel_level)
+ if (create_target)
{
- Assert(root->join_cur_level > 0);
- Assert(root->join_cur_level <= bms_num_members(joinrel->relids));
- root->join_rel_level[root->join_cur_level] =
- lappend(root->join_rel_level[root->join_cur_level], joinrel);
+ set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
+ sjinfo, restrictlist);
+
+ /*
+ * Set the consider_parallel flag if this joinrel could potentially be
+ * scanned within a parallel worker. If this flag is false for either
+ * inner_rel or outer_rel, then it must be false for the joinrel also.
+ * Even if both are true, there might be parallel-restricted
+ * expressions in the targetlist or quals.
+ *
+ * Note that if there are more than two rels in this relation, they
+ * could be divided between inner_rel and outer_rel in any arbitrary
+ * way. We assume this doesn't matter, because we should hit all the
+ * same baserels and joinclauses while building up to this joinrel no
+ * matter which we take; therefore, we should make the same decision
+ * here however we get here.
+ */
+ if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
+ is_parallel_safe(root, (Node *) restrictlist) &&
+ is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
+ joinrel->consider_parallel = true;
}
return joinrel;
@@ -741,6 +900,7 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
JoinType jointype)
{
RelOptInfo *joinrel = makeNode(RelOptInfo);
+ RelOptInfoSet *joinrelset = makeNode(RelOptInfoSet);
AppendRelInfo **appinfos;
int nappinfos;
@@ -852,7 +1012,8 @@ build_child_join_rel(PlannerInfo *root, RelOptInfo *outer_rel,
Assert(!find_join_rel(root, joinrel->relids));
/* Add the relation to the PlannerInfo. */
- add_join_rel(root, joinrel);
+ joinrelset->rel_plain = joinrel;
+ add_join_rel(root, joinrelset);
return joinrel;
}
@@ -1749,3 +1910,626 @@ build_child_join_reltarget(PlannerInfo *root,
childrel->reltarget->cost.per_tuple = parentrel->reltarget->cost.per_tuple;
childrel->reltarget->width = parentrel->reltarget->width;
}
+
+/*
+ * Check if the relation can produce grouped paths and return the information
+ * it'll need for it. The passed relation is the non-grouped one which has the
+ * reltarget already constructed.
+ */
+RelAggInfo *
+create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel)
+{
+ List *gvis;
+ List *aggregates = NIL;
+ bool found_other_rel_agg;
+ ListCell *lc;
+ RelAggInfo *result;
+ PathTarget *agg_input;
+ PathTarget *target = NULL;
+ List *grp_exprs_extra = NIL;
+ List *group_clauses_final;
+ int i;
+
+ /*
+ * The function shouldn't have been called if there's no opportunity for
+ * aggregation push-down.
+ */
+ Assert(root->grouped_var_list != NIL);
+
+ result = makeNode(RelAggInfo);
+
+ /*
+ * The current implementation of aggregation push-down cannot handle
+ * PlaceHolderVar (PHV).
+ *
+ * If we knew that the PHV should be evaluated in this target (and of
+ * course, if its expression matched some Aggref argument), we'd just let
+ * init_grouping_targets add that Aggref. On the other hand, if we knew
+ * that the PHV is evaluated below the current rel, we could ignore it
+ * because the referencing Aggref would take care of propagation of the
+ * value to upper joins.
+ *
+ * The problem is that the same PHV can be evaluated in the target of the
+ * current rel or in that of lower rel --- depending on the input paths.
+ * For example, consider rel->relids = {A, B, C} and if ph_eval_at = {B,
+ * C}. Path "A JOIN (B JOIN C)" implies that the PHV is evaluated by the
+ * "(B JOIN C)", while path "(A JOIN B) JOIN C" evaluates the PHV itself.
+ */
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *expr = lfirst(lc);
+
+ if (IsA(expr, PlaceHolderVar))
+ return NULL;
+ }
+
+ if (IS_SIMPLE_REL(rel))
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return NULL;
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the Agg plan
+ * would receive different input at the base rel level.
+ *
+ * XXX For RELOPT_JOINREL, do not return if all the joins that can set any
+ * entry of the grouped target (do we need to postpone this check until
+ * the grouped target is available, and init_grouping_targets take care?)
+ * of this rel to NULL are provably below rel. (It's ok if rel is one of
+ * these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return NULL;
+
+ /*
+ * Use equivalence classes to generate additional grouping expressions for
+ * the current rel. Without these we might not be able to apply
+ * aggregation to the relation result set.
+ *
+ * It's important that create_grouping_expr_grouped_var_infos has
+ * processed the explicit grouping columns by now. If the grouping clause
+ * contains multiple expressions belonging to the same EC, the original
+ * (i.e. not derived) one should be preferred when we build grouping
+ * target for a relation. Otherwise we have a problem when trying to match
+ * target entries to grouping clauses during plan creation, see
+ * get_grouping_expression().
+ */
+ gvis = list_copy(root->grouped_var_list);
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+ int relid = -1;
+
+ /* Only interested in grouping expressions. */
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ while ((relid = bms_next_member(rel->relids, relid)) >= 0)
+ {
+ GroupedVarInfo *gvi_trans;
+
+ gvi_trans = translate_expression_to_rels(root, gvi, relid);
+ if (gvi_trans != NULL)
+ gvis = lappend(gvis, gvi_trans);
+ }
+ }
+
+ /*
+ * Check if some aggregates or grouping expressions can be evaluated in
+ * this relation's target, and collect all vars referenced by these
+ * aggregates / grouping expressions;
+ */
+ found_other_rel_agg = false;
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which includes
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * init_grouping_targets will handle plain Var grouping
+ * expressions because it needs to look them up in
+ * grouped_var_list anyway.
+ */
+ if (IsA(gvi->gvexpr, Var))
+ continue;
+
+ /*
+ * Currently, GroupedVarInfo only handles Vars and Aggrefs.
+ */
+ Assert(IsA(gvi->gvexpr, Aggref));
+
+ /* We only derive grouped expressions using ECs, not aggregates */
+ Assert(!gvi->derived);
+
+ gvi->agg_partial = (Aggref *) copyObject(gvi->gvexpr);
+ mark_partial_aggref(gvi->agg_partial, AGGSPLIT_INITIAL_SERIAL);
+
+ /*
+ * Accept the aggregate.
+ */
+ aggregates = lappend(aggregates, gvi);
+ }
+ else if (IsA(gvi->gvexpr, Aggref))
+ {
+ /*
+ * Remember that there is at least one aggregate expression that
+ * needs something else than this rel.
+ */
+ found_other_rel_agg = true;
+
+ /*
+ * This condition effectively terminates creation of the
+ * RelAggInfo, so there's no reason to check the next
+ * GroupedVarInfo.
+ */
+ break;
+ }
+ }
+
+ /*
+ * Grouping makes little sense w/o aggregate function and w/o grouping
+ * expressions.
+ */
+ if (aggregates == NIL)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Give up if some other aggregate(s) need relations other than the
+ * current one.
+ *
+ * If the aggregate needs the current rel plus anything else, then the
+ * problem is that grouping of the current relation could make some input
+ * variables unavailable for the "higher aggregate", and it'd also
+ * decrease the number of input rows the "higher aggregate" receives.
+ *
+ * If the aggregate does not even need the current rel, then neither the
+ * current rel nor anything else should be grouped because we do not
+ * support join of two grouped relations.
+ */
+ if (found_other_rel_agg)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Create target for grouped paths as well as one for the input paths of
+ * the aggregation paths.
+ */
+ target = create_empty_pathtarget();
+ agg_input = create_empty_pathtarget();
+
+ /*
+ * Cannot suitable targets for the aggregation push-down be derived?
+ */
+ if (!init_grouping_targets(root, rel, target, agg_input, gvis,
+ &grp_exprs_extra))
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ list_free(gvis);
+
+ /*
+ * Aggregation push-down makes no sense w/o grouping expressions.
+ */
+ if ((list_length(target->exprs) + list_length(grp_exprs_extra)) == 0)
+ return NULL;
+
+ group_clauses_final = root->parse->groupClause;
+
+ /*
+ * If the aggregation target should have extra grouping expressions (in
+ * order to emit input vars for join conditions), add them now. This step
+ * includes assignment of tleSortGroupRef's which we can generate now.
+ */
+ if (list_length(grp_exprs_extra) > 0)
+ {
+ Index sortgroupref;
+
+ /*
+ * We'll have to add some clauses, but query group clause must be
+ * preserved.
+ */
+ group_clauses_final = list_copy(group_clauses_final);
+
+ /*
+ * Always start at root->max_sortgroupref. The extra grouping
+ * expressions aren't used during the final aggregation, so the
+ * sortgroupref values don't need to be unique across the query. Thus
+ * we don't have to increase root->max_sortgroupref, which makes
+ * recognition of the extra grouping expressions pretty easy.
+ */
+ sortgroupref = root->max_sortgroupref;
+
+ /*
+ * Generate the SortGroupClause's and add the expressions to the
+ * target.
+ */
+ foreach(lc, grp_exprs_extra)
+ {
+ Var *var = lfirst_node(Var, lc);
+ SortGroupClause *cl = makeNode(SortGroupClause);
+
+ /*
+ * Initialize the SortGroupClause.
+ *
+ * As the final aggregation will not use this grouping expression,
+ * we don't care whether sortop is < or >. The value of
+ * nulls_first should not matter for the same reason.
+ */
+ cl->tleSortGroupRef = ++sortgroupref;
+ get_sort_group_operators(var->vartype,
+ false, true, false,
+ &cl->sortop, &cl->eqop, NULL,
+ &cl->hashable);
+ group_clauses_final = lappend(group_clauses_final, cl);
+ add_column_to_pathtarget(target, (Expr *) var,
+ cl->tleSortGroupRef);
+
+ /*
+ * The aggregation input target must emit this var too.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) var,
+ cl->tleSortGroupRef);
+ }
+ }
+
+ /*
+ * Add aggregates to the grouping target.
+ */
+ add_aggregates_to_target(root, target, aggregates);
+
+ /*
+ * Build a list of grouping expressions and a list of the corresponding
+ * SortGroupClauses.
+ */
+ i = 0;
+ foreach(lc, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(lc);
+
+ if (IsA(texpr, Aggref))
+ {
+ /*
+ * Once we see Aggref, no grouping expressions should follow.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides being an aggregate, the target expression should have no
+ * other reason then being a column of a relation functionally
+ * dependent on the GROUP BY clause. So it's not actually a grouping
+ * column.
+ */
+ if (sortgroupref == 0)
+ continue;
+
+ /*
+ * group_clause_final contains the "local" clauses, so this search
+ * should succeed.
+ */
+ cl = get_sortgroupref_clause(sortgroupref, group_clauses_final);
+
+ result->group_clauses = list_append_unique(result->group_clauses,
+ cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ result->group_exprs = list_append_unique(result->group_exprs,
+ texpr);
+ }
+
+ /*
+ * Since neither target nor agg_input is supposed to be identical to the
+ * source reltarget, compute the width and cost again.
+ *
+ * target does not yet contain aggregates, but these will be accounted by
+ * AggPath.
+ */
+ set_pathtarget_cost_width(root, target);
+ set_pathtarget_cost_width(root, agg_input);
+
+ result->target = target;
+ result->input = agg_input;
+
+ /* Finally collect the aggregates. */
+ while (lc != NULL)
+ {
+ Aggref *aggref = lfirst_node(Aggref, lc);
+
+ /*
+ * Partial aggregation is what the grouped paths should do.
+ */
+ result->agg_exprs = lappend(result->agg_exprs, aggref);
+ lc = lnext(lc);
+ }
+
+ /*
+ * The "input_rows" field should be set by caller.
+ */
+ return result;
+}
+
+/*
+ * Initialize target for grouped paths (target) as well as a target for paths
+ * that generate input for aggregation (agg_input).
+ *
+ * group_exprs_extra_p receives a list of Var nodes for which we need to
+ * construct SortGroupClause. Those vars will then be used as additional
+ * grouping expressions, for the sake of join clauses.
+ *
+ * gvis a list of GroupedVarInfo's possibly useful for rel.
+ *
+ * Return true iff the targets could be initialized.
+ */
+static bool
+init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p)
+{
+ ListCell *lc1,
+ *lc2;
+ List *unresolved = NIL;
+ List *unresolved_sortgrouprefs = NIL;
+
+ foreach(lc1, rel->reltarget->exprs)
+ {
+ Var *tvar;
+ bool is_grouping;
+ Index sortgroupref = 0;
+ bool derived = false;
+ bool needed_by_aggregate;
+
+ /*
+ * Given that PlaceHolderVar currently prevents us from doing
+ * aggregation push-down, the source target cannot contain anything
+ * more complex than a Var.
+ */
+ tvar = lfirst_node(Var, lc1);
+
+ is_grouping = is_grouping_expression(gvis, (Expr *) tvar,
+ &sortgroupref, &derived);
+
+ /*
+ * Derived grouping expressions should not be referenced by the query
+ * targetlist, so let them fall into vars_unresolved. It'll be checked
+ * later if the current targetlist needs them. For example, we should
+ * not automatically use Var as a grouping expression if the only
+ * reason for it to be in the plain relation target is that it's
+ * referenced by aggregate argument, and it happens to be in the same
+ * EC as any grouping expression.
+ */
+ if (is_grouping && !derived)
+ {
+ Assert(sortgroupref > 0);
+
+ /*
+ * It's o.k. to use the target expression for grouping.
+ */
+ add_column_to_pathtarget(target, (Expr *) tvar, sortgroupref);
+
+ /*
+ * As for agg_input, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) tvar, sortgroupref);
+
+ /* Process the next expression. */
+ continue;
+ }
+
+ /*
+ * Is this Var needed in the query targetlist for anything else than
+ * aggregate input?
+ */
+ needed_by_aggregate = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc2);
+ ListCell *lc3;
+ List *vars;
+
+ if (!IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (!bms_is_member(tvar->varno, gvi->gv_eval_at))
+ continue;
+
+ /*
+ * XXX Consider some sort of caching.
+ */
+ vars = pull_var_clause((Node *) gvi->gvexpr, PVC_RECURSE_AGGREGATES);
+ foreach(lc3, vars)
+ {
+ Var *var = lfirst_node(Var, lc3);
+
+ if (equal(var, tvar))
+ {
+ needed_by_aggregate = true;
+ break;
+ }
+ }
+ list_free(vars);
+ if (needed_by_aggregate)
+ break;
+ }
+
+ if (needed_by_aggregate)
+ {
+ bool found = false;
+
+ foreach(lc2, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc2);
+
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ if (equal(te->expr, tvar))
+ {
+ found = true;
+ break;
+ }
+ }
+
+ /*
+ * If it's only Aggref input, add it to the aggregation input
+ * target and that's it.
+ */
+ if (!found)
+ {
+ add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
+ continue;
+ }
+ }
+
+ /*
+ * Further investigation involves dependency check, for which we need
+ * to have all the (plain-var) grouping expressions gathered.
+ */
+ unresolved = lappend(unresolved, tvar);
+ unresolved_sortgrouprefs = lappend_int(unresolved_sortgrouprefs,
+ sortgroupref);
+ }
+
+ /*
+ * Check for other possible reasons for the var to be in the plain target.
+ */
+ forboth(lc1, unresolved, lc2, unresolved_sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, lc1);
+ Index sortgroupref = lfirst_int(lc2);
+ RangeTblEntry *rte;
+ List *deps = NIL;
+ Relids relids_subtract;
+ int ndx;
+ RelOptInfo *baserel;
+
+ rte = root->simple_rte_array[var->varno];
+
+ /*
+ * Check if the Var can be in the grouping key even though it's not
+ * mentioned by the GROUP BY clause (and could not be derived using
+ * ECs).
+ */
+ if (sortgroupref == 0 &&
+ check_functional_grouping(rte->relid, var->varno,
+ var->varlevelsup,
+ target->exprs, &deps))
+ {
+ /*
+ * The var shouldn't be actually used as a grouping key (instead,
+ * the one this depends on will be), so sortgroupref should not be
+ * important.
+ */
+ add_new_column_to_pathtarget(target, (Expr *) var);
+ add_new_column_to_pathtarget(agg_input, (Expr *) var);
+
+ /*
+ * The var may or may not be present in generic grouping
+ * expression(s) in addition, but this is handled elsewhere.
+ */
+ continue;
+ }
+
+ /*
+ * Isn't the expression needed by joins above the current rel?
+ *
+ * The relids we're not interested in do include 0, which is the
+ * top-level targetlist. The only reason for relids to contain 0
+ * should be that arg_var is referenced either by aggregate or by
+ * grouping expression, but right now we're interested in the *other*
+ * reasons. (As soon aggregation is pushed down, the aggregates in the
+ * query targetlist no longer need direct reference to arg_var
+ * anyway.)
+ */
+ relids_subtract = bms_copy(rel->relids);
+ bms_add_member(relids_subtract, 0);
+
+ baserel = find_base_rel(root, var->varno);
+ ndx = var->varattno - baserel->min_attr;
+ if (bms_nonempty_difference(baserel->attr_needed[ndx],
+ relids_subtract))
+ {
+ /*
+ * The variable is needed by a join involving this relation. That
+ * case includes variable that is referenced by a generic grouping
+ * expression.
+ *
+ * The only way to bring this var to the aggregation output is to
+ * add it to the grouping expressions too.
+ */
+ if (sortgroupref > 0)
+ {
+ /*
+ * The var could be recognized as a potentially useful
+ * grouping expression at the top of the loop, so we can add
+ * it to the grouping target, as well as to the agg_input.
+ */
+ add_column_to_pathtarget(target, (Expr *) var, sortgroupref);
+ add_column_to_pathtarget(agg_input, (Expr *) var, sortgroupref);
+ }
+ else
+ {
+ /*
+ * Since root->parse->groupClause is not supposed to contain
+ * this expression, we need to construct special
+ * SortGroupClause. Its tleSortGroupRef needs to be unique
+ * within target_agg, so postpone creation of the
+ * SortGroupRefs until we're done with the iteration of
+ * rel->reltarget->exprs.
+ */
+ *group_exprs_extra_p = lappend(*group_exprs_extra_p, var);
+ }
+ }
+ else
+ {
+ /*
+ * As long as the query is semantically correct, arriving here
+ * means that the var is referenced by a generic grouping
+ * expression but not referenced by any join.
+ *
+ * create_rel_agg_info() should add this variable to "agg_input"
+ * target and also add the whole generic expression to "target",
+ * but that's subject to future enhancement of the aggregate
+ * push-down feature.
+ */
+ return false;
+ }
+ }
+
+ return true;
+}
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
index 14d1c67a94..bf820fe66c 100644
--- a/src/backend/optimizer/util/tlist.c
+++ b/src/backend/optimizer/util/tlist.c
@@ -427,7 +427,6 @@ get_sortgrouplist_exprs(List *sgClauses, List *targetList)
return result;
}
-
/*****************************************************************************
* Functions to extract data from a list of SortGroupClauses
*
@@ -802,6 +801,62 @@ apply_pathtarget_labeling_to_tlist(List *tlist, PathTarget *target)
}
/*
+ * For each aggregate grouping expression or aggregate to the grouped target.
+ *
+ * Caller passes the expressions in the form of GroupedVarInfos so that we
+ * don't have to look for gvid.
+ */
+void
+add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions)
+{
+ ListCell *lc;
+
+ /* Create the vars and add them to the target. */
+ foreach(lc, expressions)
+ {
+ GroupedVarInfo *gvi;
+
+ gvi = lfirst_node(GroupedVarInfo, lc);
+ add_column_to_pathtarget(target, (Expr *) gvi->agg_partial,
+ gvi->sortgroupref);
+ }
+}
+
+/*
+ * Find out if expr can be used as grouping expression in reltarget.
+ *
+ * sortgroupref and is_derived reflect the ->sortgroupref and ->derived fields
+ * of the corresponding GroupedVarInfo.
+ */
+bool
+is_grouping_expression(List *gvis, Expr *expr, Index *sortgroupref,
+ bool *is_derived)
+{
+ ListCell *lc;
+
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (equal(gvi->gvexpr, expr))
+ {
+ Assert(gvi->sortgroupref > 0);
+
+ *sortgroupref = gvi->sortgroupref;
+ *is_derived = gvi->derived;
+ return true;
+ }
+ }
+
+ /* The expression cannot be used as grouping key. */
+ return false;
+}
+
+/*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
*
diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c
index 67e268cc7a..0cc56f8b66 100644
--- a/src/backend/utils/adt/selfuncs.c
+++ b/src/backend/utils/adt/selfuncs.c
@@ -4884,8 +4884,12 @@ examine_variable(PlannerInfo *root, Node *node, int varRelid,
case BMS_MULTIPLE:
if (varRelid == 0)
{
+ RelOptInfoSet *relset;
+
/* treat it as a variable of a join relation */
- vardata->rel = find_join_rel(root, varnos);
+ relset = find_join_rel(root, varnos);
+ if (relset)
+ vardata->rel = relset->rel_plain;
node = basenode; /* strip any relabeling */
}
else if (bms_is_member(varRelid, varnos))
@@ -5743,7 +5747,13 @@ find_join_input_rel(PlannerInfo *root, Relids relids)
rel = find_base_rel(root, bms_singleton_member(relids));
break;
case BMS_MULTIPLE:
- rel = find_join_rel(root, relids);
+ {
+ RelOptInfoSet *relset;
+
+ relset = find_join_rel(root, relids);
+ if (relset)
+ rel = relset->rel_plain;
+ }
break;
}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 8681ada33a..52b1204345 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -1007,6 +1007,15 @@ static struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
{
+ {"enable_agg_pushdown", PGC_USERSET, QUERY_TUNING_METHOD,
+ gettext_noop("Enables aggregation push-down."),
+ NULL
+ },
+ &enable_agg_pushdown,
+ false,
+ NULL, NULL, NULL
+ },
+ {
{"enable_parallel_append", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables the planner's use of parallel append plans."),
NULL
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index e215ad4978..96fa16c2d3 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -223,6 +223,8 @@ typedef enum NodeTag
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_RelAggInfo,
+ T_RelOptInfoSet,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
@@ -266,6 +268,7 @@ typedef enum NodeTag
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index d3c477a542..60b059ad05 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -284,6 +284,8 @@ struct PlannerInfo
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
@@ -296,7 +298,7 @@ struct PlannerInfo
List *part_schemes; /* Canonicalised partition schemes used in the
* query. */
- List *initial_rels; /* RelOptInfos we are now trying to join */
+ List *initial_rels; /* RelOptInfoSets we are now trying to join */
/* Use fetch_upper_rel() to get any particular upper rel */
List *upper_rels[UPPERREL_FINAL + 1]; /* upper-rel RelOptInfos */
@@ -310,6 +312,12 @@ struct PlannerInfo
*/
List *processed_tlist;
+ /*
+ * The maximum ressortgroupref among target entries in processed_list.
+ * Useful when adding extra grouping expressions for partial aggregation.
+ */
+ int max_sortgroupref;
+
/* Fields filled during create_plan() for use in setrefs.c */
AttrNumber *grouping_map; /* for GroupingFunc fixup */
List *minmax_aggs; /* List of MinMaxAggInfos */
@@ -734,6 +742,82 @@ typedef struct RelOptInfo
(rel)->part_rels && (rel)->partexprs && (rel)->nullable_partexprs)
/*
+ * RelAggInfo
+ *
+ * RelOptInfo needs information contained here if its paths should be
+ * aggregated.
+ *
+ * "target" will be used as pathtarget for aggregation if "explicit
+ * aggregation" is applied to base relation or join. The same target will also
+ * --- if the relation is a join --- be used to joinin grouped path to a
+ * non-grouped one.
+ *
+ * These targets contain plain-Var grouping expressions and Aggrefs which.
+ * Once Aggref is evaluated, its value is passed to the upper paths w/o being
+ * evaluated again.
+ *
+ * Note: There's a convention that Aggref expressions are supposed to follow
+ * the other expressions of the target. Iterations of ->exprs may rely on this
+ * arrangement.
+ *
+ * "input" contains Vars used either as grouping expressions or aggregate
+ * arguments. Paths providing the aggregation plan with input data should use
+ * this target.
+ *
+ * "input_rows" is the estimated number of input rows for AggPath. It's
+ * actually just a workspace for users of the structure, i.e. not initialized
+ * when instance of the structure is created.
+ *
+ * "group_clauses" and "group_exprs" are lists of SortGroupClause and the
+ * corresponding grouping expressions respectively.
+ *
+ * "agg_exprs" is a list of Aggref nodes for the aggregation of the relation's
+ * paths.
+ */
+typedef struct RelAggInfo
+{
+ NodeTag type;
+
+ struct PathTarget *target; /* Target for grouped paths.. */
+
+ struct PathTarget *input; /* pathtarget of paths that generate input for
+ * aggregation paths. */
+ double input_rows;
+
+ List *group_clauses;
+ List *group_exprs;
+
+ List *agg_exprs; /* Aggref expressions. */
+} RelAggInfo;
+
+/*
+ * RelOptInfoSet
+ *
+ * Structure to use where RelOptInfo for other UpperRelationKind than
+ * UPPERREL_FINAL may be needed. Currently we only need UPPERREL_FINAL
+ * and UPPERREL_PARTIAL_GROUP_AGG.
+ *
+ * rel_plain should always be initialized. Code paths that do not
+ * distinguish between plain and grouped relation use rel_plain,
+ * e.g. has_legal_joinclause().
+ *
+ * agg_info is initialized iff rel_grouped is.
+ *
+ * XXX Is RelOptInfoPair more appropriate name?
+ */
+typedef struct RelOptInfoSet
+{
+ NodeTag type;
+
+ RelOptInfo *rel_plain; /* UPPERREL_FINAL */
+ RelOptInfo *rel_grouped; /* UPPERREL_PARTIAL_GROUP_AGG */
+
+ RelAggInfo *agg_info; /* Information needed to create rel_grouped
+ * and its paths. It seems just convenient to
+ * store it here. */
+} RelOptInfoSet;
+
+/*
* IndexOptInfo
* Per-index information for planning/optimization
*
@@ -2214,6 +2298,26 @@ typedef struct PlaceHolderInfo
} PlaceHolderInfo;
/*
+ * GroupedVarInfo exists for each expression that can be used as an aggregate
+ * or grouping expression evaluated below a join.
+ */
+typedef struct GroupedVarInfo
+{
+ NodeTag type;
+
+ Expr *gvexpr; /* the represented expression. */
+ Aggref *agg_partial; /* if gvexpr is aggregate, agg_partial is the
+ * corresponding partial aggregate */
+ Index sortgroupref; /* If gvexpr is a grouping expression, this is
+ * the tleSortGroupRef of the corresponding
+ * SortGroupClause. */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+ bool derived; /* derived from another GroupedVarInfo using
+ * equeivalence classes? */
+} GroupedVarInfo;
+
+/*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
* path, the list is stored into root->minmax_aggs for use during setrefs.c.
@@ -2320,6 +2424,11 @@ typedef struct SemiAntiJoinFactors
* sjinfo is extra info about special joins for selectivity estimation
* semifactors is as shown above (only valid for SEMI/ANTI/inner_unique joins)
* param_source_rels are OK targets for parameterization of result paths
+ * agg_info passes information necessary for joins to produce partially
+ * grouped data.
+ * rel_agg_input describes the AggPath input relation if the join output
+ * should be aggregated. If NULL is passed, do not aggregate the join
+ * output.
*/
typedef struct JoinPathExtraData
{
@@ -2329,6 +2438,8 @@ typedef struct JoinPathExtraData
SpecialJoinInfo *sjinfo;
SemiAntiJoinFactors semifactors;
Relids param_source_rels;
+ RelAggInfo *agg_info;
+ RelOptInfo *rel_agg_input;
} JoinPathExtraData;
/*
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
index 23073c0402..25ab90e7f2 100644
--- a/src/include/optimizer/clauses.h
+++ b/src/include/optimizer/clauses.h
@@ -56,4 +56,6 @@ extern void CommuteRowCompareExpr(RowCompareExpr *clause);
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
+extern GroupedVarInfo *translate_expression_to_rels(PlannerInfo *root,
+ GroupedVarInfo *gvi, Index relid);
#endif /* CLAUSES_H */
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
index ac6de0f6be..b9757c31fd 100644
--- a/src/include/optimizer/cost.h
+++ b/src/include/optimizer/cost.h
@@ -64,6 +64,7 @@ extern PGDLLIMPORT bool enable_partitionwise_aggregate;
extern PGDLLIMPORT bool enable_parallel_append;
extern PGDLLIMPORT bool enable_parallel_hash;
extern PGDLLIMPORT bool enable_partition_pruning;
+extern PGDLLIMPORT bool enable_agg_pushdown;
extern PGDLLIMPORT int constraint_exclusion;
extern double index_pages_fetched(double tuples_fetched, BlockNumber pages,
@@ -171,7 +172,8 @@ extern void compute_semi_anti_join_factors(PlannerInfo *root,
SpecialJoinInfo *sjinfo,
List *restrictlist,
SemiAntiJoinFactors *semifactors);
-extern void set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel);
+extern void set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel,
+ RelAggInfo *agg_info);
extern double get_parameterized_baserel_size(PlannerInfo *root,
RelOptInfo *rel,
List *param_clauses);
diff --git a/src/include/optimizer/geqo.h b/src/include/optimizer/geqo.h
index 5b3327665e..8a4054b59a 100644
--- a/src/include/optimizer/geqo.h
+++ b/src/include/optimizer/geqo.h
@@ -78,7 +78,7 @@ typedef struct
/* routines in geqo_main.c */
-extern RelOptInfo *geqo(PlannerInfo *root,
+extern RelOptInfoSet *geqo(PlannerInfo *root,
int number_of_rels, List *initial_rels);
/* routines in geqo_eval.c */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index d0c8f99d0a..88b460ee58 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -35,7 +35,8 @@ extern bool add_partial_path_precheck(RelOptInfo *parent_rel,
Cost total_cost, List *pathkeys);
extern Path *create_seqscan_path(PlannerInfo *root, RelOptInfo *rel,
- Relids required_outer, int parallel_workers);
+ Relids required_outer, int parallel_workers,
+ RelOptInfo *rel_grouped, RelAggInfo *agg_info);
extern Path *create_samplescan_path(PlannerInfo *root, RelOptInfo *rel,
Relids required_outer);
extern IndexPath *create_index_path(PlannerInfo *root,
@@ -49,7 +50,9 @@ extern IndexPath *create_index_path(PlannerInfo *root,
bool indexonly,
Relids required_outer,
double loop_count,
- bool partial_path);
+ bool partial_path,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
extern BitmapHeapPath *create_bitmap_heap_path(PlannerInfo *root,
RelOptInfo *rel,
Path *bitmapqual,
@@ -127,6 +130,7 @@ extern Relids calc_non_nestloop_required_outer(Path *outer_path, Path *inner_pat
extern NestPath *create_nestloop_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -138,6 +142,7 @@ extern NestPath *create_nestloop_path(PlannerInfo *root,
extern MergePath *create_mergejoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -152,6 +157,7 @@ extern MergePath *create_mergejoin_path(PlannerInfo *root,
extern HashPath *create_hashjoin_path(PlannerInfo *root,
RelOptInfo *joinrel,
+ PathTarget *target,
JoinType jointype,
JoinCostWorkspace *workspace,
JoinPathExtraData *extra,
@@ -200,6 +206,12 @@ extern AggPath *create_agg_path(PlannerInfo *root,
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+extern AggPath *create_agg_sorted_path(PlannerInfo *root,
+ Path *subpath,
+ RelAggInfo *agg_info);
+extern AggPath *create_agg_hashed_path(PlannerInfo *root,
+ Path *subpath,
+ RelAggInfo *agg_info);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
@@ -267,14 +279,16 @@ extern void setup_simple_rel_arrays(PlannerInfo *root);
extern void setup_append_rel_array(PlannerInfo *root);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
+extern void build_simple_grouped_rel(PlannerInfo *root, RelOptInfoSet *rel);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
-extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
+extern RelOptInfoSet *find_join_rel(PlannerInfo *root, Relids relids);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr);
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info);
extern Relids min_join_parameterization(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
@@ -300,5 +314,5 @@ extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo, JoinType jointype);
-
+extern RelAggInfo *create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index 1b02b3b889..ad9439439d 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -21,6 +21,7 @@
* allpaths.c
*/
extern PGDLLIMPORT bool enable_geqo;
+extern PGDLLIMPORT bool enable_agg_pushdown;
extern PGDLLIMPORT int geqo_threshold;
extern PGDLLIMPORT int min_parallel_table_scan_size;
extern PGDLLIMPORT int min_parallel_index_scan_size;
@@ -29,7 +30,9 @@ extern PGDLLIMPORT int min_parallel_index_scan_size;
typedef void (*set_rel_pathlist_hook_type) (PlannerInfo *root,
RelOptInfo *rel,
Index rti,
- RangeTblEntry *rte);
+ RangeTblEntry *rte,
+ RelOptInfo *rel_grouped,
+ RelAggInfo *agg_info);
extern PGDLLIMPORT set_rel_pathlist_hook_type set_rel_pathlist_hook;
/* Hook for plugins to get control in add_paths_to_joinrel() */
@@ -42,19 +45,24 @@ typedef void (*set_join_pathlist_hook_type) (PlannerInfo *root,
extern PGDLLIMPORT set_join_pathlist_hook_type set_join_pathlist_hook;
/* Hook for plugins to replace standard_join_search() */
-typedef RelOptInfo *(*join_search_hook_type) (PlannerInfo *root,
- int levels_needed,
- List *initial_rels);
+typedef RelOptInfoSet *(*join_search_hook_type) (PlannerInfo *root,
+ int levels_needed,
+ List *initial_rels);
extern PGDLLIMPORT join_search_hook_type join_search_hook;
-extern RelOptInfo *make_one_rel(PlannerInfo *root, List *joinlist);
+extern RelOptInfoSet *make_one_rel(PlannerInfo *root, List *joinlist);
extern void set_dummy_rel_pathlist(RelOptInfo *rel);
-extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
+extern RelOptInfoSet *standard_join_search(PlannerInfo *root,
+ int levels_needed,
List *initial_rels);
extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
+
+extern bool add_grouped_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, AggStrategy aggstrategy,
+ RelAggInfo *agg_info);
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages, int max_workers);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
@@ -70,7 +78,8 @@ extern void debug_print_rel(PlannerInfo *root, RelOptInfo *rel);
* indxpath.c
* routines to generate index paths
*/
-extern void create_index_paths(PlannerInfo *root, RelOptInfo *rel);
+extern void create_index_paths(PlannerInfo *root, RelOptInfo *rel,
+ RelOptInfo *rel_grouped, RelAggInfo *agg_info);
extern bool relation_has_unique_index_for(PlannerInfo *root, RelOptInfo *rel,
List *restrictlist,
List *exprlist, List *oprlist);
@@ -101,7 +110,9 @@ extern void create_tidscan_paths(PlannerInfo *root, RelOptInfo *rel);
extern void add_paths_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinType jointype, SpecialJoinInfo *sjinfo,
- List *restrictlist);
+ List *restrictlist,
+ RelAggInfo *agg_info,
+ RelOptInfo *rel_agg_input);
/*
* joinrels.c
@@ -109,7 +120,7 @@ extern void add_paths_to_joinrel(PlannerInfo *root, RelOptInfo *joinrel,
*/
extern void join_search_one_level(PlannerInfo *root, int level);
extern RelOptInfo *make_join_rel(PlannerInfo *root,
- RelOptInfo *rel1, RelOptInfo *rel2);
+ RelOptInfoSet *relset1, RelOptInfoSet *relset2);
extern bool have_join_order_restriction(PlannerInfo *root,
RelOptInfo *rel1, RelOptInfo *rel2);
extern bool have_dangerous_phv(PlannerInfo *root,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index 3bbdb5e2f7..75e48185a3 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -27,7 +27,7 @@ typedef void (*query_pathkeys_callback) (PlannerInfo *root, void *extra);
/*
* prototypes for plan/planmain.c
*/
-extern RelOptInfo *query_planner(PlannerInfo *root, List *tlist,
+extern RelOptInfoSet *query_planner(PlannerInfo *root, List *tlist,
query_pathkeys_callback qp_callback, void *qp_extra);
/*
@@ -68,6 +68,7 @@ extern void add_base_rels_to_query(PlannerInfo *root, Node *jtnode);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
Relids where_needed, bool create_new_ph);
+extern void setup_aggregate_pushdown(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
index 58db79203b..fa927cd6f4 100644
--- a/src/include/optimizer/tlist.h
+++ b/src/include/optimizer/tlist.h
@@ -49,6 +49,13 @@ extern void split_pathtarget_at_srfs(PlannerInfo *root,
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+/* TODO Find the best location (position and in some cases even file) for the
+ * following ones. */
+extern void add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions);
+extern bool is_grouping_expression(List *gvis, Expr *expr,
+ Index *sortgroupref, bool *is_derived);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 4051a4ad4e..22058b1f91 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -104,6 +104,9 @@ test: rules psql_crosstab amutils
test: select_parallel
test: write_parallel
+# this one runs parallel workers too
+test: agg_pushdown
+
# no relation related tests can be put in this group
test: publication subscription
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index ac1ea622d6..bc1ed68a9e 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -136,6 +136,7 @@ test: rules
test: psql_crosstab
test: select_parallel
test: write_parallel
+test: agg_pushdown
test: publication
test: subscription
test: amutils
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
new file mode 100644
index 0000000000..b3a97f86d6
--- /dev/null
+++ b/src/test/regress/expected/agg_pushdown.out
@@ -0,0 +1,217 @@
+BEGIN;
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+CREATE INDEX ON agg_pushdown_child1(parent);
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+COMMIT;
+ANALYZE;
+SET enable_agg_pushdown TO on;
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+--------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(10 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(11 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (p.i = c1.parent)
+ -> Sort
+ Sort Key: p.i
+ -> Seq Scan on agg_pushdown_parent p
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(12 rows)
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(8 rows)
+
+SET enable_seqscan TO on;
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.x = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(11 rows)
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: ((k = c1.j) AND (parent = c1.parent))
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(13 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+----------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Hash
+ -> Seq Scan on agg_pushdown_child2 c2
+(15 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: ((c1.j = c2.k) AND (c1.parent = c2.parent))
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(13 rows)
+
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
new file mode 100644
index 0000000000..6cf2ed9c20
--- /dev/null
+++ b/src/test/regress/sql/agg_pushdown.sql
@@ -0,0 +1,117 @@
+BEGIN;
+
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+
+CREATE INDEX ON agg_pushdown_child1(parent);
+
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+COMMIT;
+ANALYZE;
+
+SET enable_agg_pushdown TO on;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_seqscan TO on;
+
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
Re: [HACKERS] WIP: Aggregation push-down
Antonin Houska <ah@cybertec.at> writes:
Michael Paquier <michael@paquier.xyz> wrote:
Latest patch set fails to apply, so moved to next CF, waiting on
author.
Rebased.
This is in need of rebasing again :-(. I went ahead and pushed the 001
part, since that seemed fairly uncontroversial. (Note that I changed
estimate_hashagg_tablesize's result type to double on the way; you
probably want to make corresponding adjustments in your patch.)
I did not spend a whole lot of time looking at the patch today, but
I'm still pretty distressed at the data structures you've chosen.
I remain of the opinion that a grouped relation and a base relation
are, er, unrelated, even if they happen to share the same relid set.
So I do not see the value of the RelOptInfoSet struct you propose here,
and I definitely don't think there's any value in having, eg,
create_seqscan_path or create_index_path dealing with this stuff.
I also don't like changing create_nestloop_path et al to take a PathTarget
rather than using the RelOptInfo's pathtarget; IMO, it's flat out wrong
for a path to generate a tlist different from what its parent RelOptInfo
says that the relation produces.
I think basically the way this ought to work is that we generate baserel
paths pretty much the same as today, and then we run through the baserels
and see which ones are potentially worth doing partial aggregation on,
and for each one that is, we create a separate "upper relation" RelOptInfo
describing that. The paths for this RelOptInfo would be
partial-aggregation paths using paths from the corresponding baserel as
input. Then we'd run a join search that only considers joining grouped
rels with plain rels (I concur that joining two grouped rels is not worth
coping with, at least for now).
regards, tom lane
Re: [HACKERS] WIP: Aggregation push-down
Tom Lane <tgl@sss.pgh.pa.us> wrote:
Antonin Houska <ah@cybertec.at> writes:
Michael Paquier <michael@paquier.xyz> wrote:
Latest patch set fails to apply, so moved to next CF, waiting on
author.Rebased.
This is in need of rebasing again :-(. I went ahead and pushed the 001
part, since that seemed fairly uncontroversial.
ok, thanks.
I did not spend a whole lot of time looking at the patch today, but
I'm still pretty distressed at the data structures you've chosen.
I remain of the opinion that a grouped relation and a base relation
are, er, unrelated, even if they happen to share the same relid set.
So I do not see the value of the RelOptInfoSet struct you propose here,
ok. As you suggested upthread, I try now to reuse the join_rel_list /
join_rel_hash structures, see v11-001-Introduce_RelInfoList.patch.
and I definitely don't think there's any value in having, eg,
create_seqscan_path or create_index_path dealing with this stuff.
Originally I tried to aggregate any path that ever gets passed to agg_path(),
but that's probably not worth the code complexity. Now the partial aggregation
is only applied to paths that survived agg_path() on the plain relation.
I also don't like changing create_nestloop_path et al to take a PathTarget
rather than using the RelOptInfo's pathtarget; IMO, it's flat out wrong
for a path to generate a tlist different from what its parent RelOptInfo
says that the relation produces.
Likewise, the current patch version is less invasive, so create_nestloop_path
et al are not touched at all.
I think basically the way this ought to work is that we generate baserel
paths pretty much the same as today, and then we run through the baserels
and see which ones are potentially worth doing partial aggregation on,
and for each one that is, we create a separate "upper relation" RelOptInfo
describing that. The paths for this RelOptInfo would be
partial-aggregation paths using paths from the corresponding baserel as
input. Then we'd run a join search that only considers joining grouped
rels with plain rels (I concur that joining two grouped rels is not worth
coping with, at least for now).
make_join_rel() is the core of my implementation: besides joining two plain
relations it tries to join plain relation to grouped one, and also to
aggregate the join of the two plain relations. I consider this approach less
invasive and more efficient than running the whole standard_join_search again
for the grouped rels.
The problem of numeric-like data types (i.e. types for wich equality of two
values of the grouping key does not justify putting them into the same group
because information like scale would be discarded this way) remains open. My
last idea was to add a boolean flag to operator class which tells that
equality implies "bitwise equality", and to disallow aggregate push-down if
SortGroupClause.eqop is in an opclass which has this field FALSE. I'd like to
hear your opinion before I do any coding.
--
Antonin Houska
https://www.cybertec-postgresql.com
Attachments:
v11-001-Introduce_RelInfoList.patchtext/x-diffDownload
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 6b96e7de0a..4484fb4fbc 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -4906,7 +4906,8 @@ foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
*/
Assert(fpinfo->relation_index == 0); /* shouldn't be set yet */
fpinfo->relation_index =
- list_length(root->parse->rtable) + list_length(root->join_rel_list);
+ list_length(root->parse->rtable) +
+ list_length(root->join_rel_list->items);
return true;
}
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 65302fe65b..8b333f069a 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2272,6 +2272,14 @@ _outRelOptInfo(StringInfo str, const RelOptInfo *node)
}
static void
+_outRelInfoList(StringInfo str, const RelInfoList *node)
+{
+ WRITE_NODE_TYPE("RELOPTINFOLIST");
+
+ WRITE_NODE_FIELD(items);
+}
+
+static void
_outIndexOptInfo(StringInfo str, const IndexOptInfo *node)
{
WRITE_NODE_TYPE("INDEXOPTINFO");
@@ -4031,6 +4039,9 @@ outNode(StringInfo str, const void *obj)
case T_RelOptInfo:
_outRelOptInfo(str, obj);
break;
+ case T_RelInfoList:
+ _outRelInfoList(str, obj);
+ break;
case T_IndexOptInfo:
_outIndexOptInfo(str, obj);
break;
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index e07bab831e..82b4f5c56a 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -92,11 +92,11 @@ geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
*
* join_rel_level[] shouldn't be in use, so just Assert it isn't.
*/
- savelength = list_length(root->join_rel_list);
- savehash = root->join_rel_hash;
+ savelength = list_length(root->join_rel_list->items);
+ savehash = root->join_rel_list->hash;
Assert(root->join_rel_level == NULL);
- root->join_rel_hash = NULL;
+ root->join_rel_list->hash = NULL;
/* construct the best path for the given combination of relations */
joinrel = gimme_tree(root, tour, num_gene);
@@ -121,9 +121,9 @@ geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
* Restore join_rel_list to its former state, and put back original
* hashtable if any.
*/
- root->join_rel_list = list_truncate(root->join_rel_list,
- savelength);
- root->join_rel_hash = savehash;
+ root->join_rel_list->items = list_truncate(root->join_rel_list->items,
+ savelength);
+ root->join_rel_list->hash = savehash;
/* release all the memory acquired within gimme_tree */
MemoryContextSwitchTo(oldcxt);
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 3cedd01c98..52d9943192 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -67,8 +67,7 @@ query_planner(PlannerInfo *root, List *tlist,
* NOTE: append_rel_list was set up by subquery_planner, so do not touch
* here.
*/
- root->join_rel_list = NIL;
- root->join_rel_hash = NULL;
+ root->join_rel_list = makeNode(RelInfoList);
root->join_rel_level = NULL;
root->join_cur_level = 0;
root->canon_pathkeys = NIL;
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 4130514952..bcf2ffefb4 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -31,11 +31,11 @@
#include "utils/hsearch.h"
-typedef struct JoinHashEntry
+typedef struct RelInfoEntry
{
- Relids join_relids; /* hash key --- MUST BE FIRST */
- RelOptInfo *join_rel;
-} JoinHashEntry;
+ Relids relids; /* hash key --- MUST BE FIRST */
+ void *data;
+} RelInfoEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *input_rel);
@@ -346,11 +346,11 @@ find_base_rel(PlannerInfo *root, int relid)
}
/*
- * build_join_rel_hash
- * Construct the auxiliary hash table for join relations.
+ * build_rel_hash
+ * Construct the auxiliary hash table for relation specific data.
*/
static void
-build_join_rel_hash(PlannerInfo *root)
+build_rel_hash(RelInfoList *list)
{
HTAB *hashtab;
HASHCTL hash_ctl;
@@ -359,47 +359,50 @@ build_join_rel_hash(PlannerInfo *root)
/* Create the hash table */
MemSet(&hash_ctl, 0, sizeof(hash_ctl));
hash_ctl.keysize = sizeof(Relids);
- hash_ctl.entrysize = sizeof(JoinHashEntry);
+ hash_ctl.entrysize = sizeof(RelInfoEntry);
hash_ctl.hash = bitmap_hash;
hash_ctl.match = bitmap_match;
hash_ctl.hcxt = CurrentMemoryContext;
- hashtab = hash_create("JoinRelHashTable",
+ hashtab = hash_create("RelHashTable",
256L,
&hash_ctl,
HASH_ELEM | HASH_FUNCTION | HASH_COMPARE | HASH_CONTEXT);
/* Insert all the already-existing joinrels */
- foreach(l, root->join_rel_list)
+ foreach(l, list->items)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
- JoinHashEntry *hentry;
+ void *item = lfirst(l);
+ RelInfoEntry *hentry;
bool found;
+ Relids relids;
- hentry = (JoinHashEntry *) hash_search(hashtab,
- &(rel->relids),
- HASH_ENTER,
- &found);
+ Assert(IsA(item, RelOptInfo));
+ relids = ((RelOptInfo *) item)->relids;
+
+ hentry = (RelInfoEntry *) hash_search(hashtab,
+ &relids,
+ HASH_ENTER,
+ &found);
Assert(!found);
- hentry->join_rel = rel;
+ hentry->data = item;
}
- root->join_rel_hash = hashtab;
+ list->hash = hashtab;
}
/*
- * find_join_rel
- * Returns relation entry corresponding to 'relids' (a set of RT indexes),
- * or NULL if none exists. This is for join relations.
+ * find_rel_info
+ * Find a base or join relation entry.
*/
-RelOptInfo *
-find_join_rel(PlannerInfo *root, Relids relids)
+static void *
+find_rel_info(RelInfoList *list, Relids relids)
{
/*
* Switch to using hash lookup when list grows "too long". The threshold
* is arbitrary and is known only here.
*/
- if (!root->join_rel_hash && list_length(root->join_rel_list) > 32)
- build_join_rel_hash(root);
+ if (!list->hash && list_length(list->items) > 32)
+ build_rel_hash(list);
/*
* Use either hashtable lookup or linear search, as appropriate.
@@ -409,28 +412,32 @@ find_join_rel(PlannerInfo *root, Relids relids)
* so would force relids out of a register and thus probably slow down the
* list-search case.
*/
- if (root->join_rel_hash)
+ if (list->hash)
{
Relids hashkey = relids;
- JoinHashEntry *hentry;
+ RelInfoEntry *hentry;
- hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
- &hashkey,
- HASH_FIND,
- NULL);
+ hentry = (RelInfoEntry *) hash_search(list->hash,
+ &hashkey,
+ HASH_FIND,
+ NULL);
if (hentry)
- return hentry->join_rel;
+ return hentry->data;
}
else
{
ListCell *l;
- foreach(l, root->join_rel_list)
+ foreach(l, list->items)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
+ void *item = lfirst(l);
+ Relids item_relids;
- if (bms_equal(rel->relids, relids))
- return rel;
+ Assert(IsA(item, RelOptInfo));
+ item_relids = ((RelOptInfo *) item)->relids;
+
+ if (bms_equal(item_relids, relids))
+ return item;
}
}
@@ -438,6 +445,58 @@ find_join_rel(PlannerInfo *root, Relids relids)
}
/*
+ * find_join_rel
+ * Returns relation entry corresponding to 'relids' (a set of RT indexes),
+ * or NULL if none exists. This is for join relations.
+ */
+RelOptInfo *
+find_join_rel(PlannerInfo *root, Relids relids)
+{
+ return (RelOptInfo *) find_rel_info(root->join_rel_list, relids);
+}
+
+/*
+ * add_rel_info
+ * Add relation specific info to a list, and also add it to the auxiliary
+ * hashtable if there is one.
+ */
+static void
+add_rel_info(RelInfoList *list, void *data)
+{
+ Assert(IsA(data, RelOptInfo));
+
+ /* GEQO requires us to append the new joinrel to the end of the list! */
+ list->items = lappend(list->items, data);
+
+ /* store it into the auxiliary hashtable if there is one. */
+ if (list->hash)
+ {
+ Relids relids;
+ RelInfoEntry *hentry;
+ bool found;
+
+ relids = ((RelOptInfo *) data)->relids;
+ hentry = (RelInfoEntry *) hash_search(list->hash,
+ &relids,
+ HASH_ENTER,
+ &found);
+ Assert(!found);
+ hentry->data = data;
+ }
+}
+
+/*
+ * add_join_rel
+ * Add given join relation to the list of join relations in the given
+ * PlannerInfo.
+ */
+static void
+add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
+{
+ add_rel_info(root->join_rel_list, joinrel);
+}
+
+/*
* set_foreign_rel_properties
* Set up foreign-join fields if outer and inner relation are foreign
* tables (or joins) belonging to the same server and assigned to the same
@@ -488,32 +547,6 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
}
/*
- * add_join_rel
- * Add given join relation to the list of join relations in the given
- * PlannerInfo. Also add it to the auxiliary hashtable if there is one.
- */
-static void
-add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
-{
- /* GEQO requires us to append the new joinrel to the end of the list! */
- root->join_rel_list = lappend(root->join_rel_list, joinrel);
-
- /* store it into the auxiliary hashtable if there is one. */
- if (root->join_rel_hash)
- {
- JoinHashEntry *hentry;
- bool found;
-
- hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
- &(joinrel->relids),
- HASH_ENTER,
- &found);
- Assert(!found);
- hentry->join_rel = joinrel;
- }
-}
-
-/*
* build_join_rel
* Returns relation entry corresponding to the union of two given rels,
* creating a new relation entry if none already exists.
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index f9389257c6..8e34496764 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -220,6 +220,7 @@ typedef enum NodeTag
T_PlannerInfo,
T_PlannerGlobal,
T_RelOptInfo,
+ T_RelInfoList,
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index a008ae07da..88a2c20fbe 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -234,15 +234,9 @@ struct PlannerInfo
/*
* join_rel_list is a list of all join-relation RelOptInfos we have
- * considered in this planning run. For small problems we just scan the
- * list to do lookups, but when there are many join relations we build a
- * hash table for faster lookups. The hash table is present and valid
- * when join_rel_hash is not NULL. Note that we still maintain the list
- * even when using the hash table for lookups; this simplifies life for
- * GEQO.
+ * considered in this planning run.
*/
- List *join_rel_list; /* list of join-relation RelOptInfos */
- struct HTAB *join_rel_hash; /* optional hashtable for join relations */
+ struct RelInfoList *join_rel_list; /* list of join-relation RelOptInfos */
/*
* When doing a dynamic-programming-style join search, join_rel_level[k]
@@ -734,6 +728,24 @@ typedef struct RelOptInfo
(rel)->part_rels && (rel)->partexprs && (rel)->nullable_partexprs)
/*
+ * RelInfoList
+ * A list to store relation specific info and to retrieve it by relids.
+ *
+ * For small problems we just scan the list to do lookups, but when there are
+ * many relations we build a hash table for faster lookups. The hash table is
+ * present and valid when rel_hash is not NULL. Note that we still maintain
+ * the list even when using the hash table for lookups; this simplifies life
+ * for GEQO.
+ */
+typedef struct RelInfoList
+{
+ NodeTag type;
+
+ List *items;
+ struct HTAB *hash;
+} RelInfoList;
+
+/*
* IndexOptInfo
* Per-index information for planning/optimization
*
v11-002-Introduce_make_join_rel_common.patchtext/x-diffDownload
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index dfbbfdac6d..45255e6ee0 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -38,6 +38,8 @@ static bool is_dummy_rel(RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
+static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1,
+ RelOptInfo *rel2);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist);
@@ -657,21 +659,12 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
return true;
}
-
/*
- * make_join_rel
- * Find or create a join RelOptInfo that represents the join of
- * the two given rels, and add to it path information for paths
- * created with the two rels as outer and inner rel.
- * (The join rel may already contain paths generated from other
- * pairs of rels that add up to the same set of base rels.)
- *
- * NB: will return NULL if attempted join is not valid. This can happen
- * when working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
- */
-RelOptInfo *
-make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+ * make_join_rel_common
+ * The workhorse of make_join_rel().
+ */
+static RelOptInfo *
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -754,6 +747,24 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
}
/*
+ * make_join_rel
+ * Find or create a join RelOptInfo that represents the join of
+ * the two given rels, and add to it path information for paths
+ * created with the two rels as outer and inner rel.
+ * (The join rel may already contain paths generated from other
+ * pairs of rels that add up to the same set of base rels.)
+ *
+ * NB: will return NULL if attempted join is not valid. This can happen when
+ * working with outer joins, or with IN or EXISTS clauses that have been
+ * turned into joins.
+ */
+RelOptInfo *
+make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+{
+ return make_join_rel_common(root, rel1, rel2);
+}
+
+/*
* populate_joinrel_with_paths
* Add paths to the given joinrel for given pair of joining relations. The
* SpecialJoinInfo provides details about the join and the restrictlist
v11-003-Agg_pushdown_basic.patchtext/x-diffDownload
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index e15724bb0e..5e2f4388a9 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2198,8 +2198,8 @@ _copyOnConflictExpr(const OnConflictExpr *from)
/* ****************************************************************
* pathnodes.h copy functions
*
- * We don't support copying RelOptInfo, IndexOptInfo, or Path nodes.
- * There are some subsidiary structs that are useful to copy, though.
+ * We don't support copying RelOptInfo, IndexOptInfo, RelAggInfo or Path
+ * nodes. There are some subsidiary structs that are useful to copy, though.
* ****************************************************************
*/
@@ -2340,6 +2340,20 @@ _copyPlaceHolderInfo(const PlaceHolderInfo *from)
return newnode;
}
+static GroupedVarInfo *
+_copyGroupedVarInfo(const GroupedVarInfo *from)
+{
+ GroupedVarInfo *newnode = makeNode(GroupedVarInfo);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gv_eval_at);
+ COPY_SCALAR_FIELD(derived);
+
+ return newnode;
+}
+
/* ****************************************************************
* parsenodes.h copy functions
* ****************************************************************
@@ -5109,6 +5123,9 @@ copyObjectImpl(const void *from)
case T_PlaceHolderInfo:
retval = _copyPlaceHolderInfo(from);
break;
+ case T_GroupedVarInfo:
+ retval = _copyGroupedVarInfo(from);
+ break;
/*
* VALUE NODES
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8b333f069a..9e86efe784 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2183,6 +2183,8 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_BITMAPSET_FIELD(all_baserels);
WRITE_BITMAPSET_FIELD(nullable_baserels);
WRITE_NODE_FIELD(join_rel_list);
+ WRITE_NODE_FIELD(grouped_rel_list);
+ WRITE_NODE_FIELD(agg_info_list);
WRITE_INT_FIELD(join_cur_level);
WRITE_NODE_FIELD(init_plans);
WRITE_NODE_FIELD(cte_plan_ids);
@@ -2196,6 +2198,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(append_rel_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
@@ -2203,6 +2206,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(distinct_pathkeys);
WRITE_NODE_FIELD(sort_pathkeys);
WRITE_NODE_FIELD(processed_tlist);
+ WRITE_INT_FIELD(max_sortgroupref);
WRITE_NODE_FIELD(minmax_aggs);
WRITE_FLOAT_FIELD(total_table_pages, "%.0f");
WRITE_FLOAT_FIELD(tuple_fraction, "%.4f");
@@ -2422,6 +2426,20 @@ _outParamPathInfo(StringInfo str, const ParamPathInfo *node)
}
static void
+_outRelAggInfo(StringInfo str, const RelAggInfo *node)
+{
+ WRITE_NODE_TYPE("RELAGGINFO");
+
+ WRITE_BITMAPSET_FIELD(relids);
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(agg_input);
+ WRITE_FLOAT_FIELD(input_rows, "%.0f");
+ WRITE_NODE_FIELD(group_clauses);
+ WRITE_NODE_FIELD(group_exprs);
+ WRITE_NODE_FIELD(agg_exprs);
+}
+
+static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
WRITE_NODE_TYPE("RESTRICTINFO");
@@ -2521,6 +2539,18 @@ _outPlaceHolderInfo(StringInfo str, const PlaceHolderInfo *node)
}
static void
+_outGroupedVarInfo(StringInfo str, const GroupedVarInfo *node)
+{
+ WRITE_NODE_TYPE("GROUPEDVARINFO");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_BITMAPSET_FIELD(gv_eval_at);
+ WRITE_BOOL_FIELD(derived);
+}
+
+static void
_outMinMaxAggInfo(StringInfo str, const MinMaxAggInfo *node)
{
WRITE_NODE_TYPE("MINMAXAGGINFO");
@@ -4063,6 +4093,9 @@ outNode(StringInfo str, const void *obj)
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_RelAggInfo:
+ _outRelAggInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
@@ -4081,6 +4114,9 @@ outNode(StringInfo str, const void *obj)
case T_PlaceHolderInfo:
_outPlaceHolderInfo(str, obj);
break;
+ case T_GroupedVarInfo:
+ _outGroupedVarInfo(str, obj);
+ break;
case T_MinMaxAggInfo:
_outMinMaxAggInfo(str, obj);
break;
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
index 89ce373d5e..4633fb8768 100644
--- a/src/backend/optimizer/README
+++ b/src/backend/optimizer/README
@@ -1127,3 +1127,90 @@ breaking down aggregation or grouping over a partitioned relation into
aggregation or grouping over its partitions is called partitionwise
aggregation. Especially when the partition keys match the GROUP BY clause,
this can be significantly faster than the regular method.
+
+Aggregate push-down
+-------------------
+
+The obvious way to evaluate aggregates is to evaluate the FROM clause of the
+SQL query (this is what query_planner does) and use the resuing paths as the
+input of Agg node. However, if the groups are large enough, it may be more
+efficient to apply the partial aggregation to the output of base relation
+scan, and finalize it when we have all relations of the query joined:
+
+ EXPLAIN
+ SELECT a.i, avg(b.u)
+ FROM a JOIN b ON b.j = a.i
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+ -> Index Only Scan using a_pkey on a
+ Index Cond: (i = b.j)
+
+Thus the join above the partial aggregate node receives fewer input rows, and
+so the number of outer-to-inner pairs of tuples to be checked can be
+significantly lower, which can in turn lead to considerably lower join cost.
+
+Note that there's often no GROUP BY expression to be used for the partial
+aggregation, so we use equivalence classes to derive grouping expression: in
+the example above, the grouping key "b.j" was derived from "a.i".
+
+Furthermore, extra grouping columns can be added to the partial Agg node if a
+join clause above that node references a column which is not in the query
+GROUP BY clause and which could not be derived using equivalence class.
+
+ EXPLAIN
+ SELECT a.i, avg(b.u)
+ FROM a JOIN b ON b.j = a.x
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (a.x = b.j)
+ -> Seq Scan on a
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+
+Here the partial aggregate uses "b.j" as grouping column although it's not in
+the same equivalence class as "a.i". Note that no column of the {a.x, b.j}
+equivalence class is used as a key for the final aggregation.
+
+Besides base relation, the aggregation can also be pushed down to join:
+
+ EXPLAIN
+ SELECT a.i, avg(b.u + c.v)
+ FROM a JOIN b ON b.j = a.i
+ JOIN c ON c.k = a.i
+ WHERE b.j = c.k GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (b.j = a.i)
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Hash Join
+ Hash Cond: (b.j = c.k)
+ -> Seq Scan on b
+ -> Hash
+ -> Seq Scan on c
+ -> Hash
+ -> Seq Scan on a
+
+Whether the Agg node is created out of base relation or out of join, it's
+added to a separate RelOptInfo that we call "grouped relation". Grouped
+relation can be joined to a non-grouped relation, which results in a grouped
+relation too. Join of two grouped relations does not seem to be very useful
+and is currently not supported.
+
+If query_planner produces a grouped relation that contains valid paths, these
+are simply added to the UPPERREL_PARTIAL_GROUP_AGG relation. Further
+processing of these paths then does not differ from processing of other
+partially grouped paths.
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 8586b8f3b8..859c0d6397 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -59,6 +59,7 @@ typedef struct pushdown_safety_info
/* These parameters are set by GUC */
bool enable_geqo = false; /* just in case GUC doesn't set it */
+bool enable_agg_pushdown;
int geqo_threshold;
int min_parallel_table_scan_size;
int min_parallel_index_scan_size;
@@ -121,6 +122,9 @@ static void set_result_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
+static void add_grouped_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, AggStrategy aggstrategy,
+ RelAggInfo *agg_info);
static RelOptInfo *make_rel_from_joinlist(PlannerInfo *root, List *joinlist);
static bool subquery_is_pushdown_safe(Query *subquery, Query *topquery,
pushdown_safety_info *safetyInfo);
@@ -1709,6 +1713,7 @@ add_paths_to_append_rel(PlannerInfo *root, RelOptInfo *rel,
}
}
+
/*
* generate_mergeappend_paths
* Generate MergeAppend paths for an append relation
@@ -2566,6 +2571,80 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
}
/*
+ * generate_grouping_paths
+ * Create partially aggregated paths and add them to grouped relation.
+ *
+ * "rel_plain" is base or join relation whose paths are not grouped.
+ */
+void
+generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
+ RelOptInfo *rel_plain, RelAggInfo *agg_info)
+{
+ ListCell *lc;
+
+ if (IS_DUMMY_REL(rel_plain))
+ {
+ mark_dummy_rel(rel_grouped);
+ return;
+ }
+
+ foreach(lc, rel_plain->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ /*
+ * Since the path originates from the non-grouped relation which is
+ * not aware of the aggregate push-down, we must ensure that it
+ * provides the correct input for aggregation.
+ */
+ path = (Path *) create_projection_path(root, rel_grouped, path,
+ agg_info->agg_input);
+
+ /*
+ * add_grouped_path() will check whether the path has suitable
+ * pathkeys.
+ */
+ add_grouped_path(root, rel_grouped, path, AGG_SORTED, agg_info);
+
+ /*
+ * Repeated creation of hash table (for new parameter values) should
+ * be possible, does not sound like a good idea in terms of
+ * efficiency.
+ */
+ if (path->param_info == NULL)
+ add_grouped_path(root, rel_grouped, path, AGG_HASHED, agg_info);
+ }
+
+ /* Could not generate any grouped paths? */
+ if (rel_grouped->pathlist == NIL)
+ mark_dummy_rel(rel_grouped);
+}
+
+/*
+ * Apply partial aggregation to a subpath and add the AggPath to the pathlist.
+ */
+static void
+add_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ AggStrategy aggstrategy, RelAggInfo *agg_info)
+{
+ Path *agg_path;
+
+
+ if (aggstrategy == AGG_HASHED)
+ agg_path = (Path *) create_agg_hashed_path(root, rel, subpath,
+ agg_info);
+ else if (aggstrategy == AGG_SORTED)
+ agg_path = (Path *) create_agg_sorted_path(root, rel, subpath,
+ agg_info);
+ else
+ elog(ERROR, "unexpected strategy %d", aggstrategy);
+
+ /* Add the grouped path to the list of grouped base paths. */
+ if (agg_path != NULL)
+ add_path(rel, (Path *) agg_path);
+}
+
+/*
* make_rel_from_joinlist
* Build access paths using a "joinlist" to guide the join path search.
*
@@ -2605,6 +2684,34 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
int varno = ((RangeTblRef *) jlnode)->rtindex;
thisrel = find_base_rel(root, varno);
+
+ /*
+ * Create a grouped relation to facilitate aggregate push-down.
+ * This makes no sense if thisrel is the only relation of the
+ * query.
+ */
+ if (bms_nonempty_difference(root->all_baserels, thisrel->relids))
+ {
+ RelOptInfo *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /*
+ * Build grouped relation if thisrel is suitable for partial
+ * aggregation.
+ */
+ rel_grouped = build_simple_grouped_rel(root, varno, &agg_info);
+
+ if (rel_grouped)
+ {
+ /* Make the relation available for joining. */
+ add_grouped_rel(root, rel_grouped, agg_info);
+
+ /* Add the aggregation paths to it. */
+ generate_grouping_paths(root, rel_grouped, thisrel,
+ agg_info);
+ set_cheapest(rel_grouped);
+ }
+ }
}
else if (IsA(jlnode, List))
{
@@ -2706,6 +2813,7 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
for (lev = 2; lev <= levels_needed; lev++)
{
+ RelOptInfo *rel_grouped;
ListCell *lc;
/*
@@ -2742,6 +2850,11 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
+ /* The same for grouped relation if one exists. */
+ rel_grouped = find_grouped_rel(root, rel->relids, NULL);
+ if (rel_grouped)
+ set_cheapest(rel_grouped);
+
#ifdef OPTIMIZER_DEBUG
debug_print_rel(root, rel);
#endif
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 4b9be13f08..47a90c3537 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4355,7 +4355,6 @@ set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel)
0,
JOIN_INNER,
NULL);
-
rel->rows = clamp_row_est(nrows);
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
@@ -5333,11 +5332,11 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
foreach(lc, target->exprs)
{
Node *node = (Node *) lfirst(lc);
+ int32 item_width;
if (IsA(node, Var))
{
Var *var = (Var *) node;
- int32 item_width;
/* We should not see any upper-level Vars here */
Assert(var->varlevelsup == 0);
@@ -5368,6 +5367,20 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
Assert(item_width > 0);
tuple_width += item_width;
}
+ else if (IsA(node, Aggref))
+ {
+ /*
+ * If the target is evaluated by AggPath, it'll care of cost
+ * estimate. If the target is above AggPath (typically target of a
+ * join relation that contains grouped relation), the cost of
+ * Aggref should not be accounted for again.
+ *
+ * On the other hand, width is always needed.
+ */
+ item_width = get_typavgwidth(exprType(node), exprTypmod(node));
+ Assert(item_width > 0);
+ tuple_width += item_width;
+ }
else
{
/*
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 61b5b119b0..52d1ddc1a9 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -2521,6 +2521,140 @@ is_redundant_derived_clause(RestrictInfo *rinfo, List *clauselist)
}
/*
+ * translate_expression_to_rels
+ * If the appropriate equivalence classes exist, replace vars in
+ * gvi->gvexpr with vars whose varno is equal to relid. Return NULL if
+ * translation is not possible or needed.
+ *
+ * Note: Currently we only translate Var expressions. This is subject to
+ * change as the aggregate push-down feature gets enhanced.
+ */
+GroupedVarInfo *
+translate_expression_to_rels(PlannerInfo *root, GroupedVarInfo *gvi,
+ Index relid)
+{
+ Var *var;
+ ListCell *l1;
+ bool found_orig = false;
+ Var *var_translated = NULL;
+ GroupedVarInfo *result;
+
+ /* Can't do anything w/o equivalence classes. */
+ if (root->eq_classes == NIL)
+ return NULL;
+
+ var = castNode(Var, gvi->gvexpr);
+
+ /*
+ * Do we need to translate the var?
+ */
+ if (var->varno == relid)
+ return NULL;
+
+ /*
+ * Find the replacement var.
+ */
+ foreach(l1, root->eq_classes)
+ {
+ EquivalenceClass *ec = lfirst_node(EquivalenceClass, l1);
+ ListCell *l2;
+
+ /* TODO Check if any other EC kind should be ignored. */
+ if (ec->ec_has_volatile || ec->ec_below_outer_join || ec->ec_broken)
+ continue;
+
+ /* Single-element EC can hardly help in translations. */
+ if (list_length(ec->ec_members) == 1)
+ continue;
+
+ /*
+ * Collect all vars of this EC and their varnos.
+ *
+ * ec->ec_relids does not help because we're only interested in a
+ * subset of EC members.
+ */
+ foreach(l2, ec->ec_members)
+ {
+ EquivalenceMember *em = lfirst_node(EquivalenceMember, l2);
+ Var *ec_var;
+
+ /*
+ * The grouping expressions derived here are used to evaluate
+ * possibility to push aggregation down to RELOPT_BASEREL or
+ * RELOPT_JOINREL relations, and to construct reltargets for the
+ * grouped rels. We're not interested at the moment whether the
+ * relations do have children.
+ */
+ if (em->em_is_child)
+ continue;
+
+ if (!IsA(em->em_expr, Var))
+ continue;
+
+ ec_var = castNode(Var, em->em_expr);
+ if (equal(ec_var, var))
+ found_orig = true;
+ else if (ec_var->varno == relid)
+ var_translated = ec_var;
+
+ if (found_orig && var_translated)
+ {
+ /*
+ * The replacement Var must have the same data type, otherwise
+ * the values are not guaranteed to be grouped in the same way
+ * as values of the original Var.
+ */
+ if (ec_var->vartype != var->vartype)
+ return NULL;
+
+ break;
+ }
+ }
+
+ if (found_orig)
+ {
+ /*
+ * The same expression probably does not exist in multiple ECs.
+ */
+ if (var_translated == NULL)
+ {
+ /*
+ * Failed to translate the expression.
+ */
+ return NULL;
+ }
+ else
+ {
+ /* Success. */
+ break;
+ }
+ }
+ else
+ {
+ /*
+ * Vars of the requested relid can be in the next ECs too.
+ */
+ var_translated = NULL;
+ }
+ }
+
+ if (!found_orig)
+ return NULL;
+
+ result = makeNode(GroupedVarInfo);
+ memcpy(result, gvi, sizeof(GroupedVarInfo));
+
+ /*
+ * translate_expression_to_rels_mutator updates gv_eval_at.
+ */
+ result->gv_eval_at = bms_make_singleton(relid);
+ result->gvexpr = (Expr *) var_translated;
+ result->derived = true;
+
+ return result;
+}
+
+/*
* is_redundant_with_indexclauses
* Test whether rinfo is redundant with any clause in the IndexClause
* list. Here, for convenience, we test both simple identity and
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 45255e6ee0..a75fc8b8c6 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -24,6 +24,7 @@
#include "partitioning/partbounds.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+#include "utils/selfuncs.h"
static void make_rels_by_clause_joins(PlannerInfo *root,
@@ -39,7 +40,9 @@ static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1,
- RelOptInfo *rel2);
+ RelOptInfo *rel2,
+ RelAggInfo *agg_info,
+ RelOptInfo *rel_agg_input);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist);
@@ -662,9 +665,17 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
/*
* make_join_rel_common
* The workhorse of make_join_rel().
- */
+ *
+ * 'agg_info' contains the reltarget of grouped relation and everything we
+ * need to aggregate the join result. If NULL, then the join relation should
+ * not be grouped.
+ *
+ * 'rel_agg_input' describes the AggPath input relation if the join output
+ * should be aggregated. If NULL is passed, do not aggregate the join output.
+ */
static RelOptInfo *
-make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ RelAggInfo *agg_info, RelOptInfo *rel_agg_input)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -725,7 +736,7 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* goes with this particular joining.
*/
joinrel = build_join_rel(root, joinrelids, rel1, rel2, sjinfo,
- &restrictlist);
+ &restrictlist, agg_info);
/*
* If we've already proven this join is empty, we needn't consider any
@@ -738,8 +749,26 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
}
/* Add paths to the join relation. */
- populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
- restrictlist);
+ if (rel_agg_input == NULL)
+ {
+ /*
+ * Simply join the input relations, whether both are plain or one of
+ * them is grouped.
+ */
+ populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
+ restrictlist);
+ }
+ else
+ {
+ /* The join relation is grouped. */
+ Assert(agg_info != NULL);
+
+ /*
+ * Apply partial aggregation to the paths of rel_agg_input and add the
+ * resulting paths to joinrel.
+ */
+ generate_grouping_paths(root, joinrel, rel_agg_input, agg_info);
+ }
bms_free(joinrelids);
@@ -747,6 +776,62 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
}
/*
+ * make_join_rel_combined
+ * Join grouped relation to non-grouped one.
+ */
+static void
+make_join_rel_combined(PlannerInfo *root, RelOptInfo *rel1,
+ RelOptInfo *rel2,
+ RelAggInfo *agg_info)
+{
+ RelOptInfo *rel1_grouped;
+ RelOptInfo *rel2_grouped;
+ bool rel1_grouped_useful = false;
+ bool rel2_grouped_useful = false;
+
+ /* Retrieve the grouped relations. */
+ rel1_grouped = find_grouped_rel(root, rel1->relids, NULL);
+ rel2_grouped = find_grouped_rel(root, rel2->relids, NULL);
+
+ /*
+ * Dummy rel may indicate a join relation that is able to generate grouped
+ * paths as such (i.e. it has valid agg_info), but for which the path
+ * actually could not be created (e.g. only AGG_HASHED strategy was
+ * possible but work_mem was not sufficient for hash table).
+ */
+ rel1_grouped_useful = rel1_grouped != NULL && !IS_DUMMY_REL(rel1_grouped);
+ rel2_grouped_useful = rel2_grouped != NULL && !IS_DUMMY_REL(rel2_grouped);
+
+ /* Nothing to do if there's no grouped relation. */
+ if (!rel1_grouped_useful && !rel2_grouped_useful)
+ return;
+
+ /*
+ * At maximum one input rel can be grouped (here we don't care if any rel
+ * is eventually dummy, the existence of grouped rel indicates that
+ * aggregates can be pushed down to it). If both were grouped, then
+ * grouping of one side would change the occurrence of the other side's
+ * aggregate transient states on the input of the final aggregation. This
+ * can be handled by adjusting the transient states, but it's not worth
+ * the effort because it's hard to find a use case for this kind of join.
+ *
+ * XXX If the join of two grouped rels is implemented someday, note that
+ * both rels can have aggregates, so it'd be hard to join grouped rel to
+ * non-grouped here: 1) such a "mixed join" would require a special
+ * target, 2) both AGGSPLIT_FINAL_DESERIAL and AGGSPLIT_SIMPLE aggregates
+ * could appear in the target of the final aggregation node, originating
+ * from the grouped and the non-grouped input rel respectively.
+ */
+ if (rel1_grouped && rel2_grouped)
+ return;
+
+ if (rel1_grouped_useful)
+ make_join_rel_common(root, rel1_grouped, rel2, agg_info, NULL);
+ else if (rel2_grouped_useful)
+ make_join_rel_common(root, rel1, rel2_grouped, agg_info, NULL);
+}
+
+/*
* make_join_rel
* Find or create a join RelOptInfo that represents the join of
* the two given rels, and add to it path information for paths
@@ -754,14 +839,83 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* (The join rel may already contain paths generated from other
* pairs of rels that add up to the same set of base rels.)
*
+ * In addition to creating an ordinary join relation, try to create a
+ * grouped one. There are two strategies to achieve that: join a grouped
+ * relation to plain one, or join two plain relations and apply partial
+ * aggregation to the result.
+ *
* NB: will return NULL if attempted join is not valid. This can happen when
* working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
+ * turned into joins. Besides that, NULL is also returned if caller is
+ * interested in a grouped relation but it could not be created.
+ *
+ * Only the plain relation is returned; if grouped relation exists, it can be
+ * retrieved using find_grouped_rel().
*/
RelOptInfo *
make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
{
- return make_join_rel_common(root, rel1, rel2);
+ Relids joinrelids;
+ RelAggInfo *agg_info = NULL;
+ RelOptInfo *joinrel,
+ *joinrel_plain;
+
+ /* 1) form the plain join. */
+ joinrel = make_join_rel_common(root, rel1, rel2, NULL, NULL);
+ joinrel_plain = joinrel;
+
+ if (joinrel_plain == NULL)
+ return joinrel_plain;
+
+ /*
+ * We're done if there are no grouping expressions nor aggregates.
+ */
+ if (root->grouped_var_list == NIL)
+ return joinrel_plain;
+
+ joinrelids = bms_union(rel1->relids, rel2->relids);
+ joinrel = find_grouped_rel(root, joinrelids, &agg_info);
+
+ if (joinrel != NULL)
+ {
+ /*
+ * If the same grouped joinrel was already formed, just with the base
+ * rels divided between rel1 and rel2 in a different way, the matching
+ * agg_info should already be there.
+ */
+ Assert(agg_info != NULL);
+ }
+ else
+ {
+ /*
+ * agg_info must be created from scratch.
+ */
+ agg_info = create_rel_agg_info(root, joinrel_plain);
+
+ /* Cannot we build grouped join? */
+ if (agg_info == NULL)
+ return joinrel_plain;
+
+ /*
+ * The number of aggregate input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ agg_info->input_rows = joinrel_plain->rows;
+ }
+
+ /*
+ * 2) join two plain rels and aggregate the join paths. Aggregate
+ * push-down only makes sense if the join is not the top-level one.
+ */
+ if (bms_nonempty_difference(root->all_baserels, joinrelids))
+ make_join_rel_common(root, rel1, rel2, agg_info, joinrel_plain);
+
+ /*
+ * 3) combine plain and grouped relations.
+ */
+ make_join_rel_combined(root, rel1, rel2, agg_info);
+
+ return joinrel_plain;
}
/*
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 2afc3f1dfe..c3033b76c5 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -47,6 +47,8 @@ typedef struct PostponedQual
} PostponedQual;
+static void create_aggregate_grouped_var_infos(PlannerInfo *root);
+static void create_grouping_expr_grouped_var_infos(PlannerInfo *root);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
@@ -242,6 +244,261 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
}
}
+/*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate as well as for
+ * each possible grouping expression.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+extern void
+setup_aggregate_pushdown(PlannerInfo *root)
+{
+ ListCell *lc;
+
+ /*
+ * Isn't user interested in the aggregate push-down feature?
+ */
+ if (!enable_agg_pushdown)
+ return;
+
+ /* The feature can only be applied to grouped aggregation. */
+ if (!root->parse->groupClause)
+ return;
+
+ /*
+ * Grouping sets require multiple different groupings but the base
+ * relation can only generate one.
+ */
+ if (root->parse->groupingSets)
+ return;
+
+ /*
+ * SRF is not allowed in the aggregate argument and we don't even want it
+ * in the GROUP BY clause, so forbid it in general. It needs to be
+ * analyzed if evaluation of a GROUP BY clause containing SRF below the
+ * query targetlist would be correct. Currently it does not seem to be an
+ * important use case.
+ */
+ if (root->parse->hasTargetSRFs)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_aggregate_grouped_var_infos(root);
+
+ /* Isn't there any aggregate to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Create GroupedVarInfo per grouping expression. */
+ create_grouping_expr_grouped_var_infos(root);
+
+ /* Isn't there any useful grouping expression for aggregate push-down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /*
+ * Now that we know that grouping can be pushed down, search for the
+ * maximum sortgroupref. The base relations may need it if extra grouping
+ * expressions get added to them.
+ */
+ Assert(root->max_sortgroupref == 0);
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc);
+
+ if (te->ressortgroupref > root->max_sortgroupref)
+ root->max_sortgroupref = te->ressortgroupref;
+ }
+}
+
+/*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ */
+static void
+create_aggregate_grouped_var_infos(PlannerInfo *root)
+{
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES |
+ PVC_INCLUDE_WINDOWFUNCS);
+
+ /*
+ * Although GroupingFunc is related to root->parse->groupingSets, this
+ * field does not necessarily reflect its presence.
+ */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, GroupingFunc))
+ return;
+ }
+
+ /*
+ * Aggregates within the HAVING clause need to be processed in the same
+ * way as those in the main targetlist.
+ *
+ * Note that the contained aggregates will be pushed down, but the
+ * containing HAVING clause must be ignored until the aggregation is
+ * finalized.
+ */
+ if (root->parse->havingQual != NULL)
+ {
+ List *having_exprs;
+
+ having_exprs = pull_var_clause((Node *) root->parse->havingQual,
+ PVC_INCLUDE_AGGREGATES);
+ if (having_exprs != NIL)
+ tlist_exprs = list_concat(tlist_exprs, having_exprs);
+ }
+
+ if (tlist_exprs == NIL)
+ return;
+
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ /*
+ * tlist_exprs may also contain Vars or WindowFuncs, but we only need
+ * Aggrefs.
+ */
+ if (IsA(expr, Var) ||IsA(expr, WindowFunc))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct)
+ {
+ /*
+ * Aggregation push-down is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ gvi = lfirst_node(GroupedVarInfo, lc2);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ gvi = makeNode(GroupedVarInfo);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ gvi->gv_eval_at = NULL;
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+}
+
+/*
+ * Create GroupedVarInfo for each expression usable as grouping key.
+ *
+ * In addition to the expressions of the query targetlist, group_pathkeys is
+ * also considered the source of grouping expressions. That increases the
+ * chance to get the relation output grouped.
+ */
+static void
+create_grouping_expr_grouped_var_infos(PlannerInfo *root)
+{
+ ListCell *l1,
+ *l2;
+ List *exprs = NIL;
+ List *sortgrouprefs = NIL;
+
+ /*
+ * Make sure GroupedVarInfo exists for each expression usable as grouping
+ * key.
+ */
+ foreach(l1, root->parse->groupClause)
+ {
+ SortGroupClause *sgClause;
+ TargetEntry *te;
+ Index sortgroupref;
+
+ sgClause = lfirst_node(SortGroupClause, l1);
+ te = get_sortgroupclause_tle(sgClause, root->processed_tlist);
+ sortgroupref = te->ressortgroupref;
+
+ Assert(sortgroupref > 0);
+
+ /*
+ * Non-zero sortgroupref does not necessarily imply grouping
+ * expression: data can also be sorted by aggregate.
+ */
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ /*
+ * The aggregate push-down feature currently supports only plain Vars
+ * as grouping expressions.
+ */
+ if (!IsA(te->expr, Var))
+ {
+ root->grouped_var_list = NIL;
+ return;
+ }
+
+ exprs = lappend(exprs, te->expr);
+ sortgrouprefs = lappend_int(sortgrouprefs, sortgroupref);
+ }
+
+ /*
+ * Construct GroupedVarInfo for each expression.
+ */
+ forboth(l1, exprs, l2, sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, l1);
+ int sortgroupref = lfirst_int(l2);
+ GroupedVarInfo *gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvexpr = (Expr *) copyObject(var);
+ gvi->sortgroupref = sortgroupref;
+
+ /* Find out where the expression should be evaluated. */
+ gvi->gv_eval_at = bms_make_singleton(var->varno);
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+}
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c
index 86617099df..474db04c97 100644
--- a/src/backend/optimizer/plan/planagg.c
+++ b/src/backend/optimizer/plan/planagg.c
@@ -442,7 +442,7 @@ build_minmax_path(PlannerInfo *root, MinMaxAggInfo *mminfo,
subroot->tuple_fraction = 1.0;
subroot->limit_tuples = 1.0;
- final_rel = query_planner(subroot, tlist, minmax_qp_callback, NULL);
+ final_rel = query_planner(subroot, tlist, minmax_qp_callback, NULL, NULL);
/*
* Since we didn't go through subquery_planner() to handle the subquery,
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 52d9943192..b344fd75ad 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -47,6 +47,9 @@
* qp_callback is a function to compute query_pathkeys once it's safe to do so
* qp_extra is optional extra data to pass to qp_callback
*
+ * If final_rel_grouped_p is valid, relation containing grouped paths may be
+ * saved to *final_rel_grouped_p.
+ *
* Note: the PlannerInfo node also includes a query_pathkeys field, which
* tells query_planner the sort order that is desired in the final output
* plan. This value is *not* available at call time, but is computed by
@@ -55,7 +58,8 @@
*/
RelOptInfo *
query_planner(PlannerInfo *root, List *tlist,
- query_pathkeys_callback qp_callback, void *qp_extra)
+ query_pathkeys_callback qp_callback, void *qp_extra,
+ RelOptInfo **final_rel_grouped_p)
{
Query *parse = root->parse;
List *joinlist;
@@ -68,6 +72,8 @@ query_planner(PlannerInfo *root, List *tlist,
* here.
*/
root->join_rel_list = makeNode(RelInfoList);
+ root->grouped_rel_list = makeNode(RelInfoList);
+ root->agg_info_list = makeNode(RelInfoList);
root->join_rel_level = NULL;
root->join_cur_level = 0;
root->canon_pathkeys = NIL;
@@ -76,6 +82,7 @@ query_planner(PlannerInfo *root, List *tlist,
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
@@ -259,6 +266,16 @@ query_planner(PlannerInfo *root, List *tlist,
extract_restriction_or_clauses(root);
/*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, setup
+ * root->grouped_var_list.
+ *
+ * The base relations should be fully initialized now, so that we have
+ * enough info to decide whether grouping is possible.
+ */
+ setup_aggregate_pushdown(root);
+
+ /*
* Ready to do the primary planning.
*/
final_rel = make_one_rel(root, joinlist);
@@ -268,5 +285,9 @@ query_planner(PlannerInfo *root, List *tlist,
final_rel->cheapest_total_path->param_info != NULL)
elog(ERROR, "failed to construct the join relation");
+ if (final_rel_grouped_p)
+ *final_rel_grouped_p = find_grouped_rel(root, final_rel->relids,
+ NULL);
+
return final_rel;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index bc81535905..2933bf7560 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -150,6 +150,7 @@ static double get_number_of_groups(PlannerInfo *root,
List *target_list);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
PathTarget *target,
bool target_parallel_safe,
const AggClauseCosts *agg_costs,
@@ -163,6 +164,7 @@ static RelOptInfo *make_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
Node *havingQual);
static void create_ordinary_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
RelOptInfo *grouped_rel,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd,
@@ -630,6 +632,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
memset(root->upper_rels, 0, sizeof(root->upper_rels));
memset(root->upper_targets, 0, sizeof(root->upper_targets));
root->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
root->grouping_map = NULL;
root->minmax_aggs = NIL;
root->qual_security_level = 0;
@@ -1820,6 +1823,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
List *activeWindows = NIL;
grouping_sets_data *gset_data = NULL;
standard_qp_extra qp_extra;
+ RelOptInfo *current_rel_grouped = NULL;
/* A recursive query should always have setOperations */
Assert(!root->hasRecursion);
@@ -1927,7 +1931,8 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
* of the query's sort clause, distinct clause, etc.
*/
current_rel = query_planner(root, tlist,
- standard_qp_callback, &qp_extra);
+ standard_qp_callback, &qp_extra,
+ ¤t_rel_grouped);
/*
* Convert the query's result tlist into PathTarget format.
@@ -2070,6 +2075,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
{
current_rel = create_grouping_paths(root,
current_rel,
+ current_rel_grouped,
grouping_target,
grouping_target_parallel_safe,
&agg_costs,
@@ -3701,6 +3707,7 @@ get_number_of_groups(PlannerInfo *root,
static RelOptInfo *
create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
PathTarget *target,
bool target_parallel_safe,
const AggClauseCosts *agg_costs,
@@ -3791,7 +3798,8 @@ create_grouping_paths(PlannerInfo *root,
else
extra.patype = PARTITIONWISE_AGGREGATE_NONE;
- create_ordinary_grouping_paths(root, input_rel, grouped_rel,
+ create_ordinary_grouping_paths(root, input_rel, input_rel_grouped,
+ grouped_rel,
agg_costs, gd, &extra,
&partially_grouped_rel);
}
@@ -3948,6 +3956,7 @@ create_degenerate_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
*/
static void
create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
RelOptInfo *grouped_rel,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd,
@@ -3997,13 +4006,23 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
if ((extra->flags & GROUPING_CAN_PARTIAL_AGG) != 0)
{
bool force_rel_creation;
+ bool have_agg_pushdown_paths;
/*
- * If we're doing partitionwise aggregation at this level, force
- * creation of a partially_grouped_rel so we can add partitionwise
- * paths to it.
+ * Check if the aggregate push-down feature succeeded to generate any
+ * paths. Dummy relation can appear here because grouped paths are not
+ * guaranteed to exist for a relation.
*/
- force_rel_creation = (patype == PARTITIONWISE_AGGREGATE_PARTIAL);
+ have_agg_pushdown_paths = input_rel_grouped != NULL &&
+ !IS_DUMMY_REL(input_rel_grouped);
+
+ /*
+ * If we're doing partitionwise aggregation at this level or if
+ * aggregate push-down succeeded to create some paths, force creation
+ * of a partially_grouped_rel so we can add the related paths to it.
+ */
+ force_rel_creation = patype == PARTITIONWISE_AGGREGATE_PARTIAL ||
+ have_agg_pushdown_paths;
partially_grouped_rel =
create_partial_grouping_paths(root,
@@ -4012,6 +4031,23 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
gd,
extra,
force_rel_creation);
+
+ /*
+ * No further processing is needed for paths provided by the aggregate
+ * push-down feature. Simply add them to the partially grouped
+ * relation.
+ */
+ if (have_agg_pushdown_paths)
+ {
+ ListCell *lc;
+
+ foreach(lc, input_rel_grouped->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ add_path(partially_grouped_rel, path);
+ }
+ }
}
/* Set out parameter. */
@@ -4036,10 +4072,14 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
/* Gather any partially grouped partial paths. */
if (partially_grouped_rel && partially_grouped_rel->partial_pathlist)
- {
gather_grouping_paths(root, partially_grouped_rel);
+
+ /*
+ * The non-partial paths can come either from the Gather above or from
+ * aggregate push-down.
+ */
+ if (partially_grouped_rel && partially_grouped_rel->pathlist)
set_cheapest(partially_grouped_rel);
- }
/*
* Estimate number of groups.
@@ -7200,6 +7240,7 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
/* Create grouping paths for this child relation. */
create_ordinary_grouping_paths(root, child_input_rel,
+ NULL,
child_grouped_rel,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 0213a37670..d618007929 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -2300,6 +2300,39 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, Aggref))
+ {
+ Aggref *aggref = castNode(Aggref, node);
+
+ /*
+ * The upper plan targetlist can contain Aggref whose value has
+ * already been evaluated by the subplan. However this can only happen
+ * with specific value of aggsplit.
+ */
+ if (aggref->aggsplit == AGGSPLIT_INITIAL_SERIAL)
+ {
+ /* See if the Aggref has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ }
+
+ /* No referent found for Aggref */
+ elog(ERROR, "Aggref not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index aebe162713..9886c18bbd 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -891,6 +891,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
memset(subroot->upper_rels, 0, sizeof(subroot->upper_rels));
memset(subroot->upper_targets, 0, sizeof(subroot->upper_targets));
subroot->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
subroot->grouping_map = NULL;
subroot->minmax_aggs = NIL;
subroot->qual_security_level = 0;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 169e51e792..7248fa0262 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2949,6 +2949,146 @@ create_agg_path(PlannerInfo *root,
}
/*
+ * Apply AGG_SORTED aggregation path to subpath if it's suitably sorted.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+AggPath *
+create_agg_sorted_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ RelAggInfo *agg_info)
+{
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ ListCell *lc1;
+ List *key_subset = NIL;
+ AggPath *result = NULL;
+
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, (Node *) agg_exprs, aggsplit, &agg_costs);
+
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until the
+ * final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath, target,
+ AGG_SORTED, aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+
+ return result;
+}
+
+/*
+ * Apply AGG_HASHED aggregation to subpath.
+ */
+AggPath *
+create_agg_hashed_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, RelAggInfo *agg_info)
+{
+ bool can_hash;
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ double hashaggtablesize;
+ Query *parse = root->parse;
+ AggPath *result = NULL;
+
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, agg_exprs, aggsplit, &agg_costs);
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until
+ * the final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath,
+ target,
+ AGG_HASHED,
+ aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+
+ /*
+ * The agg path should require no fewer parameters than the plain
+ * one.
+ */
+ result->path.param_info = subpath->param_info;
+ }
+ }
+
+ return result;
+}
+
+/*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
*
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index bcf2ffefb4..190513fb7d 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -17,18 +17,24 @@
#include <limits.h>
#include "miscadmin.h"
+#include "catalog/pg_class_d.h"
+#include "catalog/pg_constraint.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
+#include "optimizer/optimizer.h"
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/placeholder.h"
#include "optimizer/plancat.h"
+#include "optimizer/planner.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+#include "parser/parse_oper.h"
#include "partitioning/partbounds.h"
#include "utils/hsearch.h"
+#include "utils/selfuncs.h"
typedef struct RelInfoEntry
@@ -63,6 +69,9 @@ static void build_child_join_reltarget(PlannerInfo *root,
RelOptInfo *childrel,
int nappinfos,
AppendRelInfo **appinfos);
+static bool init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p);
/*
@@ -323,6 +332,102 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
}
/*
+ * build_simple_grouped_rel
+ * Construct a new RelOptInfo for a grouped base relation out of an
+ * existing non-grouped relation. On success, pointer to the corresponding
+ * RelAggInfo is stored in *agg_info_p in addition to returning the grouped
+ * relation.
+ */
+RelOptInfo *
+build_simple_grouped_rel(PlannerInfo *root, int relid,
+ RelAggInfo **agg_info_p)
+{
+ RangeTblEntry *rte;
+ RelOptInfo *rel_plain,
+ *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /* Isn't there any grouping expression to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return NULL;
+
+ rel_plain = root->simple_rel_array[relid];
+
+ /* Caller should only pass rti that represents base relation. */
+ Assert(rel_plain != NULL);
+
+ /*
+ * Not all RTE kinds are supported when grouping is considered.
+ *
+ * TODO Consider relaxing some of these restrictions.
+ */
+ rte = root->simple_rte_array[rel_plain->relid];
+ if (rte->rtekind != RTE_RELATION ||
+ rte->relkind == RELKIND_FOREIGN_TABLE ||
+ rte->tablesample != NULL)
+ return NULL;
+
+ /*
+ * Grouped append relation is not supported yet.
+ */
+ if (rte->inh)
+ return NULL;
+
+ /*
+ * Currently we do not support child relations ("other rels").
+ */
+ if (rel_plain->reloptkind != RELOPT_BASEREL)
+ return NULL;
+
+ /*
+ * Prepare the information we need for aggregation of the rel contents.
+ */
+ agg_info = create_rel_agg_info(root, rel_plain);
+ if (agg_info == NULL)
+ return NULL;
+
+ /*
+ * TODO Consider if 1) a flat copy is o.k., 2) it's safer in terms of
+ * adding new fields to RelOptInfo) to copy everything and then reset some
+ * fields, or to zero the structure and copy individual fields.
+ */
+ rel_grouped = makeNode(RelOptInfo);
+ memcpy(rel_grouped, rel_plain, sizeof(RelOptInfo));
+
+ /*
+ * Note on consider_startup: while the AGG_HASHED strategy needs the whole
+ * relation, AGG_SORTED does not. Therefore we do not force
+ * consider_startup to false.
+ */
+
+ /*
+ * Set the appropriate target for grouped paths.
+ *
+ * reltarget should match the target of partially aggregated paths.
+ */
+ rel_grouped->reltarget = agg_info->target;
+
+ /*
+ * Grouped paths must not be mixed with the plain ones.
+ */
+ rel_grouped->pathlist = NIL;
+ rel_grouped->partial_pathlist = NIL;
+ rel_grouped->cheapest_startup_path = NULL;
+ rel_grouped->cheapest_total_path = NULL;
+ rel_grouped->cheapest_unique_path = NULL;
+ rel_grouped->cheapest_parameterized_paths = NIL;
+
+ /*
+ * The number of aggregation input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ agg_info->input_rows = rel_plain->rows;
+
+ *agg_info_p = agg_info;
+ return rel_grouped;
+}
+
+/*
* find_base_rel
* Find a base or other relation entry, which must already exist.
*/
@@ -433,8 +538,12 @@ find_rel_info(RelInfoList *list, Relids relids)
void *item = lfirst(l);
Relids item_relids;
- Assert(IsA(item, RelOptInfo));
- item_relids = ((RelOptInfo *) item)->relids;
+ Assert(IsA(item, RelOptInfo) ||IsA(item, RelAggInfo));
+
+ if (IsA(item, RelOptInfo))
+ item_relids = ((RelOptInfo *) item)->relids;
+ else if (IsA(item, RelAggInfo))
+ item_relids = ((RelAggInfo *) item)->relids;
if (bms_equal(item_relids, relids))
return item;
@@ -463,7 +572,7 @@ find_join_rel(PlannerInfo *root, Relids relids)
static void
add_rel_info(RelInfoList *list, void *data)
{
- Assert(IsA(data, RelOptInfo));
+ Assert(IsA(data, RelOptInfo) ||IsA(data, RelAggInfo));
/* GEQO requires us to append the new joinrel to the end of the list! */
list->items = lappend(list->items, data);
@@ -475,7 +584,11 @@ add_rel_info(RelInfoList *list, void *data)
RelInfoEntry *hentry;
bool found;
- relids = ((RelOptInfo *) data)->relids;
+ if (IsA(data, RelOptInfo))
+ relids = ((RelOptInfo *) data)->relids;
+ else if (IsA(data, RelAggInfo))
+ relids = ((RelAggInfo *) data)->relids;
+
hentry = (RelInfoEntry *) hash_search(list->hash,
&relids,
HASH_ENTER,
@@ -497,6 +610,57 @@ add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
}
/*
+ * add_grouped_rel
+ * Add grouped base or join relation to the list of grouped relations in
+ * the given PlannerInfo. Also add the corresponding RelAggInfo to
+ * agg_info_list.
+ */
+void
+add_grouped_rel(PlannerInfo *root, RelOptInfo *rel, RelAggInfo *agg_info)
+{
+ add_rel_info(root->grouped_rel_list, rel);
+ add_rel_info(root->agg_info_list, rel);
+}
+
+/*
+ * find_grouped_rel
+ * Returns grouped relation entry (base or join relation) corresponding to
+ * 'relids' or NULL if none exists.
+ *
+ * If agg_info_p is a valid pointer, then pointer to RelAggInfo that
+ * corresponds to the relation returned is assigned to *agg_info_p.
+ */
+RelOptInfo *
+find_grouped_rel(PlannerInfo *root, Relids relids, RelAggInfo **agg_info_p)
+{
+ RelOptInfo *rel;
+
+ rel = (RelOptInfo *) find_rel_info(root->grouped_rel_list, relids);
+ if (rel == NULL)
+ {
+ if (agg_info_p)
+ *agg_info_p = NULL;
+
+ return NULL;
+ }
+
+ /* Is caller interested in RelAggInfo? */
+ if (agg_info_p)
+ {
+ RelAggInfo *agg_info;
+
+ agg_info = (RelAggInfo *) find_rel_info(root->agg_info_list, relids);
+
+ /* The relation exists, so the agg_info should be there too. */
+ Assert(agg_info != NULL);
+
+ *agg_info_p = agg_info;
+ }
+
+ return rel;
+}
+
+/*
* set_foreign_rel_properties
* Set up foreign-join fields if outer and inner relation are foreign
* tables (or joins) belonging to the same server and assigned to the same
@@ -558,6 +722,7 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* 'restrictlist_ptr': result variable. If not NULL, *restrictlist_ptr
* receives the list of RestrictInfo nodes that apply to this
* particular pair of joinable relations.
+ * 'agg_info' indicates that grouped join relation should be created.
*
* restrictlist_ptr makes the routine's API a little grotty, but it saves
* duplicated calculation of the restrictlist...
@@ -568,10 +733,12 @@ build_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr)
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info)
{
RelOptInfo *joinrel;
List *restrictlist;
+ bool grouped = agg_info != NULL;
/* This function should be used only for join between parents. */
Assert(!IS_OTHER_REL(outer_rel) && !IS_OTHER_REL(inner_rel));
@@ -579,7 +746,8 @@ build_join_rel(PlannerInfo *root,
/*
* See if we already have a joinrel for this set of base rels.
*/
- joinrel = find_join_rel(root, joinrelids);
+ joinrel = !grouped ? find_join_rel(root, joinrelids) :
+ find_grouped_rel(root, joinrelids, NULL);
if (joinrel)
{
@@ -671,9 +839,21 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ if (!grouped)
+ {
+ joinrel->reltarget = create_empty_pathtarget();
+ build_joinrel_tlist(root, joinrel, outer_rel);
+ build_joinrel_tlist(root, joinrel, inner_rel);
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ }
+ else
+ {
+ /*
+ * The target for grouped join should already have its cost and width
+ * computed, see create_rel_agg_info().
+ */
+ joinrel->reltarget = agg_info->target;
+ }
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -705,49 +885,73 @@ build_join_rel(PlannerInfo *root,
joinrel->has_eclass_joins = has_relevant_eclass_joinclause(root, joinrel);
/* Store the partition information. */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- sjinfo->jointype);
-
- /*
- * Set estimates of the joinrel's size.
- */
- set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
- sjinfo, restrictlist);
-
- /*
- * Set the consider_parallel flag if this joinrel could potentially be
- * scanned within a parallel worker. If this flag is false for either
- * inner_rel or outer_rel, then it must be false for the joinrel also.
- * Even if both are true, there might be parallel-restricted expressions
- * in the targetlist or quals.
- *
- * Note that if there are more than two rels in this relation, they could
- * be divided between inner_rel and outer_rel in any arbitrary way. We
- * assume this doesn't matter, because we should hit all the same baserels
- * and joinclauses while building up to this joinrel no matter which we
- * take; therefore, we should make the same decision here however we get
- * here.
- */
- if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
- is_parallel_safe(root, (Node *) restrictlist) &&
- is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
- joinrel->consider_parallel = true;
+ if (!grouped)
+ build_joinrel_partition_info(joinrel, outer_rel, inner_rel,
+ restrictlist, sjinfo->jointype);
/* Add the joinrel to the PlannerInfo. */
- add_join_rel(root, joinrel);
+ if (!grouped)
+ add_join_rel(root, joinrel);
+ else
+ add_grouped_rel(root, joinrel, agg_info);
/*
- * Also, if dynamic-programming join search is active, add the new joinrel
- * to the appropriate sublist. Note: you might think the Assert on number
- * of members should be for equality, but some of the level 1 rels might
- * have been joinrels already, so we can only assert <=.
+ * Also, if dynamic-programming join search is active, add the new
+ * joinrelset to the appropriate sublist. Note: you might think the
+ * Assert on number of members should be for equality, but some of the
+ * level 1 rels might have been joinrels already, so we can only assert
+ * <=.
+ *
+ * Do noting for grouped relation as it's stored aside from
+ * join_rel_level.
*/
- if (root->join_rel_level)
+ if (root->join_rel_level && !grouped)
{
Assert(root->join_cur_level > 0);
- Assert(root->join_cur_level <= bms_num_members(joinrel->relids));
+ Assert(root->join_cur_level <= bms_num_members(joinrelids));
root->join_rel_level[root->join_cur_level] =
- lappend(root->join_rel_level[root->join_cur_level], joinrel);
+ lappend(root->join_rel_level[root->join_cur_level],
+ joinrel);
+ }
+
+ /* Set estimates of the joinrel's size. */
+ if (!grouped)
+ {
+ set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
+ sjinfo, restrictlist);
+
+ /*
+ * Set the consider_parallel flag if this joinrel could potentially be
+ * scanned within a parallel worker. If this flag is false for either
+ * inner_rel or outer_rel, then it must be false for the joinrel also.
+ * Even if both are true, there might be parallel-restricted
+ * expressions in the targetlist or quals.
+ *
+ * Note that if there are more than two rels in this relation, they
+ * could be divided between inner_rel and outer_rel in any arbitrary
+ * way. We assume this doesn't matter, because we should hit all the
+ * same baserels and joinclauses while building up to this joinrel no
+ * matter which we take; therefore, we should make the same decision
+ * here however we get here.
+ */
+ if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
+ is_parallel_safe(root, (Node *) restrictlist) &&
+ is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
+ joinrel->consider_parallel = true;
+ }
+ else
+ {
+ /*
+ * Grouping essentially changes the number of rows.
+ *
+ * XXX We do not distinguish whether two plain rels are joined and the
+ * result is aggregated, or the aggregation has been already applied
+ * to one of the input rels. Is this worth extra effort, e.g.
+ * maintaining a separate RelOptInfo for each case (one difficulty
+ * that would introduce is construction of AppendPath)?
+ */
+ joinrel->rows = estimate_num_groups(root, agg_info->group_exprs,
+ agg_info->input_rows, NULL);
}
return joinrel;
@@ -1791,3 +1995,625 @@ build_child_join_reltarget(PlannerInfo *root,
childrel->reltarget->cost.per_tuple = parentrel->reltarget->cost.per_tuple;
childrel->reltarget->width = parentrel->reltarget->width;
}
+
+/*
+ * Check if the relation can produce grouped paths and return the information
+ * it'll need for it. The passed relation is the non-grouped one which has the
+ * reltarget already constructed.
+ */
+RelAggInfo *
+create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel)
+{
+ List *gvis;
+ List *aggregates = NIL;
+ bool found_other_rel_agg;
+ ListCell *lc;
+ RelAggInfo *result;
+ PathTarget *agg_input;
+ PathTarget *target = NULL;
+ List *grp_exprs_extra = NIL;
+ List *group_clauses_final;
+ int i;
+
+ /*
+ * The function shouldn't have been called if there's no opportunity for
+ * aggregation push-down.
+ */
+ Assert(root->grouped_var_list != NIL);
+
+ result = makeNode(RelAggInfo);
+
+ /*
+ * The current implementation of aggregation push-down cannot handle
+ * PlaceHolderVar (PHV).
+ *
+ * If we knew that the PHV should be evaluated in this target (and of
+ * course, if its expression matched some Aggref argument), we'd just let
+ * init_grouping_targets add that Aggref. On the other hand, if we knew
+ * that the PHV is evaluated below the current rel, we could ignore it
+ * because the referencing Aggref would take care of propagation of the
+ * value to upper joins.
+ *
+ * The problem is that the same PHV can be evaluated in the target of the
+ * current rel or in that of lower rel --- depending on the input paths.
+ * For example, consider rel->relids = {A, B, C} and if ph_eval_at = {B,
+ * C}. Path "A JOIN (B JOIN C)" implies that the PHV is evaluated by the
+ * "(B JOIN C)", while path "(A JOIN B) JOIN C" evaluates the PHV itself.
+ */
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *expr = lfirst(lc);
+
+ if (IsA(expr, PlaceHolderVar))
+ return NULL;
+ }
+
+ if (IS_SIMPLE_REL(rel))
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return NULL;
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the Agg plan
+ * would receive different input at the base rel level.
+ *
+ * XXX For RELOPT_JOINREL, do not return if all the joins that can set any
+ * entry of the grouped target (do we need to postpone this check until
+ * the grouped target is available, and init_grouping_targets take care?)
+ * of this rel to NULL are provably below rel. (It's ok if rel is one of
+ * these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return NULL;
+
+ /*
+ * Use equivalence classes to generate additional grouping expressions for
+ * the current rel. Without these we might not be able to apply
+ * aggregation to the relation result set.
+ *
+ * It's important that create_grouping_expr_grouped_var_infos has
+ * processed the explicit grouping columns by now. If the grouping clause
+ * contains multiple expressions belonging to the same EC, the original
+ * (i.e. not derived) one should be preferred when we build grouping
+ * target for a relation. Otherwise we have a problem when trying to match
+ * target entries to grouping clauses during plan creation, see
+ * get_grouping_expression().
+ */
+ gvis = list_copy(root->grouped_var_list);
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+ int relid = -1;
+
+ /* Only interested in grouping expressions. */
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ while ((relid = bms_next_member(rel->relids, relid)) >= 0)
+ {
+ GroupedVarInfo *gvi_trans;
+
+ gvi_trans = translate_expression_to_rels(root, gvi, relid);
+ if (gvi_trans != NULL)
+ gvis = lappend(gvis, gvi_trans);
+ }
+ }
+
+ /*
+ * Check if some aggregates or grouping expressions can be evaluated in
+ * this relation's target, and collect all vars referenced by these
+ * aggregates / grouping expressions;
+ */
+ found_other_rel_agg = false;
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which includes
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * init_grouping_targets will handle plain Var grouping
+ * expressions because it needs to look them up in
+ * grouped_var_list anyway.
+ */
+ if (IsA(gvi->gvexpr, Var))
+ continue;
+
+ /*
+ * Currently, GroupedVarInfo only handles Vars and Aggrefs.
+ */
+ Assert(IsA(gvi->gvexpr, Aggref));
+
+ /* We only derive grouped expressions using ECs, not aggregates */
+ Assert(!gvi->derived);
+
+ gvi->agg_partial = (Aggref *) copyObject(gvi->gvexpr);
+ mark_partial_aggref(gvi->agg_partial, AGGSPLIT_INITIAL_SERIAL);
+
+ /*
+ * Accept the aggregate.
+ */
+ aggregates = lappend(aggregates, gvi);
+ }
+ else if (IsA(gvi->gvexpr, Aggref))
+ {
+ /*
+ * Remember that there is at least one aggregate expression that
+ * needs something else than this rel.
+ */
+ found_other_rel_agg = true;
+
+ /*
+ * This condition effectively terminates creation of the
+ * RelAggInfo, so there's no reason to check the next
+ * GroupedVarInfo.
+ */
+ break;
+ }
+ }
+
+ /*
+ * Grouping makes little sense w/o aggregate function and w/o grouping
+ * expressions.
+ */
+ if (aggregates == NIL)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Give up if some other aggregate(s) need relations other than the
+ * current one.
+ *
+ * If the aggregate needs the current rel plus anything else, then the
+ * problem is that grouping of the current relation could make some input
+ * variables unavailable for the "higher aggregate", and it'd also
+ * decrease the number of input rows the "higher aggregate" receives.
+ *
+ * If the aggregate does not even need the current rel, then neither the
+ * current rel nor anything else should be grouped because we do not
+ * support join of two grouped relations.
+ */
+ if (found_other_rel_agg)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Create target for grouped paths as well as one for the input paths of
+ * the aggregation paths.
+ */
+ target = create_empty_pathtarget();
+ agg_input = create_empty_pathtarget();
+
+ /*
+ * Cannot suitable targets for the aggregation push-down be derived?
+ */
+ if (!init_grouping_targets(root, rel, target, agg_input, gvis,
+ &grp_exprs_extra))
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ list_free(gvis);
+
+ /*
+ * Aggregation push-down makes no sense w/o grouping expressions.
+ */
+ if ((list_length(target->exprs) + list_length(grp_exprs_extra)) == 0)
+ return NULL;
+
+ group_clauses_final = root->parse->groupClause;
+
+ /*
+ * If the aggregation target should have extra grouping expressions (in
+ * order to emit input vars for join conditions), add them now. This step
+ * includes assignment of tleSortGroupRef's which we can generate now.
+ */
+ if (list_length(grp_exprs_extra) > 0)
+ {
+ Index sortgroupref;
+
+ /*
+ * We'll have to add some clauses, but query group clause must be
+ * preserved.
+ */
+ group_clauses_final = list_copy(group_clauses_final);
+
+ /*
+ * Always start at root->max_sortgroupref. The extra grouping
+ * expressions aren't used during the final aggregation, so the
+ * sortgroupref values don't need to be unique across the query. Thus
+ * we don't have to increase root->max_sortgroupref, which makes
+ * recognition of the extra grouping expressions pretty easy.
+ */
+ sortgroupref = root->max_sortgroupref;
+
+ /*
+ * Generate the SortGroupClause's and add the expressions to the
+ * target.
+ */
+ foreach(lc, grp_exprs_extra)
+ {
+ Var *var = lfirst_node(Var, lc);
+ SortGroupClause *cl = makeNode(SortGroupClause);
+
+ /*
+ * Initialize the SortGroupClause.
+ *
+ * As the final aggregation will not use this grouping expression,
+ * we don't care whether sortop is < or >. The value of
+ * nulls_first should not matter for the same reason.
+ */
+ cl->tleSortGroupRef = ++sortgroupref;
+ get_sort_group_operators(var->vartype,
+ false, true, false,
+ &cl->sortop, &cl->eqop, NULL,
+ &cl->hashable);
+ group_clauses_final = lappend(group_clauses_final, cl);
+ add_column_to_pathtarget(target, (Expr *) var,
+ cl->tleSortGroupRef);
+
+ /*
+ * The aggregation input target must emit this var too.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) var,
+ cl->tleSortGroupRef);
+ }
+ }
+
+ /*
+ * Add aggregates to the grouping target.
+ */
+ add_aggregates_to_target(root, target, aggregates);
+
+ /*
+ * Build a list of grouping expressions and a list of the corresponding
+ * SortGroupClauses.
+ */
+ i = 0;
+ foreach(lc, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(lc);
+
+ if (IsA(texpr, Aggref))
+ {
+ /*
+ * Once we see Aggref, no grouping expressions should follow.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides being an aggregate, the target expression should have no
+ * other reason then being a column of a relation functionally
+ * dependent on the GROUP BY clause. So it's not actually a grouping
+ * column.
+ */
+ if (sortgroupref == 0)
+ continue;
+
+ /*
+ * group_clause_final contains the "local" clauses, so this search
+ * should succeed.
+ */
+ cl = get_sortgroupref_clause(sortgroupref, group_clauses_final);
+
+ result->group_clauses = list_append_unique(result->group_clauses,
+ cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ result->group_exprs = list_append_unique(result->group_exprs,
+ texpr);
+ }
+
+ /*
+ * Since neither target nor agg_input is supposed to be identical to the
+ * source reltarget, compute the width and cost again.
+ *
+ * target does not yet contain aggregates, but these will be accounted by
+ * AggPath.
+ */
+ set_pathtarget_cost_width(root, target);
+ set_pathtarget_cost_width(root, agg_input);
+
+ result->relids = bms_copy(rel->relids);
+ result->target = target;
+ result->agg_input = agg_input;
+
+ /* Finally collect the aggregates. */
+ while (lc != NULL)
+ {
+ Aggref *aggref = lfirst_node(Aggref, lc);
+
+ /*
+ * Partial aggregation is what the grouped paths should do.
+ */
+ result->agg_exprs = lappend(result->agg_exprs, aggref);
+ lc = lnext(lc);
+ }
+
+ /* The "input_rows" field should be set by caller. */
+ return result;
+}
+
+/*
+ * Initialize target for grouped paths (target) as well as a target for paths
+ * that generate input for aggregation (agg_input).
+ *
+ * group_exprs_extra_p receives a list of Var nodes for which we need to
+ * construct SortGroupClause. Those vars will then be used as additional
+ * grouping expressions, for the sake of join clauses.
+ *
+ * gvis a list of GroupedVarInfo's possibly useful for rel.
+ *
+ * Return true iff the targets could be initialized.
+ */
+static bool
+init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p)
+{
+ ListCell *lc1,
+ *lc2;
+ List *unresolved = NIL;
+ List *unresolved_sortgrouprefs = NIL;
+
+ foreach(lc1, rel->reltarget->exprs)
+ {
+ Var *tvar;
+ bool is_grouping;
+ Index sortgroupref = 0;
+ bool derived = false;
+ bool needed_by_aggregate;
+
+ /*
+ * Given that PlaceHolderVar currently prevents us from doing
+ * aggregation push-down, the source target cannot contain anything
+ * more complex than a Var.
+ */
+ tvar = lfirst_node(Var, lc1);
+
+ is_grouping = is_grouping_expression(gvis, (Expr *) tvar,
+ &sortgroupref, &derived);
+
+ /*
+ * Derived grouping expressions should not be referenced by the query
+ * targetlist, so let them fall into vars_unresolved. It'll be checked
+ * later if the current targetlist needs them. For example, we should
+ * not automatically use Var as a grouping expression if the only
+ * reason for it to be in the plain relation target is that it's
+ * referenced by aggregate argument, and it happens to be in the same
+ * EC as any grouping expression.
+ */
+ if (is_grouping && !derived)
+ {
+ Assert(sortgroupref > 0);
+
+ /*
+ * It's o.k. to use the target expression for grouping.
+ */
+ add_column_to_pathtarget(target, (Expr *) tvar, sortgroupref);
+
+ /*
+ * As for agg_input, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) tvar, sortgroupref);
+
+ /* Process the next expression. */
+ continue;
+ }
+
+ /*
+ * Is this Var needed in the query targetlist for anything else than
+ * aggregate input?
+ */
+ needed_by_aggregate = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc2);
+ ListCell *lc3;
+ List *vars;
+
+ if (!IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (!bms_is_member(tvar->varno, gvi->gv_eval_at))
+ continue;
+
+ /*
+ * XXX Consider some sort of caching.
+ */
+ vars = pull_var_clause((Node *) gvi->gvexpr, PVC_RECURSE_AGGREGATES);
+ foreach(lc3, vars)
+ {
+ Var *var = lfirst_node(Var, lc3);
+
+ if (equal(var, tvar))
+ {
+ needed_by_aggregate = true;
+ break;
+ }
+ }
+ list_free(vars);
+ if (needed_by_aggregate)
+ break;
+ }
+
+ if (needed_by_aggregate)
+ {
+ bool found = false;
+
+ foreach(lc2, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc2);
+
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ if (equal(te->expr, tvar))
+ {
+ found = true;
+ break;
+ }
+ }
+
+ /*
+ * If it's only Aggref input, add it to the aggregation input
+ * target and that's it.
+ */
+ if (!found)
+ {
+ add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
+ continue;
+ }
+ }
+
+ /*
+ * Further investigation involves dependency check, for which we need
+ * to have all the (plain-var) grouping expressions gathered.
+ */
+ unresolved = lappend(unresolved, tvar);
+ unresolved_sortgrouprefs = lappend_int(unresolved_sortgrouprefs,
+ sortgroupref);
+ }
+
+ /*
+ * Check for other possible reasons for the var to be in the plain target.
+ */
+ forboth(lc1, unresolved, lc2, unresolved_sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, lc1);
+ Index sortgroupref = lfirst_int(lc2);
+ RangeTblEntry *rte;
+ List *deps = NIL;
+ Relids relids_subtract;
+ int ndx;
+ RelOptInfo *baserel;
+
+ rte = root->simple_rte_array[var->varno];
+
+ /*
+ * Check if the Var can be in the grouping key even though it's not
+ * mentioned by the GROUP BY clause (and could not be derived using
+ * ECs).
+ */
+ if (sortgroupref == 0 &&
+ check_functional_grouping(rte->relid, var->varno,
+ var->varlevelsup,
+ target->exprs, &deps))
+ {
+ /*
+ * The var shouldn't be actually used as a grouping key (instead,
+ * the one this depends on will be), so sortgroupref should not be
+ * important.
+ */
+ add_new_column_to_pathtarget(target, (Expr *) var);
+ add_new_column_to_pathtarget(agg_input, (Expr *) var);
+
+ /*
+ * The var may or may not be present in generic grouping
+ * expression(s) in addition, but this is handled elsewhere.
+ */
+ continue;
+ }
+
+ /*
+ * Isn't the expression needed by joins above the current rel?
+ *
+ * The relids we're not interested in do include 0, which is the
+ * top-level targetlist. The only reason for relids to contain 0
+ * should be that arg_var is referenced either by aggregate or by
+ * grouping expression, but right now we're interested in the *other*
+ * reasons. (As soon aggregation is pushed down, the aggregates in the
+ * query targetlist no longer need direct reference to arg_var
+ * anyway.)
+ */
+ relids_subtract = bms_copy(rel->relids);
+ bms_add_member(relids_subtract, 0);
+
+ baserel = find_base_rel(root, var->varno);
+ ndx = var->varattno - baserel->min_attr;
+ if (bms_nonempty_difference(baserel->attr_needed[ndx],
+ relids_subtract))
+ {
+ /*
+ * The variable is needed by a join involving this relation. That
+ * case includes variable that is referenced by a generic grouping
+ * expression.
+ *
+ * The only way to bring this var to the aggregation output is to
+ * add it to the grouping expressions too.
+ */
+ if (sortgroupref > 0)
+ {
+ /*
+ * The var could be recognized as a potentially useful
+ * grouping expression at the top of the loop, so we can add
+ * it to the grouping target, as well as to the agg_input.
+ */
+ add_column_to_pathtarget(target, (Expr *) var, sortgroupref);
+ add_column_to_pathtarget(agg_input, (Expr *) var, sortgroupref);
+ }
+ else
+ {
+ /*
+ * Since root->parse->groupClause is not supposed to contain
+ * this expression, we need to construct special
+ * SortGroupClause. Its tleSortGroupRef needs to be unique
+ * within target_agg, so postpone creation of the
+ * SortGroupRefs until we're done with the iteration of
+ * rel->reltarget->exprs.
+ */
+ *group_exprs_extra_p = lappend(*group_exprs_extra_p, var);
+ }
+ }
+ else
+ {
+ /*
+ * As long as the query is semantically correct, arriving here
+ * means that the var is referenced by a generic grouping
+ * expression but not referenced by any join.
+ *
+ * create_rel_agg_info() should add this variable to "agg_input"
+ * target and also add the whole generic expression to "target",
+ * but that's subject to future enhancement of the aggregate
+ * push-down feature.
+ */
+ return false;
+ }
+ }
+
+ return true;
+}
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
index 14d1c67a94..3b81b39c8e 100644
--- a/src/backend/optimizer/util/tlist.c
+++ b/src/backend/optimizer/util/tlist.c
@@ -802,6 +802,62 @@ apply_pathtarget_labeling_to_tlist(List *tlist, PathTarget *target)
}
/*
+ * For each aggregate grouping expression or aggregate to the grouped target.
+ *
+ * Caller passes the expressions in the form of GroupedVarInfos so that we
+ * don't have to look for gvid.
+ */
+void
+add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions)
+{
+ ListCell *lc;
+
+ /* Create the vars and add them to the target. */
+ foreach(lc, expressions)
+ {
+ GroupedVarInfo *gvi;
+
+ gvi = lfirst_node(GroupedVarInfo, lc);
+ add_column_to_pathtarget(target, (Expr *) gvi->agg_partial,
+ gvi->sortgroupref);
+ }
+}
+
+/*
+ * Find out if expr can be used as grouping expression in reltarget.
+ *
+ * sortgroupref and is_derived reflect the ->sortgroupref and ->derived fields
+ * of the corresponding GroupedVarInfo.
+ */
+bool
+is_grouping_expression(List *gvis, Expr *expr, Index *sortgroupref,
+ bool *is_derived)
+{
+ ListCell *lc;
+
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (equal(gvi->gvexpr, expr))
+ {
+ Assert(gvi->sortgroupref > 0);
+
+ *sortgroupref = gvi->sortgroupref;
+ *is_derived = gvi->derived;
+ return true;
+ }
+ }
+
+ /* The expression cannot be used as grouping key. */
+ return false;
+}
+
+/*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
*
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 62b34f1b8e..d4e63a54f4 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -1008,6 +1008,15 @@ static struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
{
+ {"enable_agg_pushdown", PGC_USERSET, QUERY_TUNING_METHOD,
+ gettext_noop("Enables aggregation push-down."),
+ NULL
+ },
+ &enable_agg_pushdown,
+ false,
+ NULL, NULL, NULL
+ },
+ {
{"enable_parallel_append", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables the planner's use of parallel append plans."),
NULL
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 8e34496764..8c15181e78 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -224,6 +224,7 @@ typedef enum NodeTag
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_RelAggInfo,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
@@ -268,6 +269,7 @@ typedef enum NodeTag
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 88a2c20fbe..24c5bfafae 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -239,6 +239,23 @@ struct PlannerInfo
struct RelInfoList *join_rel_list; /* list of join-relation RelOptInfos */
/*
+ * grouped_rel_list is a list of RelOptInfos that represent grouped
+ * relations, both base relations and joins. Unlike join_rel_list, base
+ * relations are accepted because grouped base relation is of rather
+ * limited scope, i.e. it's only needed during join search. Thus it does
+ * not deserve separate storage like simple_rel_array.
+ */
+ struct RelInfoList *grouped_rel_list; /* list of grouped relation
+ * RelOptInfos */
+
+ /*
+ * agg_info_list contains one instance of RelAggInfo per an item of
+ * grouped_rel_list.
+ */
+ struct RelInfoList *agg_info_list; /* list of grouped relation
+ * RelAggInfos */
+
+ /*
* When doing a dynamic-programming-style join search, join_rel_level[k]
* is a list of all join-relation RelOptInfos of level k, and
* join_cur_level is the current level. New join-relation RelOptInfos are
@@ -278,6 +295,8 @@ struct PlannerInfo
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
@@ -304,6 +323,12 @@ struct PlannerInfo
*/
List *processed_tlist;
+ /*
+ * The maximum ressortgroupref among target entries in processed_list.
+ * Useful when adding extra grouping expressions for partial aggregation.
+ */
+ int max_sortgroupref;
+
/* Fields filled during create_plan() for use in setrefs.c */
AttrNumber *grouping_map; /* for GroupingFunc fixup */
List *minmax_aggs; /* List of MinMaxAggInfos */
@@ -746,6 +771,60 @@ typedef struct RelInfoList
} RelInfoList;
/*
+ * RelAggInfo
+ * Information needed to create grouped paths for base rels and joins.
+ *
+ * "relids" is the set of base-relation identifiers, just like with
+ * RelOptInfo.
+ *
+ * "target" will be used as pathtarget if partial aggregation is applied to
+ * base relation or join. The same target will also --- if the relation is a
+ * join --- be used to joinin grouped path to a non-grouped one. This target
+ * can contain plain-Var grouping expressions and Aggref nodes.
+ *
+ * Note: There's a convention that Aggref expressions are supposed to follow
+ * the other expressions of the target. Iterations of ->exprs may rely on this
+ * arrangement.
+ *
+ * "agg_input" contains Vars used either as grouping expressions or aggregate
+ * arguments. Paths providing the aggregation plan with input data should use
+ * this target. The only difference from reltarget of the non-grouped relation
+ * is that some items can have sortgroupref initialized.
+ *
+ * "input_rows" is the estimated number of input rows for AggPath. It's
+ * actually just a workspace for users of the structure, i.e. not initialized
+ * when instance of the structure is created.
+ *
+ * "group_clauses" and "group_exprs" are lists of SortGroupClause and the
+ * corresponding grouping expressions respectively.
+ *
+ * "agg_exprs" is a list of Aggref nodes for the aggregation of the relation's
+ * paths.
+ *
+ * "rel_grouped" is the relation containing the partially aggregated paths.
+ */
+typedef struct RelAggInfo
+{
+ NodeTag type;
+
+ Relids relids; /* Base rels contained in this grouped rel. */
+
+ struct PathTarget *target; /* Target for grouped paths. */
+
+ struct PathTarget *agg_input; /* pathtarget of paths that generate input
+ * for aggregation paths. */
+
+ double input_rows;
+
+ List *group_clauses;
+ List *group_exprs;
+
+ List *agg_exprs; /* Aggref expressions. */
+
+ RelOptInfo *rel_grouped; /* Grouped relation. */
+} RelAggInfo;
+
+/*
* IndexOptInfo
* Per-index information for planning/optimization
*
@@ -2259,6 +2338,26 @@ typedef struct PlaceHolderInfo
} PlaceHolderInfo;
/*
+ * GroupedVarInfo exists for each expression that can be used as an aggregate
+ * or grouping expression evaluated below a join.
+ */
+typedef struct GroupedVarInfo
+{
+ NodeTag type;
+
+ Expr *gvexpr; /* the represented expression. */
+ Aggref *agg_partial; /* if gvexpr is aggregate, agg_partial is the
+ * corresponding partial aggregate */
+ Index sortgroupref; /* If gvexpr is a grouping expression, this is
+ * the tleSortGroupRef of the corresponding
+ * SortGroupClause. */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+ bool derived; /* derived from another GroupedVarInfo using
+ * equeivalence classes? */
+} GroupedVarInfo;
+
+/*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
* path, the list is stored into root->minmax_aggs for use during setrefs.c.
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
index 5e10fb1d50..db838a3581 100644
--- a/src/include/optimizer/clauses.h
+++ b/src/include/optimizer/clauses.h
@@ -55,4 +55,6 @@ extern void CommuteOpExpr(OpExpr *clause);
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
+extern GroupedVarInfo *translate_expression_to_rels(PlannerInfo *root,
+ GroupedVarInfo *gvi, Index relid);
#endif /* CLAUSES_H */
diff --git a/src/include/optimizer/cost.h b/src/include/optimizer/cost.h
index ac6de0f6be..0871e8a516 100644
--- a/src/include/optimizer/cost.h
+++ b/src/include/optimizer/cost.h
@@ -64,6 +64,7 @@ extern PGDLLIMPORT bool enable_partitionwise_aggregate;
extern PGDLLIMPORT bool enable_parallel_append;
extern PGDLLIMPORT bool enable_parallel_hash;
extern PGDLLIMPORT bool enable_partition_pruning;
+extern PGDLLIMPORT bool enable_agg_pushdown;
extern PGDLLIMPORT int constraint_exclusion;
extern double index_pages_fetched(double tuples_fetched, BlockNumber pages,
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 574bb85b50..973ac5f7d5 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -212,6 +212,14 @@ extern AggPath *create_agg_path(PlannerInfo *root,
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+extern AggPath *create_agg_sorted_path(PlannerInfo *root,
+ RelOptInfo *rel,
+ Path *subpath,
+ RelAggInfo *agg_info);
+extern AggPath *create_agg_hashed_path(PlannerInfo *root,
+ RelOptInfo *rel,
+ Path *subpath,
+ RelAggInfo *agg_info);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
@@ -279,14 +287,21 @@ extern void setup_simple_rel_arrays(PlannerInfo *root);
extern void setup_append_rel_array(PlannerInfo *root);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
+extern RelOptInfo *build_simple_grouped_rel(PlannerInfo *root, int relid,
+ RelAggInfo **agg_info_p);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
+extern void add_grouped_rel(PlannerInfo *root, RelOptInfo *rel,
+ RelAggInfo *agg_info);
+extern RelOptInfo *find_grouped_rel(PlannerInfo *root, Relids relids,
+ RelAggInfo **agg_info_p);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr);
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info);
extern Relids min_join_parameterization(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
@@ -312,5 +327,5 @@ extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo, JoinType jointype);
-
+extern RelAggInfo *create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index 040335a7c5..90d2d52660 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -21,6 +21,7 @@
* allpaths.c
*/
extern PGDLLIMPORT bool enable_geqo;
+extern PGDLLIMPORT bool enable_agg_pushdown;
extern PGDLLIMPORT int geqo_threshold;
extern PGDLLIMPORT int min_parallel_table_scan_size;
extern PGDLLIMPORT int min_parallel_index_scan_size;
@@ -55,6 +56,11 @@ extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
+extern void generate_grouping_paths(PlannerInfo *root,
+ RelOptInfo *rel_grouped,
+ RelOptInfo *rel_plain,
+ RelAggInfo *agg_info);
+
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages, int max_workers);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index 3bbdb5e2f7..1aa8570d0d 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -28,7 +28,8 @@ typedef void (*query_pathkeys_callback) (PlannerInfo *root, void *extra);
* prototypes for plan/planmain.c
*/
extern RelOptInfo *query_planner(PlannerInfo *root, List *tlist,
- query_pathkeys_callback qp_callback, void *qp_extra);
+ query_pathkeys_callback qp_callback, void *qp_extra,
+ RelOptInfo **final_rel_grouped_p);
/*
* prototypes for plan/planagg.c
@@ -68,6 +69,7 @@ extern void add_base_rels_to_query(PlannerInfo *root, Node *jtnode);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
Relids where_needed, bool create_new_ph);
+extern void setup_aggregate_pushdown(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
index 58db79203b..fa927cd6f4 100644
--- a/src/include/optimizer/tlist.h
+++ b/src/include/optimizer/tlist.h
@@ -49,6 +49,13 @@ extern void split_pathtarget_at_srfs(PlannerInfo *root,
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+/* TODO Find the best location (position and in some cases even file) for the
+ * following ones. */
+extern void add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions);
+extern bool is_grouping_expression(List *gvis, Expr *expr,
+ Index *sortgroupref, bool *is_derived);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
new file mode 100644
index 0000000000..786f4f3cff
--- /dev/null
+++ b/src/test/regress/expected/agg_pushdown.out
@@ -0,0 +1,214 @@
+BEGIN;
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+CREATE INDEX ON agg_pushdown_child1(parent);
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+COMMIT;
+ANALYZE;
+SET enable_agg_pushdown TO on;
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ Index Cond: (parent = i)
+ -> Index Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(9 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Hash Join
+ Hash Cond: (c1.parent = p.i)
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ Index Cond: (parent = i)
+ -> Hash
+ -> Seq Scan on agg_pushdown_parent p
+(10 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ Index Cond: (parent = i)
+ -> Sort
+ Sort Key: p.i
+ -> Seq Scan on agg_pushdown_parent p
+(11 rows)
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ Index Cond: (parent = i)
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(9 rows)
+
+SET enable_seqscan TO on;
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Hash Join
+ Hash Cond: (p.x = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ Index Cond: (parent = x)
+(10 rows)
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: ((k = c1.j) AND (parent = c1.parent))
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(13 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+----------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Hash
+ -> Seq Scan on agg_pushdown_child2 c2
+(15 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: ((c1.j = c2.k) AND (c1.parent = c2.parent))
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(13 rows)
+
diff --git a/src/test/regress/expected/sysviews.out b/src/test/regress/expected/sysviews.out
index a1c90eb905..55cfd3fff7 100644
--- a/src/test/regress/expected/sysviews.out
+++ b/src/test/regress/expected/sysviews.out
@@ -72,6 +72,7 @@ select count(*) >= 0 as ok from pg_prepared_xacts;
select name, setting from pg_settings where name like 'enable%';
name | setting
--------------------------------+---------
+ enable_agg_pushdown | off
enable_bitmapscan | on
enable_gathermerge | on
enable_hashagg | on
@@ -89,7 +90,7 @@ select name, setting from pg_settings where name like 'enable%';
enable_seqscan | on
enable_sort | on
enable_tidscan | on
-(17 rows)
+(18 rows)
-- Test that the pg_timezone_names and pg_timezone_abbrevs views are
-- more-or-less working. We can't test their contents in any great detail
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 4051a4ad4e..22058b1f91 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -104,6 +104,9 @@ test: rules psql_crosstab amutils
test: select_parallel
test: write_parallel
+# this one runs parallel workers too
+test: agg_pushdown
+
# no relation related tests can be put in this group
test: publication subscription
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index ac1ea622d6..bc1ed68a9e 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -136,6 +136,7 @@ test: rules
test: psql_crosstab
test: select_parallel
test: write_parallel
+test: agg_pushdown
test: publication
test: subscription
test: amutils
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
new file mode 100644
index 0000000000..6cf2ed9c20
--- /dev/null
+++ b/src/test/regress/sql/agg_pushdown.sql
@@ -0,0 +1,117 @@
+BEGIN;
+
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+
+CREATE INDEX ON agg_pushdown_child1(parent);
+
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+COMMIT;
+ANALYZE;
+
+SET enable_agg_pushdown TO on;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_seqscan TO on;
+
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
Re: [HACKERS] WIP: Aggregation push-down
On Fri, Mar 1, 2019 at 12:01 AM Antonin Houska <ah@cybertec.at> wrote:
Tom Lane <tgl@sss.pgh.pa.us> wrote:
Antonin Houska <ah@cybertec.at> writes:
Michael Paquier <michael@paquier.xyz> wrote:
Latest patch set fails to apply, so moved to next CF, waiting on
author.Rebased.
This is in need of rebasing again :-(. I went ahead and pushed the 001
part, since that seemed fairly uncontroversial.ok, thanks.
Another rebase is needed for the patches.
Thanks
Richard
Re: [HACKERS] WIP: Aggregation push-down
Richard Guo <riguo@pivotal.io> wrote:
Another rebase is needed for the patches.
Done.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
Attachments:
v12-0001-Introduce-RelInfoList-structure.patchtext/x-diffDownload
From f656bd8d46afb9cb0a331cf3d34b9eed39f5e360 Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Tue, 9 Jul 2019 15:30:13 +0200
Subject: [PATCH 1/3] Introduce RelInfoList structure.
---
contrib/postgres_fdw/postgres_fdw.c | 3 +-
src/backend/nodes/outfuncs.c | 11 +++
src/backend/optimizer/geqo/geqo_eval.c | 12 +--
src/backend/optimizer/plan/planmain.c | 3 +-
src/backend/optimizer/util/relnode.c | 157 ++++++++++++++++++++-------------
src/include/nodes/nodes.h | 1 +
src/include/nodes/pathnodes.h | 28 ++++--
7 files changed, 136 insertions(+), 79 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 033aeb2556..90414f1168 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -5205,7 +5205,8 @@ foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
*/
Assert(fpinfo->relation_index == 0); /* shouldn't be set yet */
fpinfo->relation_index =
- list_length(root->parse->rtable) + list_length(root->join_rel_list);
+ list_length(root->parse->rtable) +
+ list_length(root->join_rel_list->items);
return true;
}
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8400dd319e..4529b5c63b 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2278,6 +2278,14 @@ _outRelOptInfo(StringInfo str, const RelOptInfo *node)
WRITE_NODE_FIELD(partitioned_child_rels);
}
+static void
+_outRelInfoList(StringInfo str, const RelInfoList *node)
+{
+ WRITE_NODE_TYPE("RELOPTINFOLIST");
+
+ WRITE_NODE_FIELD(items);
+}
+
static void
_outIndexOptInfo(StringInfo str, const IndexOptInfo *node)
{
@@ -4052,6 +4060,9 @@ outNode(StringInfo str, const void *obj)
case T_RelOptInfo:
_outRelOptInfo(str, obj);
break;
+ case T_RelInfoList:
+ _outRelInfoList(str, obj);
+ break;
case T_IndexOptInfo:
_outIndexOptInfo(str, obj);
break;
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index 6c69c1c147..c69f3469ba 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -92,11 +92,11 @@ geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
*
* join_rel_level[] shouldn't be in use, so just Assert it isn't.
*/
- savelength = list_length(root->join_rel_list);
- savehash = root->join_rel_hash;
+ savelength = list_length(root->join_rel_list->items);
+ savehash = root->join_rel_list->hash;
Assert(root->join_rel_level == NULL);
- root->join_rel_hash = NULL;
+ root->join_rel_list->hash = NULL;
/* construct the best path for the given combination of relations */
joinrel = gimme_tree(root, tour, num_gene);
@@ -121,9 +121,9 @@ geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
* Restore join_rel_list to its former state, and put back original
* hashtable if any.
*/
- root->join_rel_list = list_truncate(root->join_rel_list,
- savelength);
- root->join_rel_hash = savehash;
+ root->join_rel_list->items = list_truncate(root->join_rel_list->items,
+ savelength);
+ root->join_rel_list->hash = savehash;
/* release all the memory acquired within gimme_tree */
MemoryContextSwitchTo(oldcxt);
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 2dbf1db844..0b9999c8a6 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -65,8 +65,7 @@ query_planner(PlannerInfo *root,
* NOTE: append_rel_list was set up by subquery_planner, so do not touch
* here.
*/
- root->join_rel_list = NIL;
- root->join_rel_hash = NULL;
+ root->join_rel_list = makeNode(RelInfoList);
root->join_rel_level = NULL;
root->join_cur_level = 0;
root->canon_pathkeys = NIL;
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 6054bd2b53..c238dd6538 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -31,11 +31,11 @@
#include "utils/hsearch.h"
-typedef struct JoinHashEntry
+typedef struct RelInfoEntry
{
- Relids join_relids; /* hash key --- MUST BE FIRST */
- RelOptInfo *join_rel;
-} JoinHashEntry;
+ Relids relids; /* hash key --- MUST BE FIRST */
+ void *data;
+} RelInfoEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *input_rel);
@@ -375,11 +375,11 @@ find_base_rel(PlannerInfo *root, int relid)
}
/*
- * build_join_rel_hash
- * Construct the auxiliary hash table for join relations.
+ * build_rel_hash
+ * Construct the auxiliary hash table for relation specific data.
*/
static void
-build_join_rel_hash(PlannerInfo *root)
+build_rel_hash(RelInfoList *list)
{
HTAB *hashtab;
HASHCTL hash_ctl;
@@ -388,47 +388,50 @@ build_join_rel_hash(PlannerInfo *root)
/* Create the hash table */
MemSet(&hash_ctl, 0, sizeof(hash_ctl));
hash_ctl.keysize = sizeof(Relids);
- hash_ctl.entrysize = sizeof(JoinHashEntry);
+ hash_ctl.entrysize = sizeof(RelInfoEntry);
hash_ctl.hash = bitmap_hash;
hash_ctl.match = bitmap_match;
hash_ctl.hcxt = CurrentMemoryContext;
- hashtab = hash_create("JoinRelHashTable",
+ hashtab = hash_create("RelHashTable",
256L,
&hash_ctl,
HASH_ELEM | HASH_FUNCTION | HASH_COMPARE | HASH_CONTEXT);
/* Insert all the already-existing joinrels */
- foreach(l, root->join_rel_list)
+ foreach(l, list->items)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
- JoinHashEntry *hentry;
+ void *item = lfirst(l);
+ RelInfoEntry *hentry;
bool found;
+ Relids relids;
- hentry = (JoinHashEntry *) hash_search(hashtab,
- &(rel->relids),
- HASH_ENTER,
- &found);
+ Assert(IsA(item, RelOptInfo));
+ relids = ((RelOptInfo *) item)->relids;
+
+ hentry = (RelInfoEntry *) hash_search(hashtab,
+ &relids,
+ HASH_ENTER,
+ &found);
Assert(!found);
- hentry->join_rel = rel;
+ hentry->data = item;
}
- root->join_rel_hash = hashtab;
+ list->hash = hashtab;
}
/*
- * find_join_rel
- * Returns relation entry corresponding to 'relids' (a set of RT indexes),
- * or NULL if none exists. This is for join relations.
+ * find_rel_info
+ * Find a base or join relation entry.
*/
-RelOptInfo *
-find_join_rel(PlannerInfo *root, Relids relids)
+static void *
+find_rel_info(RelInfoList *list, Relids relids)
{
/*
* Switch to using hash lookup when list grows "too long". The threshold
* is arbitrary and is known only here.
*/
- if (!root->join_rel_hash && list_length(root->join_rel_list) > 32)
- build_join_rel_hash(root);
+ if (!list->hash && list_length(list->items) > 32)
+ build_rel_hash(list);
/*
* Use either hashtable lookup or linear search, as appropriate.
@@ -438,34 +441,90 @@ find_join_rel(PlannerInfo *root, Relids relids)
* so would force relids out of a register and thus probably slow down the
* list-search case.
*/
- if (root->join_rel_hash)
+ if (list->hash)
{
Relids hashkey = relids;
- JoinHashEntry *hentry;
+ RelInfoEntry *hentry;
- hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
- &hashkey,
- HASH_FIND,
- NULL);
+ hentry = (RelInfoEntry *) hash_search(list->hash,
+ &hashkey,
+ HASH_FIND,
+ NULL);
if (hentry)
- return hentry->join_rel;
+ return hentry->data;
}
else
{
ListCell *l;
- foreach(l, root->join_rel_list)
+ foreach(l, list->items)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
+ void *item = lfirst(l);
+ Relids item_relids;
- if (bms_equal(rel->relids, relids))
- return rel;
+ Assert(IsA(item, RelOptInfo));
+ item_relids = ((RelOptInfo *) item)->relids;
+
+ if (bms_equal(item_relids, relids))
+ return item;
}
}
return NULL;
}
+/*
+ * find_join_rel
+ * Returns relation entry corresponding to 'relids' (a set of RT indexes),
+ * or NULL if none exists. This is for join relations.
+ */
+RelOptInfo *
+find_join_rel(PlannerInfo *root, Relids relids)
+{
+ return (RelOptInfo *) find_rel_info(root->join_rel_list, relids);
+}
+
+/*
+ * add_rel_info
+ * Add relation specific info to a list, and also add it to the auxiliary
+ * hashtable if there is one.
+ */
+static void
+add_rel_info(RelInfoList *list, void *data)
+{
+ Assert(IsA(data, RelOptInfo));
+
+ /* GEQO requires us to append the new joinrel to the end of the list! */
+ list->items = lappend(list->items, data);
+
+ /* store it into the auxiliary hashtable if there is one. */
+ if (list->hash)
+ {
+ Relids relids;
+ RelInfoEntry *hentry;
+ bool found;
+
+ relids = ((RelOptInfo *) data)->relids;
+ hentry = (RelInfoEntry *) hash_search(list->hash,
+ &relids,
+ HASH_ENTER,
+ &found);
+ Assert(!found);
+ hentry->data = data;
+ }
+}
+
+/*
+ * add_join_rel
+ * Add given join relation to the list of join relations in the given
+ * PlannerInfo.
+ */
+static void
+add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
+{
+ add_rel_info(root->join_rel_list, joinrel);
+}
+
/*
* set_foreign_rel_properties
* Set up foreign-join fields if outer and inner relation are foreign
@@ -516,32 +575,6 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
}
}
-/*
- * add_join_rel
- * Add given join relation to the list of join relations in the given
- * PlannerInfo. Also add it to the auxiliary hashtable if there is one.
- */
-static void
-add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
-{
- /* GEQO requires us to append the new joinrel to the end of the list! */
- root->join_rel_list = lappend(root->join_rel_list, joinrel);
-
- /* store it into the auxiliary hashtable if there is one. */
- if (root->join_rel_hash)
- {
- JoinHashEntry *hentry;
- bool found;
-
- hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
- &(joinrel->relids),
- HASH_ENTER,
- &found);
- Assert(!found);
- hentry->join_rel = joinrel;
- }
-}
-
/*
* build_join_rel
* Returns relation entry corresponding to the union of two given rels,
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 4e2fb39105..11027cdb10 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -220,6 +220,7 @@ typedef enum NodeTag
T_PlannerInfo,
T_PlannerGlobal,
T_RelOptInfo,
+ T_RelInfoList,
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 441e64eca9..38dc186623 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -236,15 +236,9 @@ struct PlannerInfo
/*
* join_rel_list is a list of all join-relation RelOptInfos we have
- * considered in this planning run. For small problems we just scan the
- * list to do lookups, but when there are many join relations we build a
- * hash table for faster lookups. The hash table is present and valid
- * when join_rel_hash is not NULL. Note that we still maintain the list
- * even when using the hash table for lookups; this simplifies life for
- * GEQO.
+ * considered in this planning run.
*/
- List *join_rel_list; /* list of join-relation RelOptInfos */
- struct HTAB *join_rel_hash; /* optional hashtable for join relations */
+ struct RelInfoList *join_rel_list; /* list of join-relation RelOptInfos */
/*
* When doing a dynamic-programming-style join search, join_rel_level[k]
@@ -742,6 +736,24 @@ typedef struct RelOptInfo
((rel)->part_scheme && (rel)->boundinfo && (rel)->nparts > 0 && \
(rel)->part_rels && (rel)->partexprs && (rel)->nullable_partexprs)
+/*
+ * RelInfoList
+ * A list to store relation specific info and to retrieve it by relids.
+ *
+ * For small problems we just scan the list to do lookups, but when there are
+ * many relations we build a hash table for faster lookups. The hash table is
+ * present and valid when rel_hash is not NULL. Note that we still maintain
+ * the list even when using the hash table for lookups; this simplifies life
+ * for GEQO.
+ */
+typedef struct RelInfoList
+{
+ NodeTag type;
+
+ List *items;
+ struct HTAB *hash;
+} RelInfoList;
+
/*
* IndexOptInfo
* Per-index information for planning/optimization
--
2.16.4
v12-0002-Introduce-make_join_rel_common-function.patchtext/x-diffDownload
From 4315b62d840d86cd25a4d73331e1c091c378805f Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Tue, 9 Jul 2019 15:30:13 +0200
Subject: [PATCH 2/3] Introduce make_join_rel_common function.
This diff tries to make the following patch a bit easier to read.
---
src/backend/optimizer/path/joinrels.c | 39 ++++++++++++++++++++++-------------
1 file changed, 25 insertions(+), 14 deletions(-)
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 43c3b7ea48..a9fc52c61f 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -35,6 +35,8 @@ static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
+static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1,
+ RelOptInfo *rel2);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist);
@@ -647,21 +649,12 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
return true;
}
-
/*
- * make_join_rel
- * Find or create a join RelOptInfo that represents the join of
- * the two given rels, and add to it path information for paths
- * created with the two rels as outer and inner rel.
- * (The join rel may already contain paths generated from other
- * pairs of rels that add up to the same set of base rels.)
- *
- * NB: will return NULL if attempted join is not valid. This can happen
- * when working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
- */
-RelOptInfo *
-make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+ * make_join_rel_common
+ * The workhorse of make_join_rel().
+ */
+static RelOptInfo *
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -743,6 +736,24 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
return joinrel;
}
+/*
+ * make_join_rel
+ * Find or create a join RelOptInfo that represents the join of
+ * the two given rels, and add to it path information for paths
+ * created with the two rels as outer and inner rel.
+ * (The join rel may already contain paths generated from other
+ * pairs of rels that add up to the same set of base rels.)
+ *
+ * NB: will return NULL if attempted join is not valid. This can happen when
+ * working with outer joins, or with IN or EXISTS clauses that have been
+ * turned into joins.
+ */
+RelOptInfo *
+make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+{
+ return make_join_rel_common(root, rel1, rel2);
+}
+
/*
* populate_joinrel_with_paths
* Add paths to the given joinrel for given pair of joining relations. The
--
2.16.4
v12-0003-Aggregate-push-down-basic-functionality.patchtext/x-diffDownload
From f66931434c1e55449d568ee038f2abea06ef532d Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Tue, 9 Jul 2019 15:30:13 +0200
Subject: [PATCH 3/3] Aggregate push-down - basic functionality.
---
src/backend/nodes/copyfuncs.c | 21 +-
src/backend/nodes/outfuncs.c | 36 ++
src/backend/optimizer/README | 87 +++
src/backend/optimizer/path/allpaths.c | 112 ++++
src/backend/optimizer/path/costsize.c | 17 +-
src/backend/optimizer/path/equivclass.c | 134 +++++
src/backend/optimizer/path/joinrels.c | 170 +++++-
src/backend/optimizer/plan/initsplan.c | 257 ++++++++
src/backend/optimizer/plan/planagg.c | 2 +-
src/backend/optimizer/plan/planmain.c | 23 +-
src/backend/optimizer/plan/planner.c | 58 +-
src/backend/optimizer/plan/setrefs.c | 33 ++
src/backend/optimizer/prep/prepjointree.c | 1 +
src/backend/optimizer/util/pathnode.c | 140 +++++
src/backend/optimizer/util/relnode.c | 914 +++++++++++++++++++++++++++--
src/backend/optimizer/util/tlist.c | 56 ++
src/backend/utils/misc/guc.c | 9 +
src/include/nodes/nodes.h | 2 +
src/include/nodes/pathnodes.h | 99 ++++
src/include/optimizer/clauses.h | 2 +
src/include/optimizer/pathnode.h | 19 +-
src/include/optimizer/paths.h | 7 +
src/include/optimizer/planmain.h | 4 +-
src/include/optimizer/tlist.h | 8 +-
src/test/regress/expected/agg_pushdown.out | 214 +++++++
src/test/regress/expected/sysviews.out | 3 +-
src/test/regress/parallel_schedule | 3 +
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/agg_pushdown.sql | 117 ++++
29 files changed, 2478 insertions(+), 71 deletions(-)
create mode 100644 src/test/regress/expected/agg_pushdown.out
create mode 100644 src/test/regress/sql/agg_pushdown.sql
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 78deade89b..56f99e2a59 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2204,8 +2204,8 @@ _copyOnConflictExpr(const OnConflictExpr *from)
/* ****************************************************************
* pathnodes.h copy functions
*
- * We don't support copying RelOptInfo, IndexOptInfo, or Path nodes.
- * There are some subsidiary structs that are useful to copy, though.
+ * We don't support copying RelOptInfo, IndexOptInfo, RelAggInfo or Path
+ * nodes. There are some subsidiary structs that are useful to copy, though.
* ****************************************************************
*/
@@ -2346,6 +2346,20 @@ _copyPlaceHolderInfo(const PlaceHolderInfo *from)
return newnode;
}
+static GroupedVarInfo *
+_copyGroupedVarInfo(const GroupedVarInfo *from)
+{
+ GroupedVarInfo *newnode = makeNode(GroupedVarInfo);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gv_eval_at);
+ COPY_SCALAR_FIELD(derived);
+
+ return newnode;
+}
+
/* ****************************************************************
* parsenodes.h copy functions
* ****************************************************************
@@ -5124,6 +5138,9 @@ copyObjectImpl(const void *from)
case T_PlaceHolderInfo:
retval = _copyPlaceHolderInfo(from);
break;
+ case T_GroupedVarInfo:
+ retval = _copyGroupedVarInfo(from);
+ break;
/*
* VALUE NODES
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 4529b5c63b..518aeeefe0 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2190,6 +2190,8 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_BITMAPSET_FIELD(all_baserels);
WRITE_BITMAPSET_FIELD(nullable_baserels);
WRITE_NODE_FIELD(join_rel_list);
+ WRITE_NODE_FIELD(grouped_rel_list);
+ WRITE_NODE_FIELD(agg_info_list);
WRITE_INT_FIELD(join_cur_level);
WRITE_NODE_FIELD(init_plans);
WRITE_NODE_FIELD(cte_plan_ids);
@@ -2203,6 +2205,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(append_rel_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
@@ -2210,6 +2213,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(distinct_pathkeys);
WRITE_NODE_FIELD(sort_pathkeys);
WRITE_NODE_FIELD(processed_tlist);
+ WRITE_INT_FIELD(max_sortgroupref);
WRITE_NODE_FIELD(minmax_aggs);
WRITE_FLOAT_FIELD(total_table_pages, "%.0f");
WRITE_FLOAT_FIELD(tuple_fraction, "%.4f");
@@ -2428,6 +2432,20 @@ _outParamPathInfo(StringInfo str, const ParamPathInfo *node)
WRITE_NODE_FIELD(ppi_clauses);
}
+static void
+_outRelAggInfo(StringInfo str, const RelAggInfo *node)
+{
+ WRITE_NODE_TYPE("RELAGGINFO");
+
+ WRITE_BITMAPSET_FIELD(relids);
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(agg_input);
+ WRITE_FLOAT_FIELD(input_rows, "%.0f");
+ WRITE_NODE_FIELD(group_clauses);
+ WRITE_NODE_FIELD(group_exprs);
+ WRITE_NODE_FIELD(agg_exprs);
+}
+
static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
@@ -2527,6 +2545,18 @@ _outPlaceHolderInfo(StringInfo str, const PlaceHolderInfo *node)
WRITE_INT_FIELD(ph_width);
}
+static void
+_outGroupedVarInfo(StringInfo str, const GroupedVarInfo *node)
+{
+ WRITE_NODE_TYPE("GROUPEDVARINFO");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_BITMAPSET_FIELD(gv_eval_at);
+ WRITE_BOOL_FIELD(derived);
+}
+
static void
_outMinMaxAggInfo(StringInfo str, const MinMaxAggInfo *node)
{
@@ -4084,6 +4114,9 @@ outNode(StringInfo str, const void *obj)
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_RelAggInfo:
+ _outRelAggInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
@@ -4102,6 +4135,9 @@ outNode(StringInfo str, const void *obj)
case T_PlaceHolderInfo:
_outPlaceHolderInfo(str, obj);
break;
+ case T_GroupedVarInfo:
+ _outGroupedVarInfo(str, obj);
+ break;
case T_MinMaxAggInfo:
_outMinMaxAggInfo(str, obj);
break;
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
index 89ce373d5e..4633fb8768 100644
--- a/src/backend/optimizer/README
+++ b/src/backend/optimizer/README
@@ -1127,3 +1127,90 @@ breaking down aggregation or grouping over a partitioned relation into
aggregation or grouping over its partitions is called partitionwise
aggregation. Especially when the partition keys match the GROUP BY clause,
this can be significantly faster than the regular method.
+
+Aggregate push-down
+-------------------
+
+The obvious way to evaluate aggregates is to evaluate the FROM clause of the
+SQL query (this is what query_planner does) and use the resuing paths as the
+input of Agg node. However, if the groups are large enough, it may be more
+efficient to apply the partial aggregation to the output of base relation
+scan, and finalize it when we have all relations of the query joined:
+
+ EXPLAIN
+ SELECT a.i, avg(b.u)
+ FROM a JOIN b ON b.j = a.i
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+ -> Index Only Scan using a_pkey on a
+ Index Cond: (i = b.j)
+
+Thus the join above the partial aggregate node receives fewer input rows, and
+so the number of outer-to-inner pairs of tuples to be checked can be
+significantly lower, which can in turn lead to considerably lower join cost.
+
+Note that there's often no GROUP BY expression to be used for the partial
+aggregation, so we use equivalence classes to derive grouping expression: in
+the example above, the grouping key "b.j" was derived from "a.i".
+
+Furthermore, extra grouping columns can be added to the partial Agg node if a
+join clause above that node references a column which is not in the query
+GROUP BY clause and which could not be derived using equivalence class.
+
+ EXPLAIN
+ SELECT a.i, avg(b.u)
+ FROM a JOIN b ON b.j = a.x
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (a.x = b.j)
+ -> Seq Scan on a
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+
+Here the partial aggregate uses "b.j" as grouping column although it's not in
+the same equivalence class as "a.i". Note that no column of the {a.x, b.j}
+equivalence class is used as a key for the final aggregation.
+
+Besides base relation, the aggregation can also be pushed down to join:
+
+ EXPLAIN
+ SELECT a.i, avg(b.u + c.v)
+ FROM a JOIN b ON b.j = a.i
+ JOIN c ON c.k = a.i
+ WHERE b.j = c.k GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (b.j = a.i)
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Hash Join
+ Hash Cond: (b.j = c.k)
+ -> Seq Scan on b
+ -> Hash
+ -> Seq Scan on c
+ -> Hash
+ -> Seq Scan on a
+
+Whether the Agg node is created out of base relation or out of join, it's
+added to a separate RelOptInfo that we call "grouped relation". Grouped
+relation can be joined to a non-grouped relation, which results in a grouped
+relation too. Join of two grouped relations does not seem to be very useful
+and is currently not supported.
+
+If query_planner produces a grouped relation that contains valid paths, these
+are simply added to the UPPERREL_PARTIAL_GROUP_AGG relation. Further
+processing of these paths then does not differ from processing of other
+partially grouped paths.
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index b7723481b0..f14eb357dc 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -60,6 +60,7 @@ typedef struct pushdown_safety_info
/* These parameters are set by GUC */
bool enable_geqo = false; /* just in case GUC doesn't set it */
+bool enable_agg_pushdown;
int geqo_threshold;
int min_parallel_table_scan_size;
int min_parallel_index_scan_size;
@@ -124,6 +125,9 @@ static void set_result_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
+static void add_grouped_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, AggStrategy aggstrategy,
+ RelAggInfo *agg_info);
static RelOptInfo *make_rel_from_joinlist(PlannerInfo *root, List *joinlist);
static bool subquery_is_pushdown_safe(Query *subquery, Query *topquery,
pushdown_safety_info *safetyInfo);
@@ -2730,6 +2734,80 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
}
}
+/*
+ * generate_grouping_paths
+ * Create partially aggregated paths and add them to grouped relation.
+ *
+ * "rel_plain" is base or join relation whose paths are not grouped.
+ */
+void
+generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
+ RelOptInfo *rel_plain, RelAggInfo *agg_info)
+{
+ ListCell *lc;
+
+ if (IS_DUMMY_REL(rel_plain))
+ {
+ mark_dummy_rel(rel_grouped);
+ return;
+ }
+
+ foreach(lc, rel_plain->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ /*
+ * Since the path originates from the non-grouped relation which is
+ * not aware of the aggregate push-down, we must ensure that it
+ * provides the correct input for aggregation.
+ */
+ path = (Path *) create_projection_path(root, rel_grouped, path,
+ agg_info->agg_input);
+
+ /*
+ * add_grouped_path() will check whether the path has suitable
+ * pathkeys.
+ */
+ add_grouped_path(root, rel_grouped, path, AGG_SORTED, agg_info);
+
+ /*
+ * Repeated creation of hash table (for new parameter values) should
+ * be possible, does not sound like a good idea in terms of
+ * efficiency.
+ */
+ if (path->param_info == NULL)
+ add_grouped_path(root, rel_grouped, path, AGG_HASHED, agg_info);
+ }
+
+ /* Could not generate any grouped paths? */
+ if (rel_grouped->pathlist == NIL)
+ mark_dummy_rel(rel_grouped);
+}
+
+/*
+ * Apply partial aggregation to a subpath and add the AggPath to the pathlist.
+ */
+static void
+add_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ AggStrategy aggstrategy, RelAggInfo *agg_info)
+{
+ Path *agg_path;
+
+
+ if (aggstrategy == AGG_HASHED)
+ agg_path = (Path *) create_agg_hashed_path(root, rel, subpath,
+ agg_info);
+ else if (aggstrategy == AGG_SORTED)
+ agg_path = (Path *) create_agg_sorted_path(root, rel, subpath,
+ agg_info);
+ else
+ elog(ERROR, "unexpected strategy %d", aggstrategy);
+
+ /* Add the grouped path to the list of grouped base paths. */
+ if (agg_path != NULL)
+ add_path(rel, (Path *) agg_path);
+}
+
/*
* make_rel_from_joinlist
* Build access paths using a "joinlist" to guide the join path search.
@@ -2770,6 +2848,34 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
int varno = ((RangeTblRef *) jlnode)->rtindex;
thisrel = find_base_rel(root, varno);
+
+ /*
+ * Create a grouped relation to facilitate aggregate push-down.
+ * This makes no sense if thisrel is the only relation of the
+ * query.
+ */
+ if (bms_nonempty_difference(root->all_baserels, thisrel->relids))
+ {
+ RelOptInfo *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /*
+ * Build grouped relation if thisrel is suitable for partial
+ * aggregation.
+ */
+ rel_grouped = build_simple_grouped_rel(root, varno, &agg_info);
+
+ if (rel_grouped)
+ {
+ /* Make the relation available for joining. */
+ add_grouped_rel(root, rel_grouped, agg_info);
+
+ /* Add the aggregation paths to it. */
+ generate_grouping_paths(root, rel_grouped, thisrel,
+ agg_info);
+ set_cheapest(rel_grouped);
+ }
+ }
}
else if (IsA(jlnode, List))
{
@@ -2871,6 +2977,7 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
for (lev = 2; lev <= levels_needed; lev++)
{
+ RelOptInfo *rel_grouped;
ListCell *lc;
/*
@@ -2907,6 +3014,11 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
+ /* The same for grouped relation if one exists. */
+ rel_grouped = find_grouped_rel(root, rel->relids, NULL);
+ if (rel_grouped)
+ set_cheapest(rel_grouped);
+
#ifdef OPTIMIZER_DEBUG
debug_print_rel(root, rel);
#endif
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 228003fc4e..ddcb4b7867 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4414,7 +4414,6 @@ set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel)
0,
JOIN_INNER,
NULL);
-
rel->rows = clamp_row_est(nrows);
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
@@ -5392,11 +5391,11 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
foreach(lc, target->exprs)
{
Node *node = (Node *) lfirst(lc);
+ int32 item_width;
if (IsA(node, Var))
{
Var *var = (Var *) node;
- int32 item_width;
/* We should not see any upper-level Vars here */
Assert(var->varlevelsup == 0);
@@ -5427,6 +5426,20 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
Assert(item_width > 0);
tuple_width += item_width;
}
+ else if (IsA(node, Aggref))
+ {
+ /*
+ * If the target is evaluated by AggPath, it'll care of cost
+ * estimate. If the target is above AggPath (typically target of a
+ * join relation that contains grouped relation), the cost of
+ * Aggref should not be accounted for again.
+ *
+ * On the other hand, width is always needed.
+ */
+ item_width = get_typavgwidth(exprType(node), exprTypmod(node));
+ Assert(item_width > 0);
+ tuple_width += item_width;
+ }
else
{
/*
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 688d9b0707..33f2aad19c 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -2542,6 +2542,140 @@ is_redundant_derived_clause(RestrictInfo *rinfo, List *clauselist)
return false;
}
+/*
+ * translate_expression_to_rels
+ * If the appropriate equivalence classes exist, replace vars in
+ * gvi->gvexpr with vars whose varno is equal to relid. Return NULL if
+ * translation is not possible or needed.
+ *
+ * Note: Currently we only translate Var expressions. This is subject to
+ * change as the aggregate push-down feature gets enhanced.
+ */
+GroupedVarInfo *
+translate_expression_to_rels(PlannerInfo *root, GroupedVarInfo *gvi,
+ Index relid)
+{
+ Var *var;
+ ListCell *l1;
+ bool found_orig = false;
+ Var *var_translated = NULL;
+ GroupedVarInfo *result;
+
+ /* Can't do anything w/o equivalence classes. */
+ if (root->eq_classes == NIL)
+ return NULL;
+
+ var = castNode(Var, gvi->gvexpr);
+
+ /*
+ * Do we need to translate the var?
+ */
+ if (var->varno == relid)
+ return NULL;
+
+ /*
+ * Find the replacement var.
+ */
+ foreach(l1, root->eq_classes)
+ {
+ EquivalenceClass *ec = lfirst_node(EquivalenceClass, l1);
+ ListCell *l2;
+
+ /* TODO Check if any other EC kind should be ignored. */
+ if (ec->ec_has_volatile || ec->ec_below_outer_join || ec->ec_broken)
+ continue;
+
+ /* Single-element EC can hardly help in translations. */
+ if (list_length(ec->ec_members) == 1)
+ continue;
+
+ /*
+ * Collect all vars of this EC and their varnos.
+ *
+ * ec->ec_relids does not help because we're only interested in a
+ * subset of EC members.
+ */
+ foreach(l2, ec->ec_members)
+ {
+ EquivalenceMember *em = lfirst_node(EquivalenceMember, l2);
+ Var *ec_var;
+
+ /*
+ * The grouping expressions derived here are used to evaluate
+ * possibility to push aggregation down to RELOPT_BASEREL or
+ * RELOPT_JOINREL relations, and to construct reltargets for the
+ * grouped rels. We're not interested at the moment whether the
+ * relations do have children.
+ */
+ if (em->em_is_child)
+ continue;
+
+ if (!IsA(em->em_expr, Var))
+ continue;
+
+ ec_var = castNode(Var, em->em_expr);
+ if (equal(ec_var, var))
+ found_orig = true;
+ else if (ec_var->varno == relid)
+ var_translated = ec_var;
+
+ if (found_orig && var_translated)
+ {
+ /*
+ * The replacement Var must have the same data type, otherwise
+ * the values are not guaranteed to be grouped in the same way
+ * as values of the original Var.
+ */
+ if (ec_var->vartype != var->vartype)
+ return NULL;
+
+ break;
+ }
+ }
+
+ if (found_orig)
+ {
+ /*
+ * The same expression probably does not exist in multiple ECs.
+ */
+ if (var_translated == NULL)
+ {
+ /*
+ * Failed to translate the expression.
+ */
+ return NULL;
+ }
+ else
+ {
+ /* Success. */
+ break;
+ }
+ }
+ else
+ {
+ /*
+ * Vars of the requested relid can be in the next ECs too.
+ */
+ var_translated = NULL;
+ }
+ }
+
+ if (!found_orig)
+ return NULL;
+
+ result = makeNode(GroupedVarInfo);
+ memcpy(result, gvi, sizeof(GroupedVarInfo));
+
+ /*
+ * translate_expression_to_rels_mutator updates gv_eval_at.
+ */
+ result->gv_eval_at = bms_make_singleton(relid);
+ result->gvexpr = (Expr *) var_translated;
+ result->derived = true;
+
+ return result;
+}
+
/*
* is_redundant_with_indexclauses
* Test whether rinfo is redundant with any clause in the IndexClause
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index a9fc52c61f..5eea155f0c 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -22,6 +22,7 @@
#include "partitioning/partbounds.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+#include "utils/selfuncs.h"
static void make_rels_by_clause_joins(PlannerInfo *root,
@@ -36,7 +37,9 @@ static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1,
- RelOptInfo *rel2);
+ RelOptInfo *rel2,
+ RelAggInfo *agg_info,
+ RelOptInfo *rel_agg_input);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist);
@@ -652,9 +655,17 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
/*
* make_join_rel_common
* The workhorse of make_join_rel().
- */
+ *
+ * 'agg_info' contains the reltarget of grouped relation and everything we
+ * need to aggregate the join result. If NULL, then the join relation should
+ * not be grouped.
+ *
+ * 'rel_agg_input' describes the AggPath input relation if the join output
+ * should be aggregated. If NULL is passed, do not aggregate the join output.
+ */
static RelOptInfo *
-make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ RelAggInfo *agg_info, RelOptInfo *rel_agg_input)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -715,7 +726,7 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* goes with this particular joining.
*/
joinrel = build_join_rel(root, joinrelids, rel1, rel2, sjinfo,
- &restrictlist);
+ &restrictlist, agg_info);
/*
* If we've already proven this join is empty, we needn't consider any
@@ -728,14 +739,88 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
}
/* Add paths to the join relation. */
- populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
- restrictlist);
+ if (rel_agg_input == NULL)
+ {
+ /*
+ * Simply join the input relations, whether both are plain or one of
+ * them is grouped.
+ */
+ populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
+ restrictlist);
+ }
+ else
+ {
+ /* The join relation is grouped. */
+ Assert(agg_info != NULL);
+
+ /*
+ * Apply partial aggregation to the paths of rel_agg_input and add the
+ * resulting paths to joinrel.
+ */
+ generate_grouping_paths(root, joinrel, rel_agg_input, agg_info);
+ }
bms_free(joinrelids);
return joinrel;
}
+/*
+ * make_join_rel_combined
+ * Join grouped relation to non-grouped one.
+ */
+static void
+make_join_rel_combined(PlannerInfo *root, RelOptInfo *rel1,
+ RelOptInfo *rel2,
+ RelAggInfo *agg_info)
+{
+ RelOptInfo *rel1_grouped;
+ RelOptInfo *rel2_grouped;
+ bool rel1_grouped_useful = false;
+ bool rel2_grouped_useful = false;
+
+ /* Retrieve the grouped relations. */
+ rel1_grouped = find_grouped_rel(root, rel1->relids, NULL);
+ rel2_grouped = find_grouped_rel(root, rel2->relids, NULL);
+
+ /*
+ * Dummy rel may indicate a join relation that is able to generate grouped
+ * paths as such (i.e. it has valid agg_info), but for which the path
+ * actually could not be created (e.g. only AGG_HASHED strategy was
+ * possible but work_mem was not sufficient for hash table).
+ */
+ rel1_grouped_useful = rel1_grouped != NULL && !IS_DUMMY_REL(rel1_grouped);
+ rel2_grouped_useful = rel2_grouped != NULL && !IS_DUMMY_REL(rel2_grouped);
+
+ /* Nothing to do if there's no grouped relation. */
+ if (!rel1_grouped_useful && !rel2_grouped_useful)
+ return;
+
+ /*
+ * At maximum one input rel can be grouped (here we don't care if any rel
+ * is eventually dummy, the existence of grouped rel indicates that
+ * aggregates can be pushed down to it). If both were grouped, then
+ * grouping of one side would change the occurrence of the other side's
+ * aggregate transient states on the input of the final aggregation. This
+ * can be handled by adjusting the transient states, but it's not worth
+ * the effort because it's hard to find a use case for this kind of join.
+ *
+ * XXX If the join of two grouped rels is implemented someday, note that
+ * both rels can have aggregates, so it'd be hard to join grouped rel to
+ * non-grouped here: 1) such a "mixed join" would require a special
+ * target, 2) both AGGSPLIT_FINAL_DESERIAL and AGGSPLIT_SIMPLE aggregates
+ * could appear in the target of the final aggregation node, originating
+ * from the grouped and the non-grouped input rel respectively.
+ */
+ if (rel1_grouped && rel2_grouped)
+ return;
+
+ if (rel1_grouped_useful)
+ make_join_rel_common(root, rel1_grouped, rel2, agg_info, NULL);
+ else if (rel2_grouped_useful)
+ make_join_rel_common(root, rel1, rel2_grouped, agg_info, NULL);
+}
+
/*
* make_join_rel
* Find or create a join RelOptInfo that represents the join of
@@ -744,14 +829,83 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* (The join rel may already contain paths generated from other
* pairs of rels that add up to the same set of base rels.)
*
+ * In addition to creating an ordinary join relation, try to create a
+ * grouped one. There are two strategies to achieve that: join a grouped
+ * relation to plain one, or join two plain relations and apply partial
+ * aggregation to the result.
+ *
* NB: will return NULL if attempted join is not valid. This can happen when
* working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
+ * turned into joins. Besides that, NULL is also returned if caller is
+ * interested in a grouped relation but it could not be created.
+ *
+ * Only the plain relation is returned; if grouped relation exists, it can be
+ * retrieved using find_grouped_rel().
*/
RelOptInfo *
make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
{
- return make_join_rel_common(root, rel1, rel2);
+ Relids joinrelids;
+ RelAggInfo *agg_info = NULL;
+ RelOptInfo *joinrel,
+ *joinrel_plain;
+
+ /* 1) form the plain join. */
+ joinrel = make_join_rel_common(root, rel1, rel2, NULL, NULL);
+ joinrel_plain = joinrel;
+
+ if (joinrel_plain == NULL)
+ return joinrel_plain;
+
+ /*
+ * We're done if there are no grouping expressions nor aggregates.
+ */
+ if (root->grouped_var_list == NIL)
+ return joinrel_plain;
+
+ joinrelids = bms_union(rel1->relids, rel2->relids);
+ joinrel = find_grouped_rel(root, joinrelids, &agg_info);
+
+ if (joinrel != NULL)
+ {
+ /*
+ * If the same grouped joinrel was already formed, just with the base
+ * rels divided between rel1 and rel2 in a different way, the matching
+ * agg_info should already be there.
+ */
+ Assert(agg_info != NULL);
+ }
+ else
+ {
+ /*
+ * agg_info must be created from scratch.
+ */
+ agg_info = create_rel_agg_info(root, joinrel_plain);
+
+ /* Cannot we build grouped join? */
+ if (agg_info == NULL)
+ return joinrel_plain;
+
+ /*
+ * The number of aggregate input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ agg_info->input_rows = joinrel_plain->rows;
+ }
+
+ /*
+ * 2) join two plain rels and aggregate the join paths. Aggregate
+ * push-down only makes sense if the join is not the top-level one.
+ */
+ if (bms_nonempty_difference(root->all_baserels, joinrelids))
+ make_join_rel_common(root, rel1, rel2, agg_info, joinrel_plain);
+
+ /*
+ * 3) combine plain and grouped relations.
+ */
+ make_join_rel_combined(root, rel1, rel2, agg_info);
+
+ return joinrel_plain;
}
/*
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 73da0c2d8e..9f97a2dd8f 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -48,6 +48,8 @@ typedef struct PostponedQual
} PostponedQual;
+static void create_aggregate_grouped_var_infos(PlannerInfo *root);
+static void create_grouping_expr_grouped_var_infos(PlannerInfo *root);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
@@ -273,6 +275,261 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
}
}
+/*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate as well as for
+ * each possible grouping expression.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+extern void
+setup_aggregate_pushdown(PlannerInfo *root)
+{
+ ListCell *lc;
+
+ /*
+ * Isn't user interested in the aggregate push-down feature?
+ */
+ if (!enable_agg_pushdown)
+ return;
+
+ /* The feature can only be applied to grouped aggregation. */
+ if (!root->parse->groupClause)
+ return;
+
+ /*
+ * Grouping sets require multiple different groupings but the base
+ * relation can only generate one.
+ */
+ if (root->parse->groupingSets)
+ return;
+
+ /*
+ * SRF is not allowed in the aggregate argument and we don't even want it
+ * in the GROUP BY clause, so forbid it in general. It needs to be
+ * analyzed if evaluation of a GROUP BY clause containing SRF below the
+ * query targetlist would be correct. Currently it does not seem to be an
+ * important use case.
+ */
+ if (root->parse->hasTargetSRFs)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_aggregate_grouped_var_infos(root);
+
+ /* Isn't there any aggregate to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Create GroupedVarInfo per grouping expression. */
+ create_grouping_expr_grouped_var_infos(root);
+
+ /* Isn't there any useful grouping expression for aggregate push-down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /*
+ * Now that we know that grouping can be pushed down, search for the
+ * maximum sortgroupref. The base relations may need it if extra grouping
+ * expressions get added to them.
+ */
+ Assert(root->max_sortgroupref == 0);
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc);
+
+ if (te->ressortgroupref > root->max_sortgroupref)
+ root->max_sortgroupref = te->ressortgroupref;
+ }
+}
+
+/*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ */
+static void
+create_aggregate_grouped_var_infos(PlannerInfo *root)
+{
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES |
+ PVC_INCLUDE_WINDOWFUNCS);
+
+ /*
+ * Although GroupingFunc is related to root->parse->groupingSets, this
+ * field does not necessarily reflect its presence.
+ */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, GroupingFunc))
+ return;
+ }
+
+ /*
+ * Aggregates within the HAVING clause need to be processed in the same
+ * way as those in the main targetlist.
+ *
+ * Note that the contained aggregates will be pushed down, but the
+ * containing HAVING clause must be ignored until the aggregation is
+ * finalized.
+ */
+ if (root->parse->havingQual != NULL)
+ {
+ List *having_exprs;
+
+ having_exprs = pull_var_clause((Node *) root->parse->havingQual,
+ PVC_INCLUDE_AGGREGATES);
+ if (having_exprs != NIL)
+ tlist_exprs = list_concat(tlist_exprs, having_exprs);
+ }
+
+ if (tlist_exprs == NIL)
+ return;
+
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ /*
+ * tlist_exprs may also contain Vars or WindowFuncs, but we only need
+ * Aggrefs.
+ */
+ if (IsA(expr, Var) ||IsA(expr, WindowFunc))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct)
+ {
+ /*
+ * Aggregation push-down is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ gvi = lfirst_node(GroupedVarInfo, lc2);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ gvi = makeNode(GroupedVarInfo);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ gvi->gv_eval_at = NULL;
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+}
+
+/*
+ * Create GroupedVarInfo for each expression usable as grouping key.
+ *
+ * In addition to the expressions of the query targetlist, group_pathkeys is
+ * also considered the source of grouping expressions. That increases the
+ * chance to get the relation output grouped.
+ */
+static void
+create_grouping_expr_grouped_var_infos(PlannerInfo *root)
+{
+ ListCell *l1,
+ *l2;
+ List *exprs = NIL;
+ List *sortgrouprefs = NIL;
+
+ /*
+ * Make sure GroupedVarInfo exists for each expression usable as grouping
+ * key.
+ */
+ foreach(l1, root->parse->groupClause)
+ {
+ SortGroupClause *sgClause;
+ TargetEntry *te;
+ Index sortgroupref;
+
+ sgClause = lfirst_node(SortGroupClause, l1);
+ te = get_sortgroupclause_tle(sgClause, root->processed_tlist);
+ sortgroupref = te->ressortgroupref;
+
+ Assert(sortgroupref > 0);
+
+ /*
+ * Non-zero sortgroupref does not necessarily imply grouping
+ * expression: data can also be sorted by aggregate.
+ */
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ /*
+ * The aggregate push-down feature currently supports only plain Vars
+ * as grouping expressions.
+ */
+ if (!IsA(te->expr, Var))
+ {
+ root->grouped_var_list = NIL;
+ return;
+ }
+
+ exprs = lappend(exprs, te->expr);
+ sortgrouprefs = lappend_int(sortgrouprefs, sortgroupref);
+ }
+
+ /*
+ * Construct GroupedVarInfo for each expression.
+ */
+ forboth(l1, exprs, l2, sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, l1);
+ int sortgroupref = lfirst_int(l2);
+ GroupedVarInfo *gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvexpr = (Expr *) copyObject(var);
+ gvi->sortgroupref = sortgroupref;
+
+ /* Find out where the expression should be evaluated. */
+ gvi->gv_eval_at = bms_make_singleton(var->varno);
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+}
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c
index 9381939c82..c4d07dfac4 100644
--- a/src/backend/optimizer/plan/planagg.c
+++ b/src/backend/optimizer/plan/planagg.c
@@ -442,7 +442,7 @@ build_minmax_path(PlannerInfo *root, MinMaxAggInfo *mminfo,
subroot->tuple_fraction = 1.0;
subroot->limit_tuples = 1.0;
- final_rel = query_planner(subroot, minmax_qp_callback, NULL);
+ final_rel = query_planner(subroot, minmax_qp_callback, NULL, NULL);
/*
* Since we didn't go through subquery_planner() to handle the subquery,
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 0b9999c8a6..4103a393e6 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -45,6 +45,9 @@
* qp_callback is a function to compute query_pathkeys once it's safe to do so
* qp_extra is optional extra data to pass to qp_callback
*
+ * If final_rel_grouped_p is valid, relation containing grouped paths may be
+ * saved to *final_rel_grouped_p.
+ *
* Note: the PlannerInfo node also includes a query_pathkeys field, which
* tells query_planner the sort order that is desired in the final output
* plan. This value is *not* available at call time, but is computed by
@@ -53,7 +56,8 @@
*/
RelOptInfo *
query_planner(PlannerInfo *root,
- query_pathkeys_callback qp_callback, void *qp_extra)
+ query_pathkeys_callback qp_callback, void *qp_extra,
+ RelOptInfo **final_rel_grouped_p)
{
Query *parse = root->parse;
List *joinlist;
@@ -66,6 +70,8 @@ query_planner(PlannerInfo *root,
* here.
*/
root->join_rel_list = makeNode(RelInfoList);
+ root->grouped_rel_list = makeNode(RelInfoList);
+ root->agg_info_list = makeNode(RelInfoList);
root->join_rel_level = NULL;
root->join_cur_level = 0;
root->canon_pathkeys = NIL;
@@ -74,6 +80,7 @@ query_planner(PlannerInfo *root,
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
@@ -254,6 +261,16 @@ query_planner(PlannerInfo *root,
*/
extract_restriction_or_clauses(root);
+ /*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, setup
+ * root->grouped_var_list.
+ *
+ * The base relations should be fully initialized now, so that we have
+ * enough info to decide whether grouping is possible.
+ */
+ setup_aggregate_pushdown(root);
+
/*
* Now expand appendrels by adding "otherrels" for their children. We
* delay this to the end so that we have as much information as possible
@@ -274,5 +291,9 @@ query_planner(PlannerInfo *root,
final_rel->cheapest_total_path->param_info != NULL)
elog(ERROR, "failed to construct the join relation");
+ if (final_rel_grouped_p)
+ *final_rel_grouped_p = find_grouped_rel(root, final_rel->relids,
+ NULL);
+
return final_rel;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 401299e542..b99b979d95 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -151,6 +151,7 @@ static double get_number_of_groups(PlannerInfo *root,
List *target_list);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
PathTarget *target,
bool target_parallel_safe,
const AggClauseCosts *agg_costs,
@@ -164,6 +165,7 @@ static RelOptInfo *make_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
Node *havingQual);
static void create_ordinary_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
RelOptInfo *grouped_rel,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd,
@@ -623,6 +625,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
memset(root->upper_rels, 0, sizeof(root->upper_rels));
memset(root->upper_targets, 0, sizeof(root->upper_targets));
root->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
root->grouping_map = NULL;
root->minmax_aggs = NIL;
root->qual_security_level = 0;
@@ -1943,6 +1946,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
List *activeWindows = NIL;
grouping_sets_data *gset_data = NULL;
standard_qp_extra qp_extra;
+ RelOptInfo *current_rel_grouped = NULL;
/* A recursive query should always have setOperations */
Assert(!root->hasRecursion);
@@ -2045,7 +2049,9 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
* We also generate (in standard_qp_callback) pathkey representations
* of the query's sort clause, distinct clause, etc.
*/
- current_rel = query_planner(root, standard_qp_callback, &qp_extra);
+ current_rel = query_planner(root,
+ standard_qp_callback, &qp_extra,
+ ¤t_rel_grouped);
/*
* Convert the query's result tlist into PathTarget format.
@@ -2190,6 +2196,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
{
current_rel = create_grouping_paths(root,
current_rel,
+ current_rel_grouped,
grouping_target,
grouping_target_parallel_safe,
&agg_costs,
@@ -3798,6 +3805,7 @@ get_number_of_groups(PlannerInfo *root,
static RelOptInfo *
create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
PathTarget *target,
bool target_parallel_safe,
const AggClauseCosts *agg_costs,
@@ -3888,7 +3896,8 @@ create_grouping_paths(PlannerInfo *root,
else
extra.patype = PARTITIONWISE_AGGREGATE_NONE;
- create_ordinary_grouping_paths(root, input_rel, grouped_rel,
+ create_ordinary_grouping_paths(root, input_rel, input_rel_grouped,
+ grouped_rel,
agg_costs, gd, &extra,
&partially_grouped_rel);
}
@@ -4046,6 +4055,7 @@ create_degenerate_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
*/
static void
create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
RelOptInfo *grouped_rel,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd,
@@ -4093,13 +4103,23 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
if ((extra->flags & GROUPING_CAN_PARTIAL_AGG) != 0)
{
bool force_rel_creation;
+ bool have_agg_pushdown_paths;
/*
- * If we're doing partitionwise aggregation at this level, force
- * creation of a partially_grouped_rel so we can add partitionwise
- * paths to it.
+ * Check if the aggregate push-down feature succeeded to generate any
+ * paths. Dummy relation can appear here because grouped paths are not
+ * guaranteed to exist for a relation.
*/
- force_rel_creation = (patype == PARTITIONWISE_AGGREGATE_PARTIAL);
+ have_agg_pushdown_paths = input_rel_grouped != NULL &&
+ !IS_DUMMY_REL(input_rel_grouped);
+
+ /*
+ * If we're doing partitionwise aggregation at this level or if
+ * aggregate push-down succeeded to create some paths, force creation
+ * of a partially_grouped_rel so we can add the related paths to it.
+ */
+ force_rel_creation = patype == PARTITIONWISE_AGGREGATE_PARTIAL ||
+ have_agg_pushdown_paths;
partially_grouped_rel =
create_partial_grouping_paths(root,
@@ -4108,6 +4128,23 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
gd,
extra,
force_rel_creation);
+
+ /*
+ * No further processing is needed for paths provided by the aggregate
+ * push-down feature. Simply add them to the partially grouped
+ * relation.
+ */
+ if (have_agg_pushdown_paths)
+ {
+ ListCell *lc;
+
+ foreach(lc, input_rel_grouped->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ add_path(partially_grouped_rel, path);
+ }
+ }
}
/* Set out parameter. */
@@ -4132,10 +4169,14 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
/* Gather any partially grouped partial paths. */
if (partially_grouped_rel && partially_grouped_rel->partial_pathlist)
- {
gather_grouping_paths(root, partially_grouped_rel);
+
+ /*
+ * The non-partial paths can come either from the Gather above or from
+ * aggregate push-down.
+ */
+ if (partially_grouped_rel && partially_grouped_rel->pathlist)
set_cheapest(partially_grouped_rel);
- }
/*
* Estimate number of groups.
@@ -7296,6 +7337,7 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
/* Create grouping paths for this child relation. */
create_ordinary_grouping_paths(root, child_input_rel,
+ NULL,
child_grouped_rel,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index dc11f098e0..3810345b3d 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -2380,6 +2380,39 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, Aggref))
+ {
+ Aggref *aggref = castNode(Aggref, node);
+
+ /*
+ * The upper plan targetlist can contain Aggref whose value has
+ * already been evaluated by the subplan. However this can only happen
+ * with specific value of aggsplit.
+ */
+ if (aggref->aggsplit == AGGSPLIT_INITIAL_SERIAL)
+ {
+ /* See if the Aggref has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ }
+
+ /* No referent found for Aggref */
+ elog(ERROR, "Aggref not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 67eeba938d..6cb33445e2 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -891,6 +891,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
memset(subroot->upper_rels, 0, sizeof(subroot->upper_rels));
memset(subroot->upper_targets, 0, sizeof(subroot->upper_targets));
subroot->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
subroot->grouping_map = NULL;
subroot->minmax_aggs = NIL;
subroot->qual_security_level = 0;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index d884d2bb00..70a3f73d93 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2992,6 +2992,146 @@ create_agg_path(PlannerInfo *root,
return pathnode;
}
+/*
+ * Apply AGG_SORTED aggregation path to subpath if it's suitably sorted.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+AggPath *
+create_agg_sorted_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ RelAggInfo *agg_info)
+{
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ ListCell *lc1;
+ List *key_subset = NIL;
+ AggPath *result = NULL;
+
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, (Node *) agg_exprs, aggsplit, &agg_costs);
+
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until the
+ * final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath, target,
+ AGG_SORTED, aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+
+ return result;
+}
+
+/*
+ * Apply AGG_HASHED aggregation to subpath.
+ */
+AggPath *
+create_agg_hashed_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, RelAggInfo *agg_info)
+{
+ bool can_hash;
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ double hashaggtablesize;
+ Query *parse = root->parse;
+ AggPath *result = NULL;
+
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, agg_exprs, aggsplit, &agg_costs);
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until
+ * the final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath,
+ target,
+ AGG_HASHED,
+ aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+
+ /*
+ * The agg path should require no fewer parameters than the plain
+ * one.
+ */
+ result->path.param_info = subpath->param_info;
+ }
+ }
+
+ return result;
+}
+
/*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index c238dd6538..10becc09e9 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -17,18 +17,24 @@
#include <limits.h>
#include "miscadmin.h"
+#include "catalog/pg_class_d.h"
+#include "catalog/pg_constraint.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
+#include "optimizer/optimizer.h"
#include "optimizer/inherit.h"
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/placeholder.h"
#include "optimizer/plancat.h"
+#include "optimizer/planner.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+#include "parser/parse_oper.h"
#include "partitioning/partbounds.h"
#include "utils/hsearch.h"
+#include "utils/selfuncs.h"
typedef struct RelInfoEntry
@@ -63,6 +69,9 @@ static void build_child_join_reltarget(PlannerInfo *root,
RelOptInfo *childrel,
int nappinfos,
AppendRelInfo **appinfos);
+static bool init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p);
/*
@@ -351,6 +360,102 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
return rel;
}
+/*
+ * build_simple_grouped_rel
+ * Construct a new RelOptInfo for a grouped base relation out of an
+ * existing non-grouped relation. On success, pointer to the corresponding
+ * RelAggInfo is stored in *agg_info_p in addition to returning the grouped
+ * relation.
+ */
+RelOptInfo *
+build_simple_grouped_rel(PlannerInfo *root, int relid,
+ RelAggInfo **agg_info_p)
+{
+ RangeTblEntry *rte;
+ RelOptInfo *rel_plain,
+ *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /* Isn't there any grouping expression to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return NULL;
+
+ rel_plain = root->simple_rel_array[relid];
+
+ /* Caller should only pass rti that represents base relation. */
+ Assert(rel_plain != NULL);
+
+ /*
+ * Not all RTE kinds are supported when grouping is considered.
+ *
+ * TODO Consider relaxing some of these restrictions.
+ */
+ rte = root->simple_rte_array[rel_plain->relid];
+ if (rte->rtekind != RTE_RELATION ||
+ rte->relkind == RELKIND_FOREIGN_TABLE ||
+ rte->tablesample != NULL)
+ return NULL;
+
+ /*
+ * Grouped append relation is not supported yet.
+ */
+ if (rte->inh)
+ return NULL;
+
+ /*
+ * Currently we do not support child relations ("other rels").
+ */
+ if (rel_plain->reloptkind != RELOPT_BASEREL)
+ return NULL;
+
+ /*
+ * Prepare the information we need for aggregation of the rel contents.
+ */
+ agg_info = create_rel_agg_info(root, rel_plain);
+ if (agg_info == NULL)
+ return NULL;
+
+ /*
+ * TODO Consider if 1) a flat copy is o.k., 2) it's safer in terms of
+ * adding new fields to RelOptInfo) to copy everything and then reset some
+ * fields, or to zero the structure and copy individual fields.
+ */
+ rel_grouped = makeNode(RelOptInfo);
+ memcpy(rel_grouped, rel_plain, sizeof(RelOptInfo));
+
+ /*
+ * Note on consider_startup: while the AGG_HASHED strategy needs the whole
+ * relation, AGG_SORTED does not. Therefore we do not force
+ * consider_startup to false.
+ */
+
+ /*
+ * Set the appropriate target for grouped paths.
+ *
+ * reltarget should match the target of partially aggregated paths.
+ */
+ rel_grouped->reltarget = agg_info->target;
+
+ /*
+ * Grouped paths must not be mixed with the plain ones.
+ */
+ rel_grouped->pathlist = NIL;
+ rel_grouped->partial_pathlist = NIL;
+ rel_grouped->cheapest_startup_path = NULL;
+ rel_grouped->cheapest_total_path = NULL;
+ rel_grouped->cheapest_unique_path = NULL;
+ rel_grouped->cheapest_parameterized_paths = NIL;
+
+ /*
+ * The number of aggregation input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ agg_info->input_rows = rel_plain->rows;
+
+ *agg_info_p = agg_info;
+ return rel_grouped;
+}
+
/*
* find_base_rel
* Find a base or other relation entry, which must already exist.
@@ -462,8 +567,12 @@ find_rel_info(RelInfoList *list, Relids relids)
void *item = lfirst(l);
Relids item_relids;
- Assert(IsA(item, RelOptInfo));
- item_relids = ((RelOptInfo *) item)->relids;
+ Assert(IsA(item, RelOptInfo) ||IsA(item, RelAggInfo));
+
+ if (IsA(item, RelOptInfo))
+ item_relids = ((RelOptInfo *) item)->relids;
+ else if (IsA(item, RelAggInfo))
+ item_relids = ((RelAggInfo *) item)->relids;
if (bms_equal(item_relids, relids))
return item;
@@ -492,7 +601,7 @@ find_join_rel(PlannerInfo *root, Relids relids)
static void
add_rel_info(RelInfoList *list, void *data)
{
- Assert(IsA(data, RelOptInfo));
+ Assert(IsA(data, RelOptInfo) ||IsA(data, RelAggInfo));
/* GEQO requires us to append the new joinrel to the end of the list! */
list->items = lappend(list->items, data);
@@ -504,7 +613,11 @@ add_rel_info(RelInfoList *list, void *data)
RelInfoEntry *hentry;
bool found;
- relids = ((RelOptInfo *) data)->relids;
+ if (IsA(data, RelOptInfo))
+ relids = ((RelOptInfo *) data)->relids;
+ else if (IsA(data, RelAggInfo))
+ relids = ((RelAggInfo *) data)->relids;
+
hentry = (RelInfoEntry *) hash_search(list->hash,
&relids,
HASH_ENTER,
@@ -525,6 +638,57 @@ add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
add_rel_info(root->join_rel_list, joinrel);
}
+/*
+ * add_grouped_rel
+ * Add grouped base or join relation to the list of grouped relations in
+ * the given PlannerInfo. Also add the corresponding RelAggInfo to
+ * agg_info_list.
+ */
+void
+add_grouped_rel(PlannerInfo *root, RelOptInfo *rel, RelAggInfo *agg_info)
+{
+ add_rel_info(root->grouped_rel_list, rel);
+ add_rel_info(root->agg_info_list, rel);
+}
+
+/*
+ * find_grouped_rel
+ * Returns grouped relation entry (base or join relation) corresponding to
+ * 'relids' or NULL if none exists.
+ *
+ * If agg_info_p is a valid pointer, then pointer to RelAggInfo that
+ * corresponds to the relation returned is assigned to *agg_info_p.
+ */
+RelOptInfo *
+find_grouped_rel(PlannerInfo *root, Relids relids, RelAggInfo **agg_info_p)
+{
+ RelOptInfo *rel;
+
+ rel = (RelOptInfo *) find_rel_info(root->grouped_rel_list, relids);
+ if (rel == NULL)
+ {
+ if (agg_info_p)
+ *agg_info_p = NULL;
+
+ return NULL;
+ }
+
+ /* Is caller interested in RelAggInfo? */
+ if (agg_info_p)
+ {
+ RelAggInfo *agg_info;
+
+ agg_info = (RelAggInfo *) find_rel_info(root->agg_info_list, relids);
+
+ /* The relation exists, so the agg_info should be there too. */
+ Assert(agg_info != NULL);
+
+ *agg_info_p = agg_info;
+ }
+
+ return rel;
+}
+
/*
* set_foreign_rel_properties
* Set up foreign-join fields if outer and inner relation are foreign
@@ -587,6 +751,7 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* 'restrictlist_ptr': result variable. If not NULL, *restrictlist_ptr
* receives the list of RestrictInfo nodes that apply to this
* particular pair of joinable relations.
+ * 'agg_info' indicates that grouped join relation should be created.
*
* restrictlist_ptr makes the routine's API a little grotty, but it saves
* duplicated calculation of the restrictlist...
@@ -597,10 +762,12 @@ build_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr)
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info)
{
RelOptInfo *joinrel;
List *restrictlist;
+ bool grouped = agg_info != NULL;
/* This function should be used only for join between parents. */
Assert(!IS_OTHER_REL(outer_rel) && !IS_OTHER_REL(inner_rel));
@@ -608,7 +775,8 @@ build_join_rel(PlannerInfo *root,
/*
* See if we already have a joinrel for this set of base rels.
*/
- joinrel = find_join_rel(root, joinrelids);
+ joinrel = !grouped ? find_join_rel(root, joinrelids) :
+ find_grouped_rel(root, joinrelids, NULL);
if (joinrel)
{
@@ -700,9 +868,21 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ if (!grouped)
+ {
+ joinrel->reltarget = create_empty_pathtarget();
+ build_joinrel_tlist(root, joinrel, outer_rel);
+ build_joinrel_tlist(root, joinrel, inner_rel);
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ }
+ else
+ {
+ /*
+ * The target for grouped join should already have its cost and width
+ * computed, see create_rel_agg_info().
+ */
+ joinrel->reltarget = agg_info->target;
+ }
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -734,49 +914,73 @@ build_join_rel(PlannerInfo *root,
joinrel->has_eclass_joins = has_relevant_eclass_joinclause(root, joinrel);
/* Store the partition information. */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- sjinfo->jointype);
-
- /*
- * Set estimates of the joinrel's size.
- */
- set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
- sjinfo, restrictlist);
-
- /*
- * Set the consider_parallel flag if this joinrel could potentially be
- * scanned within a parallel worker. If this flag is false for either
- * inner_rel or outer_rel, then it must be false for the joinrel also.
- * Even if both are true, there might be parallel-restricted expressions
- * in the targetlist or quals.
- *
- * Note that if there are more than two rels in this relation, they could
- * be divided between inner_rel and outer_rel in any arbitrary way. We
- * assume this doesn't matter, because we should hit all the same baserels
- * and joinclauses while building up to this joinrel no matter which we
- * take; therefore, we should make the same decision here however we get
- * here.
- */
- if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
- is_parallel_safe(root, (Node *) restrictlist) &&
- is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
- joinrel->consider_parallel = true;
+ if (!grouped)
+ build_joinrel_partition_info(joinrel, outer_rel, inner_rel,
+ restrictlist, sjinfo->jointype);
/* Add the joinrel to the PlannerInfo. */
- add_join_rel(root, joinrel);
+ if (!grouped)
+ add_join_rel(root, joinrel);
+ else
+ add_grouped_rel(root, joinrel, agg_info);
/*
- * Also, if dynamic-programming join search is active, add the new joinrel
- * to the appropriate sublist. Note: you might think the Assert on number
- * of members should be for equality, but some of the level 1 rels might
- * have been joinrels already, so we can only assert <=.
+ * Also, if dynamic-programming join search is active, add the new
+ * joinrelset to the appropriate sublist. Note: you might think the
+ * Assert on number of members should be for equality, but some of the
+ * level 1 rels might have been joinrels already, so we can only assert
+ * <=.
+ *
+ * Do noting for grouped relation as it's stored aside from
+ * join_rel_level.
*/
- if (root->join_rel_level)
+ if (root->join_rel_level && !grouped)
{
Assert(root->join_cur_level > 0);
- Assert(root->join_cur_level <= bms_num_members(joinrel->relids));
+ Assert(root->join_cur_level <= bms_num_members(joinrelids));
root->join_rel_level[root->join_cur_level] =
- lappend(root->join_rel_level[root->join_cur_level], joinrel);
+ lappend(root->join_rel_level[root->join_cur_level],
+ joinrel);
+ }
+
+ /* Set estimates of the joinrel's size. */
+ if (!grouped)
+ {
+ set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
+ sjinfo, restrictlist);
+
+ /*
+ * Set the consider_parallel flag if this joinrel could potentially be
+ * scanned within a parallel worker. If this flag is false for either
+ * inner_rel or outer_rel, then it must be false for the joinrel also.
+ * Even if both are true, there might be parallel-restricted
+ * expressions in the targetlist or quals.
+ *
+ * Note that if there are more than two rels in this relation, they
+ * could be divided between inner_rel and outer_rel in any arbitrary
+ * way. We assume this doesn't matter, because we should hit all the
+ * same baserels and joinclauses while building up to this joinrel no
+ * matter which we take; therefore, we should make the same decision
+ * here however we get here.
+ */
+ if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
+ is_parallel_safe(root, (Node *) restrictlist) &&
+ is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
+ joinrel->consider_parallel = true;
+ }
+ else
+ {
+ /*
+ * Grouping essentially changes the number of rows.
+ *
+ * XXX We do not distinguish whether two plain rels are joined and the
+ * result is aggregated, or the aggregation has been already applied
+ * to one of the input rels. Is this worth extra effort, e.g.
+ * maintaining a separate RelOptInfo for each case (one difficulty
+ * that would introduce is construction of AppendPath)?
+ */
+ joinrel->rows = estimate_num_groups(root, agg_info->group_exprs,
+ agg_info->input_rows, NULL);
}
return joinrel;
@@ -1820,3 +2024,625 @@ build_child_join_reltarget(PlannerInfo *root,
childrel->reltarget->cost.per_tuple = parentrel->reltarget->cost.per_tuple;
childrel->reltarget->width = parentrel->reltarget->width;
}
+
+/*
+ * Check if the relation can produce grouped paths and return the information
+ * it'll need for it. The passed relation is the non-grouped one which has the
+ * reltarget already constructed.
+ */
+RelAggInfo *
+create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel)
+{
+ List *gvis;
+ List *aggregates = NIL;
+ bool found_other_rel_agg;
+ ListCell *lc;
+ RelAggInfo *result;
+ PathTarget *agg_input;
+ PathTarget *target = NULL;
+ List *grp_exprs_extra = NIL;
+ List *group_clauses_final;
+ int i;
+
+ /*
+ * The function shouldn't have been called if there's no opportunity for
+ * aggregation push-down.
+ */
+ Assert(root->grouped_var_list != NIL);
+
+ result = makeNode(RelAggInfo);
+
+ /*
+ * The current implementation of aggregation push-down cannot handle
+ * PlaceHolderVar (PHV).
+ *
+ * If we knew that the PHV should be evaluated in this target (and of
+ * course, if its expression matched some Aggref argument), we'd just let
+ * init_grouping_targets add that Aggref. On the other hand, if we knew
+ * that the PHV is evaluated below the current rel, we could ignore it
+ * because the referencing Aggref would take care of propagation of the
+ * value to upper joins.
+ *
+ * The problem is that the same PHV can be evaluated in the target of the
+ * current rel or in that of lower rel --- depending on the input paths.
+ * For example, consider rel->relids = {A, B, C} and if ph_eval_at = {B,
+ * C}. Path "A JOIN (B JOIN C)" implies that the PHV is evaluated by the
+ * "(B JOIN C)", while path "(A JOIN B) JOIN C" evaluates the PHV itself.
+ */
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *expr = lfirst(lc);
+
+ if (IsA(expr, PlaceHolderVar))
+ return NULL;
+ }
+
+ if (IS_SIMPLE_REL(rel))
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return NULL;
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the Agg plan
+ * would receive different input at the base rel level.
+ *
+ * XXX For RELOPT_JOINREL, do not return if all the joins that can set any
+ * entry of the grouped target (do we need to postpone this check until
+ * the grouped target is available, and init_grouping_targets take care?)
+ * of this rel to NULL are provably below rel. (It's ok if rel is one of
+ * these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return NULL;
+
+ /*
+ * Use equivalence classes to generate additional grouping expressions for
+ * the current rel. Without these we might not be able to apply
+ * aggregation to the relation result set.
+ *
+ * It's important that create_grouping_expr_grouped_var_infos has
+ * processed the explicit grouping columns by now. If the grouping clause
+ * contains multiple expressions belonging to the same EC, the original
+ * (i.e. not derived) one should be preferred when we build grouping
+ * target for a relation. Otherwise we have a problem when trying to match
+ * target entries to grouping clauses during plan creation, see
+ * get_grouping_expression().
+ */
+ gvis = list_copy(root->grouped_var_list);
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+ int relid = -1;
+
+ /* Only interested in grouping expressions. */
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ while ((relid = bms_next_member(rel->relids, relid)) >= 0)
+ {
+ GroupedVarInfo *gvi_trans;
+
+ gvi_trans = translate_expression_to_rels(root, gvi, relid);
+ if (gvi_trans != NULL)
+ gvis = lappend(gvis, gvi_trans);
+ }
+ }
+
+ /*
+ * Check if some aggregates or grouping expressions can be evaluated in
+ * this relation's target, and collect all vars referenced by these
+ * aggregates / grouping expressions;
+ */
+ found_other_rel_agg = false;
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which includes
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * init_grouping_targets will handle plain Var grouping
+ * expressions because it needs to look them up in
+ * grouped_var_list anyway.
+ */
+ if (IsA(gvi->gvexpr, Var))
+ continue;
+
+ /*
+ * Currently, GroupedVarInfo only handles Vars and Aggrefs.
+ */
+ Assert(IsA(gvi->gvexpr, Aggref));
+
+ /* We only derive grouped expressions using ECs, not aggregates */
+ Assert(!gvi->derived);
+
+ gvi->agg_partial = (Aggref *) copyObject(gvi->gvexpr);
+ mark_partial_aggref(gvi->agg_partial, AGGSPLIT_INITIAL_SERIAL);
+
+ /*
+ * Accept the aggregate.
+ */
+ aggregates = lappend(aggregates, gvi);
+ }
+ else if (IsA(gvi->gvexpr, Aggref))
+ {
+ /*
+ * Remember that there is at least one aggregate expression that
+ * needs something else than this rel.
+ */
+ found_other_rel_agg = true;
+
+ /*
+ * This condition effectively terminates creation of the
+ * RelAggInfo, so there's no reason to check the next
+ * GroupedVarInfo.
+ */
+ break;
+ }
+ }
+
+ /*
+ * Grouping makes little sense w/o aggregate function and w/o grouping
+ * expressions.
+ */
+ if (aggregates == NIL)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Give up if some other aggregate(s) need relations other than the
+ * current one.
+ *
+ * If the aggregate needs the current rel plus anything else, then the
+ * problem is that grouping of the current relation could make some input
+ * variables unavailable for the "higher aggregate", and it'd also
+ * decrease the number of input rows the "higher aggregate" receives.
+ *
+ * If the aggregate does not even need the current rel, then neither the
+ * current rel nor anything else should be grouped because we do not
+ * support join of two grouped relations.
+ */
+ if (found_other_rel_agg)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Create target for grouped paths as well as one for the input paths of
+ * the aggregation paths.
+ */
+ target = create_empty_pathtarget();
+ agg_input = create_empty_pathtarget();
+
+ /*
+ * Cannot suitable targets for the aggregation push-down be derived?
+ */
+ if (!init_grouping_targets(root, rel, target, agg_input, gvis,
+ &grp_exprs_extra))
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ list_free(gvis);
+
+ /*
+ * Aggregation push-down makes no sense w/o grouping expressions.
+ */
+ if ((list_length(target->exprs) + list_length(grp_exprs_extra)) == 0)
+ return NULL;
+
+ group_clauses_final = root->parse->groupClause;
+
+ /*
+ * If the aggregation target should have extra grouping expressions (in
+ * order to emit input vars for join conditions), add them now. This step
+ * includes assignment of tleSortGroupRef's which we can generate now.
+ */
+ if (list_length(grp_exprs_extra) > 0)
+ {
+ Index sortgroupref;
+
+ /*
+ * We'll have to add some clauses, but query group clause must be
+ * preserved.
+ */
+ group_clauses_final = list_copy(group_clauses_final);
+
+ /*
+ * Always start at root->max_sortgroupref. The extra grouping
+ * expressions aren't used during the final aggregation, so the
+ * sortgroupref values don't need to be unique across the query. Thus
+ * we don't have to increase root->max_sortgroupref, which makes
+ * recognition of the extra grouping expressions pretty easy.
+ */
+ sortgroupref = root->max_sortgroupref;
+
+ /*
+ * Generate the SortGroupClause's and add the expressions to the
+ * target.
+ */
+ foreach(lc, grp_exprs_extra)
+ {
+ Var *var = lfirst_node(Var, lc);
+ SortGroupClause *cl = makeNode(SortGroupClause);
+
+ /*
+ * Initialize the SortGroupClause.
+ *
+ * As the final aggregation will not use this grouping expression,
+ * we don't care whether sortop is < or >. The value of
+ * nulls_first should not matter for the same reason.
+ */
+ cl->tleSortGroupRef = ++sortgroupref;
+ get_sort_group_operators(var->vartype,
+ false, true, false,
+ &cl->sortop, &cl->eqop, NULL,
+ &cl->hashable);
+ group_clauses_final = lappend(group_clauses_final, cl);
+ add_column_to_pathtarget(target, (Expr *) var,
+ cl->tleSortGroupRef);
+
+ /*
+ * The aggregation input target must emit this var too.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) var,
+ cl->tleSortGroupRef);
+ }
+ }
+
+ /*
+ * Add aggregates to the grouping target.
+ */
+ add_aggregates_to_target(root, target, aggregates);
+
+ /*
+ * Build a list of grouping expressions and a list of the corresponding
+ * SortGroupClauses.
+ */
+ i = 0;
+ foreach(lc, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(lc);
+
+ if (IsA(texpr, Aggref))
+ {
+ /*
+ * Once we see Aggref, no grouping expressions should follow.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides being an aggregate, the target expression should have no
+ * other reason then being a column of a relation functionally
+ * dependent on the GROUP BY clause. So it's not actually a grouping
+ * column.
+ */
+ if (sortgroupref == 0)
+ continue;
+
+ /*
+ * group_clause_final contains the "local" clauses, so this search
+ * should succeed.
+ */
+ cl = get_sortgroupref_clause(sortgroupref, group_clauses_final);
+
+ result->group_clauses = list_append_unique(result->group_clauses,
+ cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ result->group_exprs = list_append_unique(result->group_exprs,
+ texpr);
+ }
+
+ /*
+ * Since neither target nor agg_input is supposed to be identical to the
+ * source reltarget, compute the width and cost again.
+ *
+ * target does not yet contain aggregates, but these will be accounted by
+ * AggPath.
+ */
+ set_pathtarget_cost_width(root, target);
+ set_pathtarget_cost_width(root, agg_input);
+
+ result->relids = bms_copy(rel->relids);
+ result->target = target;
+ result->agg_input = agg_input;
+
+ /* Finally collect the aggregates. */
+ while (lc != NULL)
+ {
+ Aggref *aggref = lfirst_node(Aggref, lc);
+
+ /*
+ * Partial aggregation is what the grouped paths should do.
+ */
+ result->agg_exprs = lappend(result->agg_exprs, aggref);
+ lc = lnext(lc);
+ }
+
+ /* The "input_rows" field should be set by caller. */
+ return result;
+}
+
+/*
+ * Initialize target for grouped paths (target) as well as a target for paths
+ * that generate input for aggregation (agg_input).
+ *
+ * group_exprs_extra_p receives a list of Var nodes for which we need to
+ * construct SortGroupClause. Those vars will then be used as additional
+ * grouping expressions, for the sake of join clauses.
+ *
+ * gvis a list of GroupedVarInfo's possibly useful for rel.
+ *
+ * Return true iff the targets could be initialized.
+ */
+static bool
+init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p)
+{
+ ListCell *lc1,
+ *lc2;
+ List *unresolved = NIL;
+ List *unresolved_sortgrouprefs = NIL;
+
+ foreach(lc1, rel->reltarget->exprs)
+ {
+ Var *tvar;
+ bool is_grouping;
+ Index sortgroupref = 0;
+ bool derived = false;
+ bool needed_by_aggregate;
+
+ /*
+ * Given that PlaceHolderVar currently prevents us from doing
+ * aggregation push-down, the source target cannot contain anything
+ * more complex than a Var.
+ */
+ tvar = lfirst_node(Var, lc1);
+
+ is_grouping = is_grouping_expression(gvis, (Expr *) tvar,
+ &sortgroupref, &derived);
+
+ /*
+ * Derived grouping expressions should not be referenced by the query
+ * targetlist, so let them fall into vars_unresolved. It'll be checked
+ * later if the current targetlist needs them. For example, we should
+ * not automatically use Var as a grouping expression if the only
+ * reason for it to be in the plain relation target is that it's
+ * referenced by aggregate argument, and it happens to be in the same
+ * EC as any grouping expression.
+ */
+ if (is_grouping && !derived)
+ {
+ Assert(sortgroupref > 0);
+
+ /*
+ * It's o.k. to use the target expression for grouping.
+ */
+ add_column_to_pathtarget(target, (Expr *) tvar, sortgroupref);
+
+ /*
+ * As for agg_input, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) tvar, sortgroupref);
+
+ /* Process the next expression. */
+ continue;
+ }
+
+ /*
+ * Is this Var needed in the query targetlist for anything else than
+ * aggregate input?
+ */
+ needed_by_aggregate = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc2);
+ ListCell *lc3;
+ List *vars;
+
+ if (!IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (!bms_is_member(tvar->varno, gvi->gv_eval_at))
+ continue;
+
+ /*
+ * XXX Consider some sort of caching.
+ */
+ vars = pull_var_clause((Node *) gvi->gvexpr, PVC_RECURSE_AGGREGATES);
+ foreach(lc3, vars)
+ {
+ Var *var = lfirst_node(Var, lc3);
+
+ if (equal(var, tvar))
+ {
+ needed_by_aggregate = true;
+ break;
+ }
+ }
+ list_free(vars);
+ if (needed_by_aggregate)
+ break;
+ }
+
+ if (needed_by_aggregate)
+ {
+ bool found = false;
+
+ foreach(lc2, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc2);
+
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ if (equal(te->expr, tvar))
+ {
+ found = true;
+ break;
+ }
+ }
+
+ /*
+ * If it's only Aggref input, add it to the aggregation input
+ * target and that's it.
+ */
+ if (!found)
+ {
+ add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
+ continue;
+ }
+ }
+
+ /*
+ * Further investigation involves dependency check, for which we need
+ * to have all the (plain-var) grouping expressions gathered.
+ */
+ unresolved = lappend(unresolved, tvar);
+ unresolved_sortgrouprefs = lappend_int(unresolved_sortgrouprefs,
+ sortgroupref);
+ }
+
+ /*
+ * Check for other possible reasons for the var to be in the plain target.
+ */
+ forboth(lc1, unresolved, lc2, unresolved_sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, lc1);
+ Index sortgroupref = lfirst_int(lc2);
+ RangeTblEntry *rte;
+ List *deps = NIL;
+ Relids relids_subtract;
+ int ndx;
+ RelOptInfo *baserel;
+
+ rte = root->simple_rte_array[var->varno];
+
+ /*
+ * Check if the Var can be in the grouping key even though it's not
+ * mentioned by the GROUP BY clause (and could not be derived using
+ * ECs).
+ */
+ if (sortgroupref == 0 &&
+ check_functional_grouping(rte->relid, var->varno,
+ var->varlevelsup,
+ target->exprs, &deps))
+ {
+ /*
+ * The var shouldn't be actually used as a grouping key (instead,
+ * the one this depends on will be), so sortgroupref should not be
+ * important.
+ */
+ add_new_column_to_pathtarget(target, (Expr *) var);
+ add_new_column_to_pathtarget(agg_input, (Expr *) var);
+
+ /*
+ * The var may or may not be present in generic grouping
+ * expression(s) in addition, but this is handled elsewhere.
+ */
+ continue;
+ }
+
+ /*
+ * Isn't the expression needed by joins above the current rel?
+ *
+ * The relids we're not interested in do include 0, which is the
+ * top-level targetlist. The only reason for relids to contain 0
+ * should be that arg_var is referenced either by aggregate or by
+ * grouping expression, but right now we're interested in the *other*
+ * reasons. (As soon aggregation is pushed down, the aggregates in the
+ * query targetlist no longer need direct reference to arg_var
+ * anyway.)
+ */
+ relids_subtract = bms_copy(rel->relids);
+ bms_add_member(relids_subtract, 0);
+
+ baserel = find_base_rel(root, var->varno);
+ ndx = var->varattno - baserel->min_attr;
+ if (bms_nonempty_difference(baserel->attr_needed[ndx],
+ relids_subtract))
+ {
+ /*
+ * The variable is needed by a join involving this relation. That
+ * case includes variable that is referenced by a generic grouping
+ * expression.
+ *
+ * The only way to bring this var to the aggregation output is to
+ * add it to the grouping expressions too.
+ */
+ if (sortgroupref > 0)
+ {
+ /*
+ * The var could be recognized as a potentially useful
+ * grouping expression at the top of the loop, so we can add
+ * it to the grouping target, as well as to the agg_input.
+ */
+ add_column_to_pathtarget(target, (Expr *) var, sortgroupref);
+ add_column_to_pathtarget(agg_input, (Expr *) var, sortgroupref);
+ }
+ else
+ {
+ /*
+ * Since root->parse->groupClause is not supposed to contain
+ * this expression, we need to construct special
+ * SortGroupClause. Its tleSortGroupRef needs to be unique
+ * within target_agg, so postpone creation of the
+ * SortGroupRefs until we're done with the iteration of
+ * rel->reltarget->exprs.
+ */
+ *group_exprs_extra_p = lappend(*group_exprs_extra_p, var);
+ }
+ }
+ else
+ {
+ /*
+ * As long as the query is semantically correct, arriving here
+ * means that the var is referenced by a generic grouping
+ * expression but not referenced by any join.
+ *
+ * create_rel_agg_info() should add this variable to "agg_input"
+ * target and also add the whole generic expression to "target",
+ * but that's subject to future enhancement of the aggregate
+ * push-down feature.
+ */
+ return false;
+ }
+ }
+
+ return true;
+}
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
index 41402821a1..c7105dfc2a 100644
--- a/src/backend/optimizer/util/tlist.c
+++ b/src/backend/optimizer/util/tlist.c
@@ -826,6 +826,62 @@ apply_pathtarget_labeling_to_tlist(List *tlist, PathTarget *target)
}
}
+/*
+ * For each aggregate grouping expression or aggregate to the grouped target.
+ *
+ * Caller passes the expressions in the form of GroupedVarInfos so that we
+ * don't have to look for gvid.
+ */
+void
+add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions)
+{
+ ListCell *lc;
+
+ /* Create the vars and add them to the target. */
+ foreach(lc, expressions)
+ {
+ GroupedVarInfo *gvi;
+
+ gvi = lfirst_node(GroupedVarInfo, lc);
+ add_column_to_pathtarget(target, (Expr *) gvi->agg_partial,
+ gvi->sortgroupref);
+ }
+}
+
+/*
+ * Find out if expr can be used as grouping expression in reltarget.
+ *
+ * sortgroupref and is_derived reflect the ->sortgroupref and ->derived fields
+ * of the corresponding GroupedVarInfo.
+ */
+bool
+is_grouping_expression(List *gvis, Expr *expr, Index *sortgroupref,
+ bool *is_derived)
+{
+ ListCell *lc;
+
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (equal(gvi->gvexpr, expr))
+ {
+ Assert(gvi->sortgroupref > 0);
+
+ *sortgroupref = gvi->sortgroupref;
+ *is_derived = gvi->derived;
+ return true;
+ }
+ }
+
+ /* The expression cannot be used as grouping key. */
+ return false;
+}
+
/*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index fc463601ff..8c1929c758 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -1021,6 +1021,15 @@ static struct config_bool ConfigureNamesBool[] =
false,
NULL, NULL, NULL
},
+ {
+ {"enable_agg_pushdown", PGC_USERSET, QUERY_TUNING_METHOD,
+ gettext_noop("Enables aggregation push-down."),
+ NULL
+ },
+ &enable_agg_pushdown,
+ false,
+ NULL, NULL, NULL
+ },
{
{"enable_parallel_append", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables the planner's use of parallel append plans."),
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 11027cdb10..25d058ef2a 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -224,6 +224,7 @@ typedef enum NodeTag
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_RelAggInfo,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
@@ -268,6 +269,7 @@ typedef enum NodeTag
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 38dc186623..988d6220c8 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -240,6 +240,23 @@ struct PlannerInfo
*/
struct RelInfoList *join_rel_list; /* list of join-relation RelOptInfos */
+ /*
+ * grouped_rel_list is a list of RelOptInfos that represent grouped
+ * relations, both base relations and joins. Unlike join_rel_list, base
+ * relations are accepted because grouped base relation is of rather
+ * limited scope, i.e. it's only needed during join search. Thus it does
+ * not deserve separate storage like simple_rel_array.
+ */
+ struct RelInfoList *grouped_rel_list; /* list of grouped relation
+ * RelOptInfos */
+
+ /*
+ * agg_info_list contains one instance of RelAggInfo per an item of
+ * grouped_rel_list.
+ */
+ struct RelInfoList *agg_info_list; /* list of grouped relation
+ * RelAggInfos */
+
/*
* When doing a dynamic-programming-style join search, join_rel_level[k]
* is a list of all join-relation RelOptInfos of level k, and
@@ -285,6 +302,8 @@ struct PlannerInfo
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
@@ -316,6 +335,12 @@ struct PlannerInfo
*/
List *processed_tlist;
+ /*
+ * The maximum ressortgroupref among target entries in processed_list.
+ * Useful when adding extra grouping expressions for partial aggregation.
+ */
+ int max_sortgroupref;
+
/* Fields filled during create_plan() for use in setrefs.c */
AttrNumber *grouping_map; /* for GroupingFunc fixup */
List *minmax_aggs; /* List of MinMaxAggInfos */
@@ -754,6 +779,60 @@ typedef struct RelInfoList
struct HTAB *hash;
} RelInfoList;
+/*
+ * RelAggInfo
+ * Information needed to create grouped paths for base rels and joins.
+ *
+ * "relids" is the set of base-relation identifiers, just like with
+ * RelOptInfo.
+ *
+ * "target" will be used as pathtarget if partial aggregation is applied to
+ * base relation or join. The same target will also --- if the relation is a
+ * join --- be used to joinin grouped path to a non-grouped one. This target
+ * can contain plain-Var grouping expressions and Aggref nodes.
+ *
+ * Note: There's a convention that Aggref expressions are supposed to follow
+ * the other expressions of the target. Iterations of ->exprs may rely on this
+ * arrangement.
+ *
+ * "agg_input" contains Vars used either as grouping expressions or aggregate
+ * arguments. Paths providing the aggregation plan with input data should use
+ * this target. The only difference from reltarget of the non-grouped relation
+ * is that some items can have sortgroupref initialized.
+ *
+ * "input_rows" is the estimated number of input rows for AggPath. It's
+ * actually just a workspace for users of the structure, i.e. not initialized
+ * when instance of the structure is created.
+ *
+ * "group_clauses" and "group_exprs" are lists of SortGroupClause and the
+ * corresponding grouping expressions respectively.
+ *
+ * "agg_exprs" is a list of Aggref nodes for the aggregation of the relation's
+ * paths.
+ *
+ * "rel_grouped" is the relation containing the partially aggregated paths.
+ */
+typedef struct RelAggInfo
+{
+ NodeTag type;
+
+ Relids relids; /* Base rels contained in this grouped rel. */
+
+ struct PathTarget *target; /* Target for grouped paths. */
+
+ struct PathTarget *agg_input; /* pathtarget of paths that generate input
+ * for aggregation paths. */
+
+ double input_rows;
+
+ List *group_clauses;
+ List *group_exprs;
+
+ List *agg_exprs; /* Aggref expressions. */
+
+ RelOptInfo *rel_grouped; /* Grouped relation. */
+} RelAggInfo;
+
/*
* IndexOptInfo
* Per-index information for planning/optimization
@@ -2272,6 +2351,26 @@ typedef struct PlaceHolderInfo
int32 ph_width; /* estimated attribute width */
} PlaceHolderInfo;
+/*
+ * GroupedVarInfo exists for each expression that can be used as an aggregate
+ * or grouping expression evaluated below a join.
+ */
+typedef struct GroupedVarInfo
+{
+ NodeTag type;
+
+ Expr *gvexpr; /* the represented expression. */
+ Aggref *agg_partial; /* if gvexpr is aggregate, agg_partial is the
+ * corresponding partial aggregate */
+ Index sortgroupref; /* If gvexpr is a grouping expression, this is
+ * the tleSortGroupRef of the corresponding
+ * SortGroupClause. */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+ bool derived; /* derived from another GroupedVarInfo using
+ * equeivalence classes? */
+} GroupedVarInfo;
+
/*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
index 2f9aeec4a7..90f6995dc2 100644
--- a/src/include/optimizer/clauses.h
+++ b/src/include/optimizer/clauses.h
@@ -55,4 +55,6 @@ extern void CommuteOpExpr(OpExpr *clause);
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
+extern GroupedVarInfo *translate_expression_to_rels(PlannerInfo *root,
+ GroupedVarInfo *gvi, Index relid);
#endif /* CLAUSES_H */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 182ffeef4b..2d05ff5a15 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -210,6 +210,14 @@ extern AggPath *create_agg_path(PlannerInfo *root,
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+extern AggPath *create_agg_sorted_path(PlannerInfo *root,
+ RelOptInfo *rel,
+ Path *subpath,
+ RelAggInfo *agg_info);
+extern AggPath *create_agg_hashed_path(PlannerInfo *root,
+ RelOptInfo *rel,
+ Path *subpath,
+ RelAggInfo *agg_info);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
@@ -281,14 +289,21 @@ extern void setup_append_rel_array(PlannerInfo *root);
extern void expand_planner_arrays(PlannerInfo *root, int add_size);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
+extern RelOptInfo *build_simple_grouped_rel(PlannerInfo *root, int relid,
+ RelAggInfo **agg_info_p);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
+extern void add_grouped_rel(PlannerInfo *root, RelOptInfo *rel,
+ RelAggInfo *agg_info);
+extern RelOptInfo *find_grouped_rel(PlannerInfo *root, Relids relids,
+ RelAggInfo **agg_info_p);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr);
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info);
extern Relids min_join_parameterization(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
@@ -314,5 +329,5 @@ extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo, JoinType jointype);
-
+extern RelAggInfo *create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index 7345137d1d..3006ebf5aa 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -21,6 +21,7 @@
* allpaths.c
*/
extern PGDLLIMPORT bool enable_geqo;
+extern PGDLLIMPORT bool enable_agg_pushdown;
extern PGDLLIMPORT int geqo_threshold;
extern PGDLLIMPORT int min_parallel_table_scan_size;
extern PGDLLIMPORT int min_parallel_index_scan_size;
@@ -52,8 +53,14 @@ extern RelOptInfo *make_one_rel(PlannerInfo *root, List *joinlist);
extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
List *initial_rels);
+
extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
+extern void generate_grouping_paths(PlannerInfo *root,
+ RelOptInfo *rel_grouped,
+ RelOptInfo *rel_plain,
+ RelAggInfo *agg_info);
+
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages, int max_workers);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index e7aaddd50d..761f610a96 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -28,7 +28,8 @@ typedef void (*query_pathkeys_callback) (PlannerInfo *root, void *extra);
* prototypes for plan/planmain.c
*/
extern RelOptInfo *query_planner(PlannerInfo *root,
- query_pathkeys_callback qp_callback, void *qp_extra);
+ query_pathkeys_callback qp_callback, void *qp_extra,
+ RelOptInfo **final_rel_grouped_p);
/*
* prototypes for plan/planagg.c
@@ -69,6 +70,7 @@ extern void add_other_rels_to_query(PlannerInfo *root);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
Relids where_needed, bool create_new_ph);
+extern void setup_aggregate_pushdown(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
index 8a2378f968..e48f37bac2 100644
--- a/src/include/optimizer/tlist.h
+++ b/src/include/optimizer/tlist.h
@@ -50,8 +50,14 @@ extern void split_pathtarget_at_srfs(PlannerInfo *root,
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+/* TODO Find the best location (position and in some cases even file) for the
+ * following ones. */
+extern void add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions);
+extern bool is_grouping_expression(List *gvis, Expr *expr,
+ Index *sortgroupref, bool *is_derived);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
-
#endif /* TLIST_H */
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
new file mode 100644
index 0000000000..786f4f3cff
--- /dev/null
+++ b/src/test/regress/expected/agg_pushdown.out
@@ -0,0 +1,214 @@
+BEGIN;
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+CREATE INDEX ON agg_pushdown_child1(parent);
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+COMMIT;
+ANALYZE;
+SET enable_agg_pushdown TO on;
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ Index Cond: (parent = i)
+ -> Index Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(9 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Hash Join
+ Hash Cond: (c1.parent = p.i)
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ Index Cond: (parent = i)
+ -> Hash
+ -> Seq Scan on agg_pushdown_parent p
+(10 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ Index Cond: (parent = i)
+ -> Sort
+ Sort Key: p.i
+ -> Seq Scan on agg_pushdown_parent p
+(11 rows)
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ Index Cond: (parent = i)
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(9 rows)
+
+SET enable_seqscan TO on;
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize HashAggregate
+ Group Key: p.i
+ -> Hash Join
+ Hash Cond: (p.x = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ Index Cond: (parent = x)
+(10 rows)
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: ((k = c1.j) AND (parent = c1.parent))
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(13 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+----------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Hash
+ -> Seq Scan on agg_pushdown_child2 c2
+(15 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: ((c1.j = c2.k) AND (c1.parent = c2.parent))
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(13 rows)
+
diff --git a/src/test/regress/expected/sysviews.out b/src/test/regress/expected/sysviews.out
index a1c90eb905..55cfd3fff7 100644
--- a/src/test/regress/expected/sysviews.out
+++ b/src/test/regress/expected/sysviews.out
@@ -72,6 +72,7 @@ select count(*) >= 0 as ok from pg_prepared_xacts;
select name, setting from pg_settings where name like 'enable%';
name | setting
--------------------------------+---------
+ enable_agg_pushdown | off
enable_bitmapscan | on
enable_gathermerge | on
enable_hashagg | on
@@ -89,7 +90,7 @@ select name, setting from pg_settings where name like 'enable%';
enable_seqscan | on
enable_sort | on
enable_tidscan | on
-(17 rows)
+(18 rows)
-- Test that the pg_timezone_names and pg_timezone_abbrevs views are
-- more-or-less working. We can't test their contents in any great detail
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 8fb55f045e..5505bfde7e 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -88,6 +88,9 @@ test: rules psql psql_crosstab amutils stats_ext
test: select_parallel
test: write_parallel
+# this one runs parallel workers too
+test: agg_pushdown
+
# no relation related tests can be put in this group
test: publication subscription
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index a39ca1012a..4f0ba643db 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -140,6 +140,7 @@ test: amutils
test: stats_ext
test: select_parallel
test: write_parallel
+test: agg_pushdown
test: publication
test: subscription
test: select_views
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
new file mode 100644
index 0000000000..6cf2ed9c20
--- /dev/null
+++ b/src/test/regress/sql/agg_pushdown.sql
@@ -0,0 +1,117 @@
+BEGIN;
+
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+
+CREATE INDEX ON agg_pushdown_child1(parent);
+
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+COMMIT;
+ANALYZE;
+
+SET enable_agg_pushdown TO on;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_seqscan TO on;
+
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
--
2.16.4
Re: [HACKERS] WIP: Aggregation push-down
On Tue, Jul 9, 2019 at 9:47 PM Antonin Houska <ah@cybertec.at> wrote:
Richard Guo <riguo@pivotal.io> wrote:
Another rebase is needed for the patches.
Done.
I didn't fully follow the whole thread and mainly looked into the latest
patch set. So what are the considerations for abandoning the aggmultifn
concept? In my opinion, aggmultifn would enable us to do a lot more
types of transformation. For example, consider the query below:
select sum(foo.c) from foo join bar on foo.b = bar.b group by foo.a, bar.a;
With the latest patch, the plan looks like:
Finalize HashAggregate <------ sum(psum)
Group Key: foo.a, bar.a
-> Hash Join
Hash Cond: (bar.b = foo.b)
-> Seq Scan on bar
-> Hash
-> Partial HashAggregate <------ sum(foo.c) as psum
Group Key: foo.a, foo.b
-> Seq Scan on foo
If we have aggmultifn, we can perform the query this way:
Finalize HashAggregate <------ sum(foo.c)*cnt
Group Key: foo.a, bar.a
-> Hash Join
Hash Cond: (foo.b = bar.b)
-> Seq Scan on foo
-> Hash
-> Partial HashAggregate <------ count(*) as cnt
Group Key: bar.a, bar.b
-> Seq Scan on bar
And this way:
Finalize HashAggregate <------ sum(psum)*cnt
Group Key: foo.a, bar.a
-> Hash Join
Hash Cond: (foo.b = bar.b)
-> Partial HashAggregate <------ sum(foo.c) as psum
Group Key: foo.a, foo.b
-> Seq Scan on foo
-> Hash
-> Partial HashAggregate <------ count(*) as cnt
Group Key: bar.a, bar.b
-> Seq Scan on bar
My another question is in function add_grouped_path(), when creating
sorted aggregation path on top of subpath. If the subpath is not sorted,
then the sorted aggregation path would not be generated. Why not in this
case we create a sort path on top of subpath first and then create group
aggregation path on top of the sort path?
Core dump when running one query in agg_pushdown.sql
EXPLAIN ANALYZE
SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
AS c1 ON c1.parent = p.i GROUP BY p.i;
#0 0x00000000006def98 in CheckVarSlotCompatibility (slot=0x0, attnum=1,
vartype=23) at execExprInterp.c:1850
#1 0x00000000006def5d in CheckExprStillValid (state=0x2b63a28,
econtext=0x2ba4958) at execExprInterp.c:1814
#2 0x00000000006dee38 in ExecInterpExprStillValid (state=0x2b63a28,
econtext=0x2ba4958, isNull=0x7fff7cd16a37) at execExprInterp.c:1763
#3 0x00000000007144dd in ExecEvalExpr (state=0x2b63a28,
econtext=0x2ba4958, isNull=0x7fff7cd16a37)
at ../../../src/include/executor/executor.h:288
#4 0x0000000000715475 in ExecIndexEvalRuntimeKeys (econtext=0x2ba4958,
runtimeKeys=0x2b63910, numRuntimeKeys=1) at nodeIndexscan.c:630
#5 0x000000000071533b in ExecReScanIndexScan (node=0x2b62bf8) at
nodeIndexscan.c:568
#6 0x00000000006d4ce6 in ExecReScan (node=0x2b62bf8) at execAmi.c:182
#7 0x00000000007152a0 in ExecIndexScan (pstate=0x2b62bf8) at
nodeIndexscan.c:530
This is really a cool feature. Thank you for working on this.
Thanks
Richard
Re: [HACKERS] WIP: Aggregation push-down
Richard Guo <riguo@pivotal.io> wrote:
I didn't fully follow the whole thread and mainly looked into the
latest
patch set. So what are the considerations for abandoning the
aggmultifn
concept?
Originally the function was there to support join where both relations are
grouped. My doubts about usefulness of such join started at [1]/messages/by-id/cc823e89-3fbc-f94e-b9d4-9c713b044b5d@2ndquadrant.com. (See the
thread referenced from [2]/messages/by-id/9666.1491295317@localhost.)
In my opinion, aggmultifn would enable us to do a lot more
types of transformation. For example, consider the query below:select sum(foo.c) from foo join bar on foo.b = bar.b group by foo.a,
bar.a;With the latest patch, the plan looks like:
Finalize HashAggregate <------ sum(psum)
Group Key: foo.a, bar.a
-> Hash Join
Hash Cond: (bar.b = foo.b)
-> Seq Scan on bar
-> Hash
-> Partial HashAggregate <------ sum(foo.c) as
psum
Group Key: foo.a, foo.b
-> Seq Scan on fooIf we have aggmultifn, we can perform the query this way:
Finalize HashAggregate <------ sum(foo.c)*cnt
Group Key: foo.a, bar.a
-> Hash Join
Hash Cond: (foo.b = bar.b)
-> Seq Scan on foo
-> Hash
-> Partial HashAggregate <------ count(*) as cnt
Group Key: bar.a, bar.b
-> Seq Scan on bar
Perhaps, but it that would require changes to nodeAgg.c, which I want to avoid
for now.
And this way:
Finalize HashAggregate <------ sum(psum)*cnt
Group Key: foo.a, bar.a
-> Hash Join
Hash Cond: (foo.b = bar.b)
-> Partial HashAggregate <------ sum(foo.c) as
psum
Group Key: foo.a, foo.b
-> Seq Scan on foo
-> Hash
-> Partial HashAggregate <------ count(*) as cnt
Group Key: bar.a, bar.b
-> Seq Scan on bar
This looks like my idea presented in [1]/messages/by-id/cc823e89-3fbc-f94e-b9d4-9c713b044b5d@2ndquadrant.com, for which there seems to be no
common use case.
My another question is in function add_grouped_path(), when creating
sorted aggregation path on top of subpath. If the subpath is not
sorted,
then the sorted aggregation path would not be generated. Why not in
this
case we create a sort path on top of subpath first and then create
group
aggregation path on top of the sort path?
I see no reason not to do it. (If you want to try, just go ahead.) However the
current patch version brings only the basic functionality and I'm not going to
add new functionality (such as parallel aggregation, partitioned tables or
postgres_fdw) until the open design questions are resolved. Otherwise there's
a risk that the additional work will be wasted due to major rework of the core
functionality.
Core dump when running one query in agg_pushdown.sql
Thanks for the report! Fixed in the new version.
This is really a cool feature. Thank you for working on this.
I appreciate to hear that :-) Let's see which postgres release will adopt it.
[1]: /messages/by-id/cc823e89-3fbc-f94e-b9d4-9c713b044b5d@2ndquadrant.com
[2]: /messages/by-id/9666.1491295317@localhost
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
Attachments:
v13-0001-Introduce-RelInfoList-structure.patchtext/x-diffDownload
From 5479312e35a0a6fa9b11318e97a626b15d98794c Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Fri, 12 Jul 2019 10:04:31 +0200
Subject: [PATCH 1/3] Introduce RelInfoList structure.
---
contrib/postgres_fdw/postgres_fdw.c | 3 +-
doc/src/sgml/catalogs.sgml | 5 +
doc/src/sgml/ddl.sgml | 9 --
doc/src/sgml/func.sgml | 81 ++++++-------
doc/src/sgml/json.sgml | 44 ++++----
src/backend/access/gist/gistbuildbuffers.c | 5 +-
src/backend/commands/copy.c | 19 ++--
src/backend/commands/extension.c | 16 ---
src/backend/commands/tablecmds.c | 23 +---
src/backend/commands/trigger.c | 1 +
src/backend/executor/execMain.c | 1 -
src/backend/nodes/outfuncs.c | 11 ++
src/backend/optimizer/geqo/geqo_eval.c | 12 +-
src/backend/optimizer/plan/planmain.c | 3 +-
src/backend/optimizer/util/relnode.c | 157 ++++++++++++++++----------
src/backend/partitioning/partprune.c | 50 +++-----
src/backend/tcop/postgres.c | 30 +----
src/bin/initdb/initdb.c | 2 +-
src/bin/pg_basebackup/pg_recvlogical.c | 5 +-
src/bin/pg_checksums/pg_checksums.c | 4 +-
src/bin/pg_dump/pg_backup_db.c | 4 +-
src/bin/pg_dump/pg_dumpall.c | 6 +-
src/bin/pg_upgrade/option.c | 2 +-
src/include/access/tableam.h | 12 +-
src/include/nodes/nodes.h | 1 +
src/include/nodes/pathnodes.h | 28 +++--
src/test/regress/expected/partition_prune.out | 15 +--
src/test/regress/expected/triggers.out | 24 ----
src/test/regress/sql/partition_prune.sql | 11 +-
src/test/regress/sql/triggers.sql | 23 ----
30 files changed, 248 insertions(+), 359 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 033aeb2556..90414f1168 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -5205,7 +5205,8 @@ foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
*/
Assert(fpinfo->relation_index == 0); /* shouldn't be set yet */
fpinfo->relation_index =
- list_length(root->parse->rtable) + list_length(root->join_rel_list);
+ list_length(root->parse->rtable) +
+ list_length(root->join_rel_list->items);
return true;
}
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 68ad5071ca..3428a7c0fa 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -9995,6 +9995,11 @@ SELECT * FROM pg_locks pl LEFT JOIN pg_prepared_xacts ppx
that blanks out the password field.
</para>
+ <para>
+ This view explicitly exposes the OID column of the underlying table,
+ since that is needed to do joins to other catalogs.
+ </para>
+
<table>
<title><structname>pg_roles</structname> Columns</title>
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 9301f0227d..ed2d9c60d5 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -4003,15 +4003,6 @@ ALTER INDEX measurement_city_id_logdate_key
</para>
</listitem>
- <listitem>
- <para>
- Unique constraints on partitioned tables must include all the
- partition key columns. This limitation exists because
- <productname>PostgreSQL</productname> can only enforce
- uniqueness in each partition individually.
- </para>
- </listitem>
-
<listitem>
<para>
<literal>BEFORE ROW</literal> triggers, if necessary, must be defined
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index a25c122ac8..185a184daa 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -11514,8 +11514,7 @@ table2-mapping
from the JSON data, similar to XPath expressions used
for SQL access to XML. In <productname>PostgreSQL</productname>,
path expressions are implemented as the <type>jsonpath</type>
- data type and can use any elements described in
- <xref linkend="datatype-jsonpath"/>.
+ data type, described in <xref linkend="datatype-jsonpath"/>.
</para>
<para>JSON query functions and operators
@@ -11562,7 +11561,7 @@ table2-mapping
},
{ "location": [ 47.706, 13.2635 ],
"start time": "2018-10-14 10:39:21",
- "HR": 135
+ "HR": 130
} ]
}
}
@@ -11614,33 +11613,23 @@ table2-mapping
<para>
When defining the path, you can also use one or more
- <firstterm>filter expressions</firstterm> that work similar to the
- <literal>WHERE</literal> clause in SQL. A filter expression begins with
- a question mark and provides a condition in parentheses:
-
- <programlisting>
-? (<replaceable>condition</replaceable>)
- </programlisting>
- </para>
-
- <para>
- Filter expressions must be specified right after the path evaluation step
- to which they are applied. The result of this step is filtered to include
- only those items that satisfy the provided condition. SQL/JSON defines
- three-valued logic, so the condition can be <literal>true</literal>, <literal>false</literal>,
+ <firstterm>filter expressions</firstterm>, which work similar to
+ the <literal>WHERE</literal> clause in SQL. Each filter expression
+ can provide one or more filtering conditions that are applied
+ to the result of the path evaluation. Each filter expression must
+ be enclosed in parentheses and preceded by a question mark.
+ Filter expressions are evaluated from left to right and can be nested.
+ The <literal>@</literal> variable denotes the current path evaluation
+ result to be filtered, and can be followed by one or more accessor
+ operators to define the JSON element by which to filter the result.
+ Functions and operators that can be used in the filtering condition
+ are listed in <xref linkend="functions-sqljson-filter-ex-table"/>.
+ SQL/JSON defines three-valued logic, so the result of the filter
+ expression may be <literal>true</literal>, <literal>false</literal>,
or <literal>unknown</literal>. The <literal>unknown</literal> value
- plays the same role as SQL <literal>NULL</literal> and can be tested
- for with the <literal>is unknown</literal> predicate. Further path
+ plays the same role as SQL <literal>NULL</literal>. Further path
evaluation steps use only those items for which filter expressions
- return <literal>true</literal>.
- </para>
-
- <para>
- Functions and operators that can be used in filter expressions are listed
- in <xref linkend="functions-sqljson-filter-ex-table"/>. The path
- evaluation result to be filtered is denoted by the <literal>@</literal>
- variable. To refer to a JSON element stored at a lower nesting level,
- add one or more accessor operators after <literal>@</literal>.
+ return true.
</para>
<para>
@@ -11654,8 +11643,8 @@ table2-mapping
<para>
To get the start time of segments with such values instead, you have to
filter out irrelevant segments before returning the start time, so the
- filter expression is applied to the previous step, and the path used
- in the condition is different:
+ filter is applied to the previous step and the path in the filtering
+ condition is different:
<programlisting>
'$.track.segments[*] ? (@.HR > 130)."start time"'
</programlisting>
@@ -11680,9 +11669,9 @@ table2-mapping
</para>
<para>
- You can also nest filter expressions within each other:
+ You can also nest filters within each other:
<programlisting>
-'$.track ? (exists(@.segments[*] ? (@.HR > 130))).segments.size()'
+'$.track ? (@.segments[*] ? (@.HR > 130)).segments.size()'
</programlisting>
This expression returns the size of the track if it contains any
segments with high heart rate values, or an empty sequence otherwise.
@@ -11965,14 +11954,14 @@ table2-mapping
<entry>Less-than operator</entry>
<entry><literal>[1, 2, 3]</literal></entry>
<entry><literal>$[*] ? (@ < 2)</literal></entry>
- <entry><literal>1</literal></entry>
+ <entry><literal>1, 2</literal></entry>
</row>
<row>
<entry><literal><=</literal></entry>
<entry>Less-than-or-equal-to operator</entry>
<entry><literal>[1, 2, 3]</literal></entry>
- <entry><literal>$[*] ? (@ <= 2)</literal></entry>
- <entry><literal>1, 2</literal></entry>
+ <entry><literal>$[*] ? (@ < 2)</literal></entry>
+ <entry><literal>1</literal></entry>
</row>
<row>
<entry><literal>></literal></entry>
@@ -11982,7 +11971,7 @@ table2-mapping
<entry><literal>3</literal></entry>
</row>
<row>
- <entry><literal>>=</literal></entry>
+ <entry><literal>></literal></entry>
<entry>Greater-than-or-equal-to operator</entry>
<entry><literal>[1, 2, 3]</literal></entry>
<entry><literal>$[*] ? (@ >= 2)</literal></entry>
@@ -12272,7 +12261,7 @@ table2-mapping
<row>
<entry><literal>@?</literal></entry>
<entry><type>jsonpath</type></entry>
- <entry>Does JSON path return any item for the specified JSON value?</entry>
+ <entry>Does JSON path returns any item for the specified JSON value?</entry>
<entry><literal>'{"a":[1,2,3,4,5]}'::jsonb @? '$.a[*] ? (@ > 2)'</literal></entry>
</row>
<row>
@@ -12300,8 +12289,8 @@ table2-mapping
<note>
<para>
The <literal>@?</literal> and <literal>@@</literal> operators suppress
- the following errors: lacking object field or array element, unexpected
- JSON item type, and numeric errors.
+ errors including: lacking object field or array element, unexpected JSON
+ item type and numeric errors.
This behavior might be helpful while searching over JSON document
collections of varying structure.
</para>
@@ -13157,17 +13146,17 @@ table2-mapping
<literal>jsonb_path_query</literal>, <literal>jsonb_path_query_array</literal> and
<literal>jsonb_path_query_first</literal>
functions have optional <literal>vars</literal> and <literal>silent</literal>
- arguments.
+ argument.
</para>
<para>
- If the <literal>vars</literal> argument is specified, it provides an
- object containing named variables to be substituted into a
- <literal>jsonpath</literal> expression.
+ When <literal>vars</literal> argument is specified, it constitutes an object
+ contained variables to be substituted into <literal>jsonpath</literal>
+ expression.
</para>
<para>
- If the <literal>silent</literal> argument is specified and has the
- <literal>true</literal> value, these functions suppress the same errors
- as the <literal>@?</literal> and <literal>@@</literal> operators.
+ When <literal>silent</literal> argument is specified and has
+ <literal>true</literal> value, the same errors are suppressed as it is in
+ the <literal>@?</literal> and <literal>@@</literal> operators.
</para>
</note>
diff --git a/doc/src/sgml/json.sgml b/doc/src/sgml/json.sgml
index 0d8e2c6de4..2aa98024ae 100644
--- a/doc/src/sgml/json.sgml
+++ b/doc/src/sgml/json.sgml
@@ -815,18 +815,21 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
<literal>.**{<replaceable>level</replaceable>}</literal>
</para>
<para>
- <literal>.**{<replaceable>start_level</replaceable> to
- <replaceable>end_level</replaceable>}</literal>
+ <literal>.**{<replaceable>lower_level</replaceable> to
+ <replaceable>upper_level</replaceable>}</literal>
+ </para>
+ <para>
+ <literal>.**{<replaceable>lower_level</replaceable> to
+ last}</literal>
</para>
</entry>
<entry>
<para>
- Same as <literal>.**</literal>, but with a filter over nesting
- levels of JSON hierarchy. Nesting levels are specified as integers.
- Zero level corresponds to the current object. To access the lowest
- nesting level, you can use the <literal>last</literal> keyword.
- This is a <productname>PostgreSQL</productname> extension of
- the SQL/JSON standard.
+ Same as <literal>.**</literal>, but with filter over nesting
+ level of JSON hierarchy. Levels are specified as integers.
+ Zero level corresponds to current object. This is a
+ <productname>PostgreSQL</productname> extension of the SQL/JSON
+ standard.
</para>
</entry>
</row>
@@ -838,22 +841,19 @@ SELECT jdoc->'guid', jdoc->'name' FROM api WHERE jdoc @> '{"tags": ["qu
</entry>
<entry>
<para>
- Array element accessor.
- <literal><replaceable>subscript</replaceable></literal> can be
- given in two forms: <literal><replaceable>index</replaceable></literal>
- or <literal><replaceable>start_index</replaceable> to <replaceable>end_index</replaceable></literal>.
- The first form returns a single array element by its index. The second
- form returns an array slice by the range of indexes, including the
- elements that correspond to the provided
- <replaceable>start_index</replaceable> and <replaceable>end_index</replaceable>.
+ Array element accessor. <literal><replaceable>subscript</replaceable></literal>
+ might be given in two forms: <literal><replaceable>expr</replaceable></literal>
+ or <literal><replaceable>lower_expr</replaceable> to <replaceable>upper_expr</replaceable></literal>.
+ The first form specifies single array element by its index. The second
+ form specified array slice by the range of indexes. Zero index
+ corresponds to the first array element.
</para>
<para>
- The specified <replaceable>index</replaceable> can be an integer, as
- well as an expression returning a single numeric value, which is
- automatically cast to integer. Zero index corresponds to the first
- array element. You can also use the <literal>last</literal> keyword
- to denote the last array element, which is useful for handling arrays
- of unknown length.
+ An expression in the subscript may be an integer,
+ numeric expression, or any other <literal>jsonpath</literal> expression
+ returning single numeric value. The <literal>last</literal> keyword
+ can be used in the expression denoting the last subscript in an array.
+ That's helpful for handling arrays of unknown length.
</para>
</entry>
</row>
diff --git a/src/backend/access/gist/gistbuildbuffers.c b/src/backend/access/gist/gistbuildbuffers.c
index 38f786848d..d71354140e 100644
--- a/src/backend/access/gist/gistbuildbuffers.c
+++ b/src/backend/access/gist/gistbuildbuffers.c
@@ -138,7 +138,6 @@ gistGetNodeBuffer(GISTBuildBuffers *gfbb, GISTSTATE *giststate,
nodeBuffer->pageBlocknum = InvalidBlockNumber;
nodeBuffer->pageBuffer = NULL;
nodeBuffer->queuedForEmptying = false;
- nodeBuffer->isTemp = false;
nodeBuffer->level = level;
/*
@@ -187,8 +186,8 @@ gistAllocateNewPageBuffer(GISTBuildBuffers *gfbb)
{
GISTNodeBufferPage *pageBuffer;
- pageBuffer = (GISTNodeBufferPage *) MemoryContextAllocZero(gfbb->context,
- BLCKSZ);
+ pageBuffer = (GISTNodeBufferPage *) MemoryContextAlloc(gfbb->context,
+ BLCKSZ);
pageBuffer->prev = InvalidBlockNumber;
/* Set page free space */
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index 4f04d122c3..f1161f0fee 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2518,8 +2518,7 @@ CopyMultiInsertBufferFlush(CopyMultiInsertInfo *miinfo,
* The buffer must be flushed before cleanup.
*/
static inline void
-CopyMultiInsertBufferCleanup(CopyMultiInsertInfo *miinfo,
- CopyMultiInsertBuffer *buffer)
+CopyMultiInsertBufferCleanup(CopyMultiInsertBuffer *buffer)
{
int i;
@@ -2535,9 +2534,6 @@ CopyMultiInsertBufferCleanup(CopyMultiInsertInfo *miinfo,
for (i = 0; i < MAX_BUFFERED_TUPLES && buffer->slots[i] != NULL; i++)
ExecDropSingleTupleTableSlot(buffer->slots[i]);
- table_finish_bulk_insert(buffer->resultRelInfo->ri_RelationDesc,
- miinfo->ti_options);
-
pfree(buffer);
}
@@ -2589,7 +2585,7 @@ CopyMultiInsertInfoFlush(CopyMultiInsertInfo *miinfo, ResultRelInfo *curr_rri)
buffer = (CopyMultiInsertBuffer *) linitial(miinfo->multiInsertBuffers);
}
- CopyMultiInsertBufferCleanup(miinfo, buffer);
+ CopyMultiInsertBufferCleanup(buffer);
miinfo->multiInsertBuffers = list_delete_first(miinfo->multiInsertBuffers);
}
}
@@ -2603,7 +2599,7 @@ CopyMultiInsertInfoCleanup(CopyMultiInsertInfo *miinfo)
ListCell *lc;
foreach(lc, miinfo->multiInsertBuffers)
- CopyMultiInsertBufferCleanup(miinfo, lfirst(lc));
+ CopyMultiInsertBufferCleanup(lfirst(lc));
list_free(miinfo->multiInsertBuffers);
}
@@ -3325,6 +3321,9 @@ CopyFrom(CopyState cstate)
{
if (!CopyMultiInsertInfoIsEmpty(&multiInsertInfo))
CopyMultiInsertInfoFlush(&multiInsertInfo, NULL);
+
+ /* Tear down the multi-insert buffer data */
+ CopyMultiInsertInfoCleanup(&multiInsertInfo);
}
/* Done, clean up */
@@ -3356,10 +3355,6 @@ CopyFrom(CopyState cstate)
target_resultRelInfo->ri_FdwRoutine->EndForeignInsert(estate,
target_resultRelInfo);
- /* Tear down the multi-insert buffer data */
- if (insertMethod != CIM_SINGLE)
- CopyMultiInsertInfoCleanup(&multiInsertInfo);
-
ExecCloseIndices(target_resultRelInfo);
/* Close all the partitioned tables, leaf partitions, and their indices */
@@ -3371,6 +3366,8 @@ CopyFrom(CopyState cstate)
FreeExecutorState(estate);
+ table_finish_bulk_insert(cstate->rel, ti_options);
+
return processed;
}
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index f7202cc9e7..59ca5cd5a9 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -717,21 +717,9 @@ execute_sql_string(const char *sql)
foreach(lc1, raw_parsetree_list)
{
RawStmt *parsetree = lfirst_node(RawStmt, lc1);
- MemoryContext per_parsetree_context,
- oldcontext;
List *stmt_list;
ListCell *lc2;
- /*
- * We do the work for each parsetree in a short-lived context, to
- * limit the memory used when there are many commands in the string.
- */
- per_parsetree_context =
- AllocSetContextCreate(CurrentMemoryContext,
- "execute_sql_string per-statement context",
- ALLOCSET_DEFAULT_SIZES);
- oldcontext = MemoryContextSwitchTo(per_parsetree_context);
-
/* Be sure parser can see any DDL done so far */
CommandCounterIncrement();
@@ -784,10 +772,6 @@ execute_sql_string(const char *sql)
PopActiveSnapshot();
}
-
- /* Clean up per-parsetree context. */
- MemoryContextSwitchTo(oldcontext);
- MemoryContextDelete(per_parsetree_context);
}
/* Be sure to advance the command counter after the last script command */
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 0f1a9f0e54..3aee2d82ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15915,7 +15915,6 @@ CloneRowTriggersToPartition(Relation parent, Relation partition)
Datum value;
bool isnull;
List *cols = NIL;
- List *trigargs = NIL;
MemoryContext oldcxt;
/*
@@ -15980,31 +15979,11 @@ CloneRowTriggersToPartition(Relation parent, Relation partition)
}
}
- /* Reconstruct trigger arguments list. */
- if (trigForm->tgnargs > 0)
- {
- char *p;
-
- value = heap_getattr(tuple, Anum_pg_trigger_tgargs,
- RelationGetDescr(pg_trigger), &isnull);
- if (isnull)
- elog(ERROR, "tgargs is null for trigger \"%s\" in partition \"%s\"",
- NameStr(trigForm->tgname), RelationGetRelationName(partition));
-
- p = (char *) VARDATA_ANY(DatumGetByteaPP(value));
-
- for (int i = 0; i < trigForm->tgnargs; i++)
- {
- trigargs = lappend(trigargs, makeString(pstrdup(p)));
- p += strlen(p) + 1;
- }
- }
-
trigStmt = makeNode(CreateTrigStmt);
trigStmt->trigname = NameStr(trigForm->tgname);
trigStmt->relation = NULL;
trigStmt->funcname = NULL; /* passed separately */
- trigStmt->args = trigargs;
+ trigStmt->args = NULL; /* passed separately */
trigStmt->row = true;
trigStmt->timing = trigForm->tgtype & TRIGGER_TYPE_TIMING_MASK;
trigStmt->events = trigForm->tgtype & TRIGGER_TYPE_EVENT_MASK;
diff --git a/src/backend/commands/trigger.c b/src/backend/commands/trigger.c
index ee878d70a9..316692b7c2 100644
--- a/src/backend/commands/trigger.c
+++ b/src/backend/commands/trigger.c
@@ -1172,6 +1172,7 @@ CreateTrigger(CreateTrigStmt *stmt, const char *queryString,
*/
childStmt = (CreateTrigStmt *) copyObject(stmt);
childStmt->funcname = NIL;
+ childStmt->args = NIL;
childStmt->whenClause = NULL;
/* If there is a WHEN clause, create a modified copy of it */
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 29e2681484..27f0345515 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -2793,7 +2793,6 @@ EvalPlanQualStart(EPQState *epqstate, EState *parentestate, Plan *planTree)
estate->es_range_table_array = parentestate->es_range_table_array;
estate->es_range_table_size = parentestate->es_range_table_size;
estate->es_relations = parentestate->es_relations;
- estate->es_queryEnv = parentestate->es_queryEnv;
estate->es_rowmarks = parentestate->es_rowmarks;
estate->es_plannedstmt = parentestate->es_plannedstmt;
estate->es_junkFilter = parentestate->es_junkFilter;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8400dd319e..4529b5c63b 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2278,6 +2278,14 @@ _outRelOptInfo(StringInfo str, const RelOptInfo *node)
WRITE_NODE_FIELD(partitioned_child_rels);
}
+static void
+_outRelInfoList(StringInfo str, const RelInfoList *node)
+{
+ WRITE_NODE_TYPE("RELOPTINFOLIST");
+
+ WRITE_NODE_FIELD(items);
+}
+
static void
_outIndexOptInfo(StringInfo str, const IndexOptInfo *node)
{
@@ -4052,6 +4060,9 @@ outNode(StringInfo str, const void *obj)
case T_RelOptInfo:
_outRelOptInfo(str, obj);
break;
+ case T_RelInfoList:
+ _outRelInfoList(str, obj);
+ break;
case T_IndexOptInfo:
_outIndexOptInfo(str, obj);
break;
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index 6c69c1c147..c69f3469ba 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -92,11 +92,11 @@ geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
*
* join_rel_level[] shouldn't be in use, so just Assert it isn't.
*/
- savelength = list_length(root->join_rel_list);
- savehash = root->join_rel_hash;
+ savelength = list_length(root->join_rel_list->items);
+ savehash = root->join_rel_list->hash;
Assert(root->join_rel_level == NULL);
- root->join_rel_hash = NULL;
+ root->join_rel_list->hash = NULL;
/* construct the best path for the given combination of relations */
joinrel = gimme_tree(root, tour, num_gene);
@@ -121,9 +121,9 @@ geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
* Restore join_rel_list to its former state, and put back original
* hashtable if any.
*/
- root->join_rel_list = list_truncate(root->join_rel_list,
- savelength);
- root->join_rel_hash = savehash;
+ root->join_rel_list->items = list_truncate(root->join_rel_list->items,
+ savelength);
+ root->join_rel_list->hash = savehash;
/* release all the memory acquired within gimme_tree */
MemoryContextSwitchTo(oldcxt);
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 2dbf1db844..0b9999c8a6 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -65,8 +65,7 @@ query_planner(PlannerInfo *root,
* NOTE: append_rel_list was set up by subquery_planner, so do not touch
* here.
*/
- root->join_rel_list = NIL;
- root->join_rel_hash = NULL;
+ root->join_rel_list = makeNode(RelInfoList);
root->join_rel_level = NULL;
root->join_cur_level = 0;
root->canon_pathkeys = NIL;
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 6054bd2b53..c238dd6538 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -31,11 +31,11 @@
#include "utils/hsearch.h"
-typedef struct JoinHashEntry
+typedef struct RelInfoEntry
{
- Relids join_relids; /* hash key --- MUST BE FIRST */
- RelOptInfo *join_rel;
-} JoinHashEntry;
+ Relids relids; /* hash key --- MUST BE FIRST */
+ void *data;
+} RelInfoEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *input_rel);
@@ -375,11 +375,11 @@ find_base_rel(PlannerInfo *root, int relid)
}
/*
- * build_join_rel_hash
- * Construct the auxiliary hash table for join relations.
+ * build_rel_hash
+ * Construct the auxiliary hash table for relation specific data.
*/
static void
-build_join_rel_hash(PlannerInfo *root)
+build_rel_hash(RelInfoList *list)
{
HTAB *hashtab;
HASHCTL hash_ctl;
@@ -388,47 +388,50 @@ build_join_rel_hash(PlannerInfo *root)
/* Create the hash table */
MemSet(&hash_ctl, 0, sizeof(hash_ctl));
hash_ctl.keysize = sizeof(Relids);
- hash_ctl.entrysize = sizeof(JoinHashEntry);
+ hash_ctl.entrysize = sizeof(RelInfoEntry);
hash_ctl.hash = bitmap_hash;
hash_ctl.match = bitmap_match;
hash_ctl.hcxt = CurrentMemoryContext;
- hashtab = hash_create("JoinRelHashTable",
+ hashtab = hash_create("RelHashTable",
256L,
&hash_ctl,
HASH_ELEM | HASH_FUNCTION | HASH_COMPARE | HASH_CONTEXT);
/* Insert all the already-existing joinrels */
- foreach(l, root->join_rel_list)
+ foreach(l, list->items)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
- JoinHashEntry *hentry;
+ void *item = lfirst(l);
+ RelInfoEntry *hentry;
bool found;
+ Relids relids;
- hentry = (JoinHashEntry *) hash_search(hashtab,
- &(rel->relids),
- HASH_ENTER,
- &found);
+ Assert(IsA(item, RelOptInfo));
+ relids = ((RelOptInfo *) item)->relids;
+
+ hentry = (RelInfoEntry *) hash_search(hashtab,
+ &relids,
+ HASH_ENTER,
+ &found);
Assert(!found);
- hentry->join_rel = rel;
+ hentry->data = item;
}
- root->join_rel_hash = hashtab;
+ list->hash = hashtab;
}
/*
- * find_join_rel
- * Returns relation entry corresponding to 'relids' (a set of RT indexes),
- * or NULL if none exists. This is for join relations.
+ * find_rel_info
+ * Find a base or join relation entry.
*/
-RelOptInfo *
-find_join_rel(PlannerInfo *root, Relids relids)
+static void *
+find_rel_info(RelInfoList *list, Relids relids)
{
/*
* Switch to using hash lookup when list grows "too long". The threshold
* is arbitrary and is known only here.
*/
- if (!root->join_rel_hash && list_length(root->join_rel_list) > 32)
- build_join_rel_hash(root);
+ if (!list->hash && list_length(list->items) > 32)
+ build_rel_hash(list);
/*
* Use either hashtable lookup or linear search, as appropriate.
@@ -438,34 +441,90 @@ find_join_rel(PlannerInfo *root, Relids relids)
* so would force relids out of a register and thus probably slow down the
* list-search case.
*/
- if (root->join_rel_hash)
+ if (list->hash)
{
Relids hashkey = relids;
- JoinHashEntry *hentry;
+ RelInfoEntry *hentry;
- hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
- &hashkey,
- HASH_FIND,
- NULL);
+ hentry = (RelInfoEntry *) hash_search(list->hash,
+ &hashkey,
+ HASH_FIND,
+ NULL);
if (hentry)
- return hentry->join_rel;
+ return hentry->data;
}
else
{
ListCell *l;
- foreach(l, root->join_rel_list)
+ foreach(l, list->items)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
+ void *item = lfirst(l);
+ Relids item_relids;
- if (bms_equal(rel->relids, relids))
- return rel;
+ Assert(IsA(item, RelOptInfo));
+ item_relids = ((RelOptInfo *) item)->relids;
+
+ if (bms_equal(item_relids, relids))
+ return item;
}
}
return NULL;
}
+/*
+ * find_join_rel
+ * Returns relation entry corresponding to 'relids' (a set of RT indexes),
+ * or NULL if none exists. This is for join relations.
+ */
+RelOptInfo *
+find_join_rel(PlannerInfo *root, Relids relids)
+{
+ return (RelOptInfo *) find_rel_info(root->join_rel_list, relids);
+}
+
+/*
+ * add_rel_info
+ * Add relation specific info to a list, and also add it to the auxiliary
+ * hashtable if there is one.
+ */
+static void
+add_rel_info(RelInfoList *list, void *data)
+{
+ Assert(IsA(data, RelOptInfo));
+
+ /* GEQO requires us to append the new joinrel to the end of the list! */
+ list->items = lappend(list->items, data);
+
+ /* store it into the auxiliary hashtable if there is one. */
+ if (list->hash)
+ {
+ Relids relids;
+ RelInfoEntry *hentry;
+ bool found;
+
+ relids = ((RelOptInfo *) data)->relids;
+ hentry = (RelInfoEntry *) hash_search(list->hash,
+ &relids,
+ HASH_ENTER,
+ &found);
+ Assert(!found);
+ hentry->data = data;
+ }
+}
+
+/*
+ * add_join_rel
+ * Add given join relation to the list of join relations in the given
+ * PlannerInfo.
+ */
+static void
+add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
+{
+ add_rel_info(root->join_rel_list, joinrel);
+}
+
/*
* set_foreign_rel_properties
* Set up foreign-join fields if outer and inner relation are foreign
@@ -516,32 +575,6 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
}
}
-/*
- * add_join_rel
- * Add given join relation to the list of join relations in the given
- * PlannerInfo. Also add it to the auxiliary hashtable if there is one.
- */
-static void
-add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
-{
- /* GEQO requires us to append the new joinrel to the end of the list! */
- root->join_rel_list = lappend(root->join_rel_list, joinrel);
-
- /* store it into the auxiliary hashtable if there is one. */
- if (root->join_rel_hash)
- {
- JoinHashEntry *hentry;
- bool found;
-
- hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
- &(joinrel->relids),
- HASH_ENTER,
- &found);
- Assert(!found);
- hentry->join_rel = joinrel;
- }
-}
-
/*
* build_join_rel
* Returns relation entry corresponding to the union of two given rels,
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index e71a21c0a7..5982af4de1 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -194,10 +194,8 @@ static PruneStepResult *perform_pruning_base_step(PartitionPruneContext *context
static PruneStepResult *perform_pruning_combine_step(PartitionPruneContext *context,
PartitionPruneStepCombine *cstep,
PruneStepResult **step_results);
-static PartClauseMatchStatus match_boolean_partition_clause(Oid partopfamily,
- Expr *clause,
- Expr *partkey,
- Expr **outconst);
+static bool match_boolean_partition_clause(Oid partopfamily, Expr *clause,
+ Expr *partkey, Expr **outconst);
static void partkey_datum_from_expr(PartitionPruneContext *context,
Expr *expr, int stateidx,
Datum *value, bool *isnull);
@@ -1625,7 +1623,6 @@ match_clause_to_partition_key(GeneratePruningStepsContext *context,
bool *clause_is_not_null, PartClauseInfo **pc,
List **clause_steps)
{
- PartClauseMatchStatus boolmatchstatus;
PartitionScheme part_scheme = context->rel->part_scheme;
Oid partopfamily = part_scheme->partopfamily[partkeyidx],
partcoll = part_scheme->partcollation[partkeyidx];
@@ -1634,10 +1631,7 @@ match_clause_to_partition_key(GeneratePruningStepsContext *context,
/*
* Recognize specially shaped clauses that match a Boolean partition key.
*/
- boolmatchstatus = match_boolean_partition_clause(partopfamily, clause,
- partkey, &expr);
-
- if (boolmatchstatus == PARTCLAUSE_MATCH_CLAUSE)
+ if (match_boolean_partition_clause(partopfamily, clause, partkey, &expr))
{
PartClauseInfo *partclause;
@@ -2153,21 +2147,7 @@ match_clause_to_partition_key(GeneratePruningStepsContext *context,
return PARTCLAUSE_MATCH_NULLNESS;
}
- /*
- * If we get here then the return value depends on the result of the
- * match_boolean_partition_clause call above. If the call returned
- * PARTCLAUSE_UNSUPPORTED then we're either not dealing with a bool qual
- * or the bool qual is not suitable for pruning. Since the qual didn't
- * match up to any of the other qual types supported here, then trying to
- * match it against any other partition key is a waste of time, so just
- * return PARTCLAUSE_UNSUPPORTED. If the qual just couldn't be matched to
- * this partition key, then it may match another, so return
- * PARTCLAUSE_NOMATCH. The only other value that
- * match_boolean_partition_clause can return is PARTCLAUSE_MATCH_CLAUSE,
- * and since that value was already dealt with above, then we can just
- * return boolmatchstatus.
- */
- return boolmatchstatus;
+ return PARTCLAUSE_UNSUPPORTED;
}
/*
@@ -3415,15 +3395,11 @@ perform_pruning_combine_step(PartitionPruneContext *context,
/*
* match_boolean_partition_clause
*
- * If we're able to match the clause to the partition key as specially-shaped
- * boolean clause, set *outconst to a Const containing a true or false value
- * and return PARTCLAUSE_MATCH_CLAUSE. Returns PARTCLAUSE_UNSUPPORTED if the
- * clause is not a boolean clause or if the boolean clause is unsuitable for
- * partition pruning. Returns PARTCLAUSE_NOMATCH if it's a bool quals but
- * just does not match this partition key. *outconst is set to NULL in the
- * latter two cases.
+ * Sets *outconst to a Const containing true or false value and returns true if
+ * we're able to match the clause to the partition key as specially-shaped
+ * Boolean clause. Returns false otherwise with *outconst set to NULL.
*/
-static PartClauseMatchStatus
+static bool
match_boolean_partition_clause(Oid partopfamily, Expr *clause, Expr *partkey,
Expr **outconst)
{
@@ -3432,7 +3408,7 @@ match_boolean_partition_clause(Oid partopfamily, Expr *clause, Expr *partkey,
*outconst = NULL;
if (!IsBooleanOpfamily(partopfamily))
- return PARTCLAUSE_UNSUPPORTED;
+ return false;
if (IsA(clause, BooleanTest))
{
@@ -3441,7 +3417,7 @@ match_boolean_partition_clause(Oid partopfamily, Expr *clause, Expr *partkey,
/* Only IS [NOT] TRUE/FALSE are any good to us */
if (btest->booltesttype == IS_UNKNOWN ||
btest->booltesttype == IS_NOT_UNKNOWN)
- return PARTCLAUSE_UNSUPPORTED;
+ return false;
leftop = btest->arg;
if (IsA(leftop, RelabelType))
@@ -3454,7 +3430,7 @@ match_boolean_partition_clause(Oid partopfamily, Expr *clause, Expr *partkey,
: (Expr *) makeBoolConst(false, false);
if (*outconst)
- return PARTCLAUSE_MATCH_CLAUSE;
+ return true;
}
else
{
@@ -3474,10 +3450,10 @@ match_boolean_partition_clause(Oid partopfamily, Expr *clause, Expr *partkey,
*outconst = (Expr *) makeBoolConst(false, false);
if (*outconst)
- return PARTCLAUSE_MATCH_CLAUSE;
+ return true;
}
- return PARTCLAUSE_NOMATCH;
+ return false;
}
/*
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index ffd84d877c..44a59e1d4f 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -1070,7 +1070,6 @@ exec_simple_query(const char *query_string)
bool snapshot_set = false;
const char *commandTag;
char completionTag[COMPLETION_TAG_BUFSIZE];
- MemoryContext per_parsetree_context = NULL;
List *querytree_list,
*plantree_list;
Portal portal;
@@ -1133,25 +1132,10 @@ exec_simple_query(const char *query_string)
/*
* OK to analyze, rewrite, and plan this query.
*
- * Switch to appropriate context for constructing query and plan trees
- * (these can't be in the transaction context, as that will get reset
- * when the command is COMMIT/ROLLBACK). If we have multiple
- * parsetrees, we use a separate context for each one, so that we can
- * free that memory before moving on to the next one. But for the
- * last (or only) parsetree, just use MessageContext, which will be
- * reset shortly after completion anyway. In event of an error, the
- * per_parsetree_context will be deleted when MessageContext is reset.
+ * Switch to appropriate context for constructing querytrees (again,
+ * these must outlive the execution context).
*/
- if (lnext(parsetree_item) != NULL)
- {
- per_parsetree_context =
- AllocSetContextCreate(MessageContext,
- "per-parsetree message context",
- ALLOCSET_DEFAULT_SIZES);
- oldcontext = MemoryContextSwitchTo(per_parsetree_context);
- }
- else
- oldcontext = MemoryContextSwitchTo(MessageContext);
+ oldcontext = MemoryContextSwitchTo(MessageContext);
querytree_list = pg_analyze_and_rewrite(parsetree, query_string,
NULL, 0, NULL);
@@ -1176,8 +1160,8 @@ exec_simple_query(const char *query_string)
/*
* We don't have to copy anything into the portal, because everything
- * we are passing here is in MessageContext or the
- * per_parsetree_context, and so will outlive the portal anyway.
+ * we are passing here is in MessageContext, which will outlive the
+ * portal anyway.
*/
PortalDefineQuery(portal,
NULL,
@@ -1279,10 +1263,6 @@ exec_simple_query(const char *query_string)
* aborted by error will not send an EndCommand report at all.)
*/
EndCommand(completionTag, dest);
-
- /* Now we may drop the per-parsetree context, if one was created. */
- if (per_parsetree_context)
- MemoryContextDelete(per_parsetree_context);
} /* end loop over parsetrees */
/*
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 04d77ad700..f1acbdfcf2 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -1051,7 +1051,7 @@ test_config_settings(void)
else
printf("%dkB\n", n_buffers * (BLCKSZ / 1024));
- printf(_("selecting default time zone ... "));
+ printf(_("selecting default timezone ... "));
fflush(stdout);
default_timezone = select_default_timezone(share_path);
printf("%s\n", default_timezone ? default_timezone : "GMT");
diff --git a/src/bin/pg_basebackup/pg_recvlogical.c b/src/bin/pg_basebackup/pg_recvlogical.c
index 90a3f41bbb..b029118bf6 100644
--- a/src/bin/pg_basebackup/pg_recvlogical.c
+++ b/src/bin/pg_basebackup/pg_recvlogical.c
@@ -1020,11 +1020,12 @@ prepareToTerminate(PGconn *conn, XLogRecPtr endpos, bool keepalive, XLogRecPtr l
if (verbose)
{
if (keepalive)
- pg_log_info("end position %X/%X reached by keepalive",
+ pg_log_info("endpos %X/%X reached by keepalive",
(uint32) (endpos >> 32), (uint32) endpos);
else
- pg_log_info("end position %X/%X reached by WAL record at %X/%X",
+ pg_log_info("endpos %X/%X reached by record at %X/%X",
(uint32) (endpos >> 32), (uint32) (endpos),
(uint32) (lsn >> 32), (uint32) lsn);
+
}
}
diff --git a/src/bin/pg_checksums/pg_checksums.c b/src/bin/pg_checksums/pg_checksums.c
index 8c00ec9a3b..b591fcc864 100644
--- a/src/bin/pg_checksums/pg_checksums.c
+++ b/src/bin/pg_checksums/pg_checksums.c
@@ -235,7 +235,7 @@ scan_file(const char *fn, BlockNumber segmentno)
/* Write block with checksum */
if (write(f, buf.data, BLCKSZ) != BLCKSZ)
{
- pg_log_error("could not write block %u in file \"%s\": %m",
+ pg_log_error("could not update checksum of block %u in file \"%s\": %m",
blockno, fn);
exit(1);
}
@@ -469,7 +469,7 @@ main(int argc, char *argv[])
/* filenode checking only works in --check mode */
if (mode != PG_MODE_CHECK && only_filenode)
{
- pg_log_error("option -f/--filenode can only be used with --check");
+ pg_log_error("--filenode option only possible with --check");
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
progname);
exit(1);
diff --git a/src/bin/pg_dump/pg_backup_db.c b/src/bin/pg_dump/pg_backup_db.c
index ee822c5249..401e0c8883 100644
--- a/src/bin/pg_dump/pg_backup_db.c
+++ b/src/bin/pg_dump/pg_backup_db.c
@@ -176,7 +176,7 @@ _connectDB(ArchiveHandle *AH, const char *reqdb, const char *requser)
newConn = PQconnectdbParams(keywords, values, true);
if (!newConn)
- fatal("could not reconnect to database");
+ fatal("failed to reconnect to database");
if (PQstatus(newConn) == CONNECTION_BAD)
{
@@ -287,7 +287,7 @@ ConnectDatabase(Archive *AHX,
AH->connection = PQconnectdbParams(keywords, values, true);
if (!AH->connection)
- fatal("could not connect to database");
+ fatal("failed to connect to database");
if (PQstatus(AH->connection) == CONNECTION_BAD &&
PQconnectionNeedsPassword(AH->connection) &&
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 0981efcf5d..158c0c74b2 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -482,7 +482,7 @@ main(int argc, char *argv[])
OPF = fopen(filename, PG_BINARY_W);
if (!OPF)
{
- pg_log_error("could not open output file \"%s\": %m",
+ pg_log_error("could not open the output file \"%s\": %m",
filename);
exit_nicely(1);
}
@@ -1492,11 +1492,11 @@ dumpDatabases(PGconn *conn)
/* Skip any explicitly excluded database */
if (simple_string_list_member(&database_exclude_names, dbname))
{
- pg_log_info("excluding database \"%s\"", dbname);
+ pg_log_info("excluding database \"%s\"...", dbname);
continue;
}
- pg_log_info("dumping database \"%s\"", dbname);
+ pg_log_info("dumping database \"%s\"...", dbname);
fprintf(OPF, "--\n-- Database \"%s\" dump\n--\n\n", dbname);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index d76f27c9e8..73f395f2a3 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -304,7 +304,7 @@ usage(void)
printf(_(" -p, --old-port=PORT old cluster port number (default %d)\n"), old_cluster.port);
printf(_(" -P, --new-port=PORT new cluster port number (default %d)\n"), new_cluster.port);
printf(_(" -r, --retain retain SQL and log files after success\n"));
- printf(_(" -s, --socketdir=DIR socket directory to use (default current dir.)\n"));
+ printf(_(" -s, --socketdir=DIR socket directory to use (default CWD)\n"));
printf(_(" -U, --username=NAME cluster superuser (default \"%s\")\n"), os_info.user);
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h
index c2b0481e7e..7edfcf3ef9 100644
--- a/src/include/access/tableam.h
+++ b/src/include/access/tableam.h
@@ -434,8 +434,8 @@ typedef struct TableAmRoutine
*
* Note that only the subset of the relcache filled by
* RelationBuildLocalRelation() can be relied upon and that the relation's
- * catalog entries will either not yet exist (new relation), or will still
- * reference the old relfilenode.
+ * catalog entries either will either not yet exist (new relation), or
+ * will still reference the old relfilenode.
*
* As output *freezeXid, *minmulti must be set to the values appropriate
* for pg_class.{relfrozenxid, relminmxid}. For AMs that don't need those
@@ -591,7 +591,7 @@ typedef struct TableAmRoutine
* See table_relation_estimate_size().
*
* While block oriented, it shouldn't be too hard for an AM that doesn't
- * internally use blocks to convert into a usable representation.
+ * doesn't internally use blocks to convert into a usable representation.
*
* This differs from the relation_size callback by returning size
* estimates (both relation size and tuple count) for planning purposes,
@@ -967,7 +967,7 @@ table_index_fetch_end(struct IndexFetchTableData *scan)
*
* *all_dead, if all_dead is not NULL, will be set to true by
* table_index_fetch_tuple() iff it is guaranteed that no backend needs to see
- * that tuple. Index AMs can use that to avoid returning that tid in future
+ * that tuple. Index AMs can use that do avoid returning that tid in future
* searches.
*
* The difference between this function and table_fetch_row_version is that
@@ -1014,8 +1014,8 @@ extern bool table_index_fetch_tuple_check(Relation rel,
* true, false otherwise.
*
* See table_index_fetch_tuple's comment about what the difference between
- * these functions is. It is correct to use this function outside of index
- * entry->table tuple lookups.
+ * these functions is. This function is the correct to use outside of
+ * index entry->table tuple lookups.
*/
static inline bool
table_tuple_fetch_row_version(Relation rel,
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 4e2fb39105..11027cdb10 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -220,6 +220,7 @@ typedef enum NodeTag
T_PlannerInfo,
T_PlannerGlobal,
T_RelOptInfo,
+ T_RelInfoList,
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 441e64eca9..38dc186623 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -236,15 +236,9 @@ struct PlannerInfo
/*
* join_rel_list is a list of all join-relation RelOptInfos we have
- * considered in this planning run. For small problems we just scan the
- * list to do lookups, but when there are many join relations we build a
- * hash table for faster lookups. The hash table is present and valid
- * when join_rel_hash is not NULL. Note that we still maintain the list
- * even when using the hash table for lookups; this simplifies life for
- * GEQO.
+ * considered in this planning run.
*/
- List *join_rel_list; /* list of join-relation RelOptInfos */
- struct HTAB *join_rel_hash; /* optional hashtable for join relations */
+ struct RelInfoList *join_rel_list; /* list of join-relation RelOptInfos */
/*
* When doing a dynamic-programming-style join search, join_rel_level[k]
@@ -742,6 +736,24 @@ typedef struct RelOptInfo
((rel)->part_scheme && (rel)->boundinfo && (rel)->nparts > 0 && \
(rel)->part_rels && (rel)->partexprs && (rel)->nullable_partexprs)
+/*
+ * RelInfoList
+ * A list to store relation specific info and to retrieve it by relids.
+ *
+ * For small problems we just scan the list to do lookups, but when there are
+ * many relations we build a hash table for faster lookups. The hash table is
+ * present and valid when rel_hash is not NULL. Note that we still maintain
+ * the list even when using the hash table for lookups; this simplifies life
+ * for GEQO.
+ */
+typedef struct RelInfoList
+{
+ NodeTag type;
+
+ List *items;
+ struct HTAB *hash;
+} RelInfoList;
+
/*
* IndexOptInfo
* Per-index information for planning/optimization
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 2eecb1744b..841bd8bc67 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -1086,19 +1086,6 @@ explain (costs off) select * from boolpart where a is not unknown;
Filter: (a IS NOT UNKNOWN)
(7 rows)
-create table boolrangep (a bool, b bool, c int) partition by range (a,b,c);
-create table boolrangep_tf partition of boolrangep for values from ('true', 'false', 0) to ('true', 'false', 100);
-create table boolrangep_ft partition of boolrangep for values from ('false', 'true', 0) to ('false', 'true', 100);
-create table boolrangep_ff1 partition of boolrangep for values from ('false', 'false', 0) to ('false', 'false', 50);
-create table boolrangep_ff2 partition of boolrangep for values from ('false', 'false', 50) to ('false', 'false', 100);
--- try a more complex case that's been known to trip up pruning in the past
-explain (costs off) select * from boolrangep where not a and not b and c = 25;
- QUERY PLAN
-----------------------------------------------
- Seq Scan on boolrangep_ff1
- Filter: ((NOT a) AND (NOT b) AND (c = 25))
-(2 rows)
-
-- test scalar-to-array operators
create table coercepart (a varchar) partition by list (a);
create table coercepart_ab partition of coercepart for values in ('ab');
@@ -1433,7 +1420,7 @@ explain (costs off) select * from rparted_by_int2 where a > 100000000000000;
Filter: (a > '100000000000000'::bigint)
(2 rows)
-drop table lp, coll_pruning, rlp, mc3p, mc2p, boolpart, boolrangep, rp, coll_pruning_multi, like_op_noprune, lparted_by_int2, rparted_by_int2;
+drop table lp, coll_pruning, rlp, mc3p, mc2p, boolpart, rp, coll_pruning_multi, like_op_noprune, lparted_by_int2, rparted_by_int2;
--
-- Test Partition pruning for HASH partitioning
--
diff --git a/src/test/regress/expected/triggers.out b/src/test/regress/expected/triggers.out
index c64151ba09..cd2b550c14 100644
--- a/src/test/regress/expected/triggers.out
+++ b/src/test/regress/expected/triggers.out
@@ -2094,30 +2094,6 @@ NOTICE: trigger zzz on parted_trig_1_1 AFTER INSERT for ROW
NOTICE: trigger bbb on parted_trig_2 AFTER INSERT for ROW
NOTICE: trigger zzz on parted_trig_2 AFTER INSERT for ROW
drop table parted_trig;
--- Verify propagation of trigger arguments to partitions
-create table parted_trig (a int) partition by list (a);
-create table parted_trig1 partition of parted_trig for values in (1);
-create or replace function trigger_notice() returns trigger as $$
- declare
- arg1 text = TG_ARGV[0];
- arg2 integer = TG_ARGV[1];
- begin
- raise notice 'trigger % on % % % for % args % %',
- TG_NAME, TG_TABLE_NAME, TG_WHEN, TG_OP, TG_LEVEL, arg1, arg2;
- return null;
- end;
- $$ language plpgsql;
-create trigger aaa after insert on parted_trig
- for each row execute procedure trigger_notice('quirky', 1);
--- Verify propagation of trigger arguments to partitions attached after creating trigger
-create table parted_trig2 partition of parted_trig for values in (2);
-create table parted_trig3 (like parted_trig);
-alter table parted_trig attach partition parted_trig3 for values in (3);
-insert into parted_trig values (1), (2), (3);
-NOTICE: trigger aaa on parted_trig1 AFTER INSERT for ROW args quirky 1
-NOTICE: trigger aaa on parted_trig2 AFTER INSERT for ROW args quirky 1
-NOTICE: trigger aaa on parted_trig3 AFTER INSERT for ROW args quirky 1
-drop table parted_trig;
-- test irregular partitions (i.e., different column definitions),
-- including that the WHEN clause works
create function bark(text) returns bool language plpgsql immutable
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index 7bb4e2fffc..071e28dce8 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -159,15 +159,6 @@ explain (costs off) select * from boolpart where a is not true and a is not fals
explain (costs off) select * from boolpart where a is unknown;
explain (costs off) select * from boolpart where a is not unknown;
-create table boolrangep (a bool, b bool, c int) partition by range (a,b,c);
-create table boolrangep_tf partition of boolrangep for values from ('true', 'false', 0) to ('true', 'false', 100);
-create table boolrangep_ft partition of boolrangep for values from ('false', 'true', 0) to ('false', 'true', 100);
-create table boolrangep_ff1 partition of boolrangep for values from ('false', 'false', 0) to ('false', 'false', 50);
-create table boolrangep_ff2 partition of boolrangep for values from ('false', 'false', 50) to ('false', 'false', 100);
-
--- try a more complex case that's been known to trip up pruning in the past
-explain (costs off) select * from boolrangep where not a and not b and c = 25;
-
-- test scalar-to-array operators
create table coercepart (a varchar) partition by list (a);
create table coercepart_ab partition of coercepart for values in ('ab');
@@ -273,7 +264,7 @@ create table rparted_by_int2_maxvalue partition of rparted_by_int2 for values fr
-- all partitions but rparted_by_int2_maxvalue pruned
explain (costs off) select * from rparted_by_int2 where a > 100000000000000;
-drop table lp, coll_pruning, rlp, mc3p, mc2p, boolpart, boolrangep, rp, coll_pruning_multi, like_op_noprune, lparted_by_int2, rparted_by_int2;
+drop table lp, coll_pruning, rlp, mc3p, mc2p, boolpart, rp, coll_pruning_multi, like_op_noprune, lparted_by_int2, rparted_by_int2;
--
-- Test Partition pruning for HASH partitioning
diff --git a/src/test/regress/sql/triggers.sql b/src/test/regress/sql/triggers.sql
index 4534dc9ebe..8f833b7d10 100644
--- a/src/test/regress/sql/triggers.sql
+++ b/src/test/regress/sql/triggers.sql
@@ -1460,29 +1460,6 @@ create trigger qqq after insert on parted_trig_1_1 for each row execute procedur
insert into parted_trig values (50), (1500);
drop table parted_trig;
--- Verify propagation of trigger arguments to partitions
-create table parted_trig (a int) partition by list (a);
-create table parted_trig1 partition of parted_trig for values in (1);
-create or replace function trigger_notice() returns trigger as $$
- declare
- arg1 text = TG_ARGV[0];
- arg2 integer = TG_ARGV[1];
- begin
- raise notice 'trigger % on % % % for % args % %',
- TG_NAME, TG_TABLE_NAME, TG_WHEN, TG_OP, TG_LEVEL, arg1, arg2;
- return null;
- end;
- $$ language plpgsql;
-create trigger aaa after insert on parted_trig
- for each row execute procedure trigger_notice('quirky', 1);
-
--- Verify propagation of trigger arguments to partitions attached after creating trigger
-create table parted_trig2 partition of parted_trig for values in (2);
-create table parted_trig3 (like parted_trig);
-alter table parted_trig attach partition parted_trig3 for values in (3);
-insert into parted_trig values (1), (2), (3);
-drop table parted_trig;
-
-- test irregular partitions (i.e., different column definitions),
-- including that the WHEN clause works
create function bark(text) returns bool language plpgsql immutable
--
2.16.4
v13-0002-Introduce-make_join_rel_common-function.patchtext/x-diffDownload
From 1a42a4e80702fc2741bc603ab5a08835a06bc51e Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Fri, 12 Jul 2019 10:04:32 +0200
Subject: [PATCH 2/3] Introduce make_join_rel_common function.
This diff tries to make the following patch a bit easier to read.
---
src/backend/optimizer/path/joinrels.c | 39 ++++++++++++++++++++++-------------
1 file changed, 25 insertions(+), 14 deletions(-)
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 43c3b7ea48..a9fc52c61f 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -35,6 +35,8 @@ static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
+static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1,
+ RelOptInfo *rel2);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist);
@@ -647,21 +649,12 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
return true;
}
-
/*
- * make_join_rel
- * Find or create a join RelOptInfo that represents the join of
- * the two given rels, and add to it path information for paths
- * created with the two rels as outer and inner rel.
- * (The join rel may already contain paths generated from other
- * pairs of rels that add up to the same set of base rels.)
- *
- * NB: will return NULL if attempted join is not valid. This can happen
- * when working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
- */
-RelOptInfo *
-make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+ * make_join_rel_common
+ * The workhorse of make_join_rel().
+ */
+static RelOptInfo *
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -743,6 +736,24 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
return joinrel;
}
+/*
+ * make_join_rel
+ * Find or create a join RelOptInfo that represents the join of
+ * the two given rels, and add to it path information for paths
+ * created with the two rels as outer and inner rel.
+ * (The join rel may already contain paths generated from other
+ * pairs of rels that add up to the same set of base rels.)
+ *
+ * NB: will return NULL if attempted join is not valid. This can happen when
+ * working with outer joins, or with IN or EXISTS clauses that have been
+ * turned into joins.
+ */
+RelOptInfo *
+make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+{
+ return make_join_rel_common(root, rel1, rel2);
+}
+
/*
* populate_joinrel_with_paths
* Add paths to the given joinrel for given pair of joining relations. The
--
2.16.4
v13-0003-Aggregate-push-down-basic-functionality.patchtext/x-diffDownload
From fbb8271ee1c2608552aee905f0a798f3b19f1a17 Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Fri, 12 Jul 2019 10:04:32 +0200
Subject: [PATCH 3/3] Aggregate push-down - basic functionality.
---
src/backend/nodes/copyfuncs.c | 21 +-
src/backend/nodes/outfuncs.c | 36 ++
src/backend/optimizer/README | 87 +++
src/backend/optimizer/path/allpaths.c | 112 ++++
src/backend/optimizer/path/costsize.c | 17 +-
src/backend/optimizer/path/equivclass.c | 134 +++++
src/backend/optimizer/path/joinrels.c | 170 +++++-
src/backend/optimizer/plan/initsplan.c | 257 ++++++++
src/backend/optimizer/plan/planagg.c | 2 +-
src/backend/optimizer/plan/planmain.c | 23 +-
src/backend/optimizer/plan/planner.c | 58 +-
src/backend/optimizer/plan/setrefs.c | 33 ++
src/backend/optimizer/prep/prepjointree.c | 1 +
src/backend/optimizer/util/pathnode.c | 143 ++++-
src/backend/optimizer/util/relnode.c | 914 +++++++++++++++++++++++++++--
src/backend/optimizer/util/tlist.c | 56 ++
src/backend/utils/misc/guc.c | 9 +
src/include/nodes/nodes.h | 2 +
src/include/nodes/pathnodes.h | 99 ++++
src/include/optimizer/clauses.h | 2 +
src/include/optimizer/pathnode.h | 19 +-
src/include/optimizer/paths.h | 7 +
src/include/optimizer/planmain.h | 4 +-
src/include/optimizer/tlist.h | 8 +-
src/test/regress/expected/agg_pushdown.out | 217 +++++++
src/test/regress/expected/sysviews.out | 3 +-
src/test/regress/parallel_schedule | 3 +
src/test/regress/serial_schedule | 1 +
28 files changed, 2365 insertions(+), 73 deletions(-)
create mode 100644 src/test/regress/expected/agg_pushdown.out
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 78deade89b..56f99e2a59 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2204,8 +2204,8 @@ _copyOnConflictExpr(const OnConflictExpr *from)
/* ****************************************************************
* pathnodes.h copy functions
*
- * We don't support copying RelOptInfo, IndexOptInfo, or Path nodes.
- * There are some subsidiary structs that are useful to copy, though.
+ * We don't support copying RelOptInfo, IndexOptInfo, RelAggInfo or Path
+ * nodes. There are some subsidiary structs that are useful to copy, though.
* ****************************************************************
*/
@@ -2346,6 +2346,20 @@ _copyPlaceHolderInfo(const PlaceHolderInfo *from)
return newnode;
}
+static GroupedVarInfo *
+_copyGroupedVarInfo(const GroupedVarInfo *from)
+{
+ GroupedVarInfo *newnode = makeNode(GroupedVarInfo);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gv_eval_at);
+ COPY_SCALAR_FIELD(derived);
+
+ return newnode;
+}
+
/* ****************************************************************
* parsenodes.h copy functions
* ****************************************************************
@@ -5124,6 +5138,9 @@ copyObjectImpl(const void *from)
case T_PlaceHolderInfo:
retval = _copyPlaceHolderInfo(from);
break;
+ case T_GroupedVarInfo:
+ retval = _copyGroupedVarInfo(from);
+ break;
/*
* VALUE NODES
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 4529b5c63b..518aeeefe0 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2190,6 +2190,8 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_BITMAPSET_FIELD(all_baserels);
WRITE_BITMAPSET_FIELD(nullable_baserels);
WRITE_NODE_FIELD(join_rel_list);
+ WRITE_NODE_FIELD(grouped_rel_list);
+ WRITE_NODE_FIELD(agg_info_list);
WRITE_INT_FIELD(join_cur_level);
WRITE_NODE_FIELD(init_plans);
WRITE_NODE_FIELD(cte_plan_ids);
@@ -2203,6 +2205,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(append_rel_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
@@ -2210,6 +2213,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(distinct_pathkeys);
WRITE_NODE_FIELD(sort_pathkeys);
WRITE_NODE_FIELD(processed_tlist);
+ WRITE_INT_FIELD(max_sortgroupref);
WRITE_NODE_FIELD(minmax_aggs);
WRITE_FLOAT_FIELD(total_table_pages, "%.0f");
WRITE_FLOAT_FIELD(tuple_fraction, "%.4f");
@@ -2428,6 +2432,20 @@ _outParamPathInfo(StringInfo str, const ParamPathInfo *node)
WRITE_NODE_FIELD(ppi_clauses);
}
+static void
+_outRelAggInfo(StringInfo str, const RelAggInfo *node)
+{
+ WRITE_NODE_TYPE("RELAGGINFO");
+
+ WRITE_BITMAPSET_FIELD(relids);
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(agg_input);
+ WRITE_FLOAT_FIELD(input_rows, "%.0f");
+ WRITE_NODE_FIELD(group_clauses);
+ WRITE_NODE_FIELD(group_exprs);
+ WRITE_NODE_FIELD(agg_exprs);
+}
+
static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
@@ -2527,6 +2545,18 @@ _outPlaceHolderInfo(StringInfo str, const PlaceHolderInfo *node)
WRITE_INT_FIELD(ph_width);
}
+static void
+_outGroupedVarInfo(StringInfo str, const GroupedVarInfo *node)
+{
+ WRITE_NODE_TYPE("GROUPEDVARINFO");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_BITMAPSET_FIELD(gv_eval_at);
+ WRITE_BOOL_FIELD(derived);
+}
+
static void
_outMinMaxAggInfo(StringInfo str, const MinMaxAggInfo *node)
{
@@ -4084,6 +4114,9 @@ outNode(StringInfo str, const void *obj)
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_RelAggInfo:
+ _outRelAggInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
@@ -4102,6 +4135,9 @@ outNode(StringInfo str, const void *obj)
case T_PlaceHolderInfo:
_outPlaceHolderInfo(str, obj);
break;
+ case T_GroupedVarInfo:
+ _outGroupedVarInfo(str, obj);
+ break;
case T_MinMaxAggInfo:
_outMinMaxAggInfo(str, obj);
break;
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
index 89ce373d5e..4633fb8768 100644
--- a/src/backend/optimizer/README
+++ b/src/backend/optimizer/README
@@ -1127,3 +1127,90 @@ breaking down aggregation or grouping over a partitioned relation into
aggregation or grouping over its partitions is called partitionwise
aggregation. Especially when the partition keys match the GROUP BY clause,
this can be significantly faster than the regular method.
+
+Aggregate push-down
+-------------------
+
+The obvious way to evaluate aggregates is to evaluate the FROM clause of the
+SQL query (this is what query_planner does) and use the resuing paths as the
+input of Agg node. However, if the groups are large enough, it may be more
+efficient to apply the partial aggregation to the output of base relation
+scan, and finalize it when we have all relations of the query joined:
+
+ EXPLAIN
+ SELECT a.i, avg(b.u)
+ FROM a JOIN b ON b.j = a.i
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+ -> Index Only Scan using a_pkey on a
+ Index Cond: (i = b.j)
+
+Thus the join above the partial aggregate node receives fewer input rows, and
+so the number of outer-to-inner pairs of tuples to be checked can be
+significantly lower, which can in turn lead to considerably lower join cost.
+
+Note that there's often no GROUP BY expression to be used for the partial
+aggregation, so we use equivalence classes to derive grouping expression: in
+the example above, the grouping key "b.j" was derived from "a.i".
+
+Furthermore, extra grouping columns can be added to the partial Agg node if a
+join clause above that node references a column which is not in the query
+GROUP BY clause and which could not be derived using equivalence class.
+
+ EXPLAIN
+ SELECT a.i, avg(b.u)
+ FROM a JOIN b ON b.j = a.x
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (a.x = b.j)
+ -> Seq Scan on a
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+
+Here the partial aggregate uses "b.j" as grouping column although it's not in
+the same equivalence class as "a.i". Note that no column of the {a.x, b.j}
+equivalence class is used as a key for the final aggregation.
+
+Besides base relation, the aggregation can also be pushed down to join:
+
+ EXPLAIN
+ SELECT a.i, avg(b.u + c.v)
+ FROM a JOIN b ON b.j = a.i
+ JOIN c ON c.k = a.i
+ WHERE b.j = c.k GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (b.j = a.i)
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Hash Join
+ Hash Cond: (b.j = c.k)
+ -> Seq Scan on b
+ -> Hash
+ -> Seq Scan on c
+ -> Hash
+ -> Seq Scan on a
+
+Whether the Agg node is created out of base relation or out of join, it's
+added to a separate RelOptInfo that we call "grouped relation". Grouped
+relation can be joined to a non-grouped relation, which results in a grouped
+relation too. Join of two grouped relations does not seem to be very useful
+and is currently not supported.
+
+If query_planner produces a grouped relation that contains valid paths, these
+are simply added to the UPPERREL_PARTIAL_GROUP_AGG relation. Further
+processing of these paths then does not differ from processing of other
+partially grouped paths.
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index b7723481b0..f14eb357dc 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -60,6 +60,7 @@ typedef struct pushdown_safety_info
/* These parameters are set by GUC */
bool enable_geqo = false; /* just in case GUC doesn't set it */
+bool enable_agg_pushdown;
int geqo_threshold;
int min_parallel_table_scan_size;
int min_parallel_index_scan_size;
@@ -124,6 +125,9 @@ static void set_result_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
+static void add_grouped_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, AggStrategy aggstrategy,
+ RelAggInfo *agg_info);
static RelOptInfo *make_rel_from_joinlist(PlannerInfo *root, List *joinlist);
static bool subquery_is_pushdown_safe(Query *subquery, Query *topquery,
pushdown_safety_info *safetyInfo);
@@ -2730,6 +2734,80 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
}
}
+/*
+ * generate_grouping_paths
+ * Create partially aggregated paths and add them to grouped relation.
+ *
+ * "rel_plain" is base or join relation whose paths are not grouped.
+ */
+void
+generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
+ RelOptInfo *rel_plain, RelAggInfo *agg_info)
+{
+ ListCell *lc;
+
+ if (IS_DUMMY_REL(rel_plain))
+ {
+ mark_dummy_rel(rel_grouped);
+ return;
+ }
+
+ foreach(lc, rel_plain->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ /*
+ * Since the path originates from the non-grouped relation which is
+ * not aware of the aggregate push-down, we must ensure that it
+ * provides the correct input for aggregation.
+ */
+ path = (Path *) create_projection_path(root, rel_grouped, path,
+ agg_info->agg_input);
+
+ /*
+ * add_grouped_path() will check whether the path has suitable
+ * pathkeys.
+ */
+ add_grouped_path(root, rel_grouped, path, AGG_SORTED, agg_info);
+
+ /*
+ * Repeated creation of hash table (for new parameter values) should
+ * be possible, does not sound like a good idea in terms of
+ * efficiency.
+ */
+ if (path->param_info == NULL)
+ add_grouped_path(root, rel_grouped, path, AGG_HASHED, agg_info);
+ }
+
+ /* Could not generate any grouped paths? */
+ if (rel_grouped->pathlist == NIL)
+ mark_dummy_rel(rel_grouped);
+}
+
+/*
+ * Apply partial aggregation to a subpath and add the AggPath to the pathlist.
+ */
+static void
+add_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ AggStrategy aggstrategy, RelAggInfo *agg_info)
+{
+ Path *agg_path;
+
+
+ if (aggstrategy == AGG_HASHED)
+ agg_path = (Path *) create_agg_hashed_path(root, rel, subpath,
+ agg_info);
+ else if (aggstrategy == AGG_SORTED)
+ agg_path = (Path *) create_agg_sorted_path(root, rel, subpath,
+ agg_info);
+ else
+ elog(ERROR, "unexpected strategy %d", aggstrategy);
+
+ /* Add the grouped path to the list of grouped base paths. */
+ if (agg_path != NULL)
+ add_path(rel, (Path *) agg_path);
+}
+
/*
* make_rel_from_joinlist
* Build access paths using a "joinlist" to guide the join path search.
@@ -2770,6 +2848,34 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
int varno = ((RangeTblRef *) jlnode)->rtindex;
thisrel = find_base_rel(root, varno);
+
+ /*
+ * Create a grouped relation to facilitate aggregate push-down.
+ * This makes no sense if thisrel is the only relation of the
+ * query.
+ */
+ if (bms_nonempty_difference(root->all_baserels, thisrel->relids))
+ {
+ RelOptInfo *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /*
+ * Build grouped relation if thisrel is suitable for partial
+ * aggregation.
+ */
+ rel_grouped = build_simple_grouped_rel(root, varno, &agg_info);
+
+ if (rel_grouped)
+ {
+ /* Make the relation available for joining. */
+ add_grouped_rel(root, rel_grouped, agg_info);
+
+ /* Add the aggregation paths to it. */
+ generate_grouping_paths(root, rel_grouped, thisrel,
+ agg_info);
+ set_cheapest(rel_grouped);
+ }
+ }
}
else if (IsA(jlnode, List))
{
@@ -2871,6 +2977,7 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
for (lev = 2; lev <= levels_needed; lev++)
{
+ RelOptInfo *rel_grouped;
ListCell *lc;
/*
@@ -2907,6 +3014,11 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
+ /* The same for grouped relation if one exists. */
+ rel_grouped = find_grouped_rel(root, rel->relids, NULL);
+ if (rel_grouped)
+ set_cheapest(rel_grouped);
+
#ifdef OPTIMIZER_DEBUG
debug_print_rel(root, rel);
#endif
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index a2a9b1f7be..f4d90b8fcf 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4414,7 +4414,6 @@ set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel)
0,
JOIN_INNER,
NULL);
-
rel->rows = clamp_row_est(nrows);
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
@@ -5392,11 +5391,11 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
foreach(lc, target->exprs)
{
Node *node = (Node *) lfirst(lc);
+ int32 item_width;
if (IsA(node, Var))
{
Var *var = (Var *) node;
- int32 item_width;
/* We should not see any upper-level Vars here */
Assert(var->varlevelsup == 0);
@@ -5427,6 +5426,20 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
Assert(item_width > 0);
tuple_width += item_width;
}
+ else if (IsA(node, Aggref))
+ {
+ /*
+ * If the target is evaluated by AggPath, it'll care of cost
+ * estimate. If the target is above AggPath (typically target of a
+ * join relation that contains grouped relation), the cost of
+ * Aggref should not be accounted for again.
+ *
+ * On the other hand, width is always needed.
+ */
+ item_width = get_typavgwidth(exprType(node), exprTypmod(node));
+ Assert(item_width > 0);
+ tuple_width += item_width;
+ }
else
{
/*
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 688d9b0707..33f2aad19c 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -2542,6 +2542,140 @@ is_redundant_derived_clause(RestrictInfo *rinfo, List *clauselist)
return false;
}
+/*
+ * translate_expression_to_rels
+ * If the appropriate equivalence classes exist, replace vars in
+ * gvi->gvexpr with vars whose varno is equal to relid. Return NULL if
+ * translation is not possible or needed.
+ *
+ * Note: Currently we only translate Var expressions. This is subject to
+ * change as the aggregate push-down feature gets enhanced.
+ */
+GroupedVarInfo *
+translate_expression_to_rels(PlannerInfo *root, GroupedVarInfo *gvi,
+ Index relid)
+{
+ Var *var;
+ ListCell *l1;
+ bool found_orig = false;
+ Var *var_translated = NULL;
+ GroupedVarInfo *result;
+
+ /* Can't do anything w/o equivalence classes. */
+ if (root->eq_classes == NIL)
+ return NULL;
+
+ var = castNode(Var, gvi->gvexpr);
+
+ /*
+ * Do we need to translate the var?
+ */
+ if (var->varno == relid)
+ return NULL;
+
+ /*
+ * Find the replacement var.
+ */
+ foreach(l1, root->eq_classes)
+ {
+ EquivalenceClass *ec = lfirst_node(EquivalenceClass, l1);
+ ListCell *l2;
+
+ /* TODO Check if any other EC kind should be ignored. */
+ if (ec->ec_has_volatile || ec->ec_below_outer_join || ec->ec_broken)
+ continue;
+
+ /* Single-element EC can hardly help in translations. */
+ if (list_length(ec->ec_members) == 1)
+ continue;
+
+ /*
+ * Collect all vars of this EC and their varnos.
+ *
+ * ec->ec_relids does not help because we're only interested in a
+ * subset of EC members.
+ */
+ foreach(l2, ec->ec_members)
+ {
+ EquivalenceMember *em = lfirst_node(EquivalenceMember, l2);
+ Var *ec_var;
+
+ /*
+ * The grouping expressions derived here are used to evaluate
+ * possibility to push aggregation down to RELOPT_BASEREL or
+ * RELOPT_JOINREL relations, and to construct reltargets for the
+ * grouped rels. We're not interested at the moment whether the
+ * relations do have children.
+ */
+ if (em->em_is_child)
+ continue;
+
+ if (!IsA(em->em_expr, Var))
+ continue;
+
+ ec_var = castNode(Var, em->em_expr);
+ if (equal(ec_var, var))
+ found_orig = true;
+ else if (ec_var->varno == relid)
+ var_translated = ec_var;
+
+ if (found_orig && var_translated)
+ {
+ /*
+ * The replacement Var must have the same data type, otherwise
+ * the values are not guaranteed to be grouped in the same way
+ * as values of the original Var.
+ */
+ if (ec_var->vartype != var->vartype)
+ return NULL;
+
+ break;
+ }
+ }
+
+ if (found_orig)
+ {
+ /*
+ * The same expression probably does not exist in multiple ECs.
+ */
+ if (var_translated == NULL)
+ {
+ /*
+ * Failed to translate the expression.
+ */
+ return NULL;
+ }
+ else
+ {
+ /* Success. */
+ break;
+ }
+ }
+ else
+ {
+ /*
+ * Vars of the requested relid can be in the next ECs too.
+ */
+ var_translated = NULL;
+ }
+ }
+
+ if (!found_orig)
+ return NULL;
+
+ result = makeNode(GroupedVarInfo);
+ memcpy(result, gvi, sizeof(GroupedVarInfo));
+
+ /*
+ * translate_expression_to_rels_mutator updates gv_eval_at.
+ */
+ result->gv_eval_at = bms_make_singleton(relid);
+ result->gvexpr = (Expr *) var_translated;
+ result->derived = true;
+
+ return result;
+}
+
/*
* is_redundant_with_indexclauses
* Test whether rinfo is redundant with any clause in the IndexClause
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index a9fc52c61f..5eea155f0c 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -22,6 +22,7 @@
#include "partitioning/partbounds.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+#include "utils/selfuncs.h"
static void make_rels_by_clause_joins(PlannerInfo *root,
@@ -36,7 +37,9 @@ static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1,
- RelOptInfo *rel2);
+ RelOptInfo *rel2,
+ RelAggInfo *agg_info,
+ RelOptInfo *rel_agg_input);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist);
@@ -652,9 +655,17 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
/*
* make_join_rel_common
* The workhorse of make_join_rel().
- */
+ *
+ * 'agg_info' contains the reltarget of grouped relation and everything we
+ * need to aggregate the join result. If NULL, then the join relation should
+ * not be grouped.
+ *
+ * 'rel_agg_input' describes the AggPath input relation if the join output
+ * should be aggregated. If NULL is passed, do not aggregate the join output.
+ */
static RelOptInfo *
-make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ RelAggInfo *agg_info, RelOptInfo *rel_agg_input)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -715,7 +726,7 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* goes with this particular joining.
*/
joinrel = build_join_rel(root, joinrelids, rel1, rel2, sjinfo,
- &restrictlist);
+ &restrictlist, agg_info);
/*
* If we've already proven this join is empty, we needn't consider any
@@ -728,14 +739,88 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
}
/* Add paths to the join relation. */
- populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
- restrictlist);
+ if (rel_agg_input == NULL)
+ {
+ /*
+ * Simply join the input relations, whether both are plain or one of
+ * them is grouped.
+ */
+ populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
+ restrictlist);
+ }
+ else
+ {
+ /* The join relation is grouped. */
+ Assert(agg_info != NULL);
+
+ /*
+ * Apply partial aggregation to the paths of rel_agg_input and add the
+ * resulting paths to joinrel.
+ */
+ generate_grouping_paths(root, joinrel, rel_agg_input, agg_info);
+ }
bms_free(joinrelids);
return joinrel;
}
+/*
+ * make_join_rel_combined
+ * Join grouped relation to non-grouped one.
+ */
+static void
+make_join_rel_combined(PlannerInfo *root, RelOptInfo *rel1,
+ RelOptInfo *rel2,
+ RelAggInfo *agg_info)
+{
+ RelOptInfo *rel1_grouped;
+ RelOptInfo *rel2_grouped;
+ bool rel1_grouped_useful = false;
+ bool rel2_grouped_useful = false;
+
+ /* Retrieve the grouped relations. */
+ rel1_grouped = find_grouped_rel(root, rel1->relids, NULL);
+ rel2_grouped = find_grouped_rel(root, rel2->relids, NULL);
+
+ /*
+ * Dummy rel may indicate a join relation that is able to generate grouped
+ * paths as such (i.e. it has valid agg_info), but for which the path
+ * actually could not be created (e.g. only AGG_HASHED strategy was
+ * possible but work_mem was not sufficient for hash table).
+ */
+ rel1_grouped_useful = rel1_grouped != NULL && !IS_DUMMY_REL(rel1_grouped);
+ rel2_grouped_useful = rel2_grouped != NULL && !IS_DUMMY_REL(rel2_grouped);
+
+ /* Nothing to do if there's no grouped relation. */
+ if (!rel1_grouped_useful && !rel2_grouped_useful)
+ return;
+
+ /*
+ * At maximum one input rel can be grouped (here we don't care if any rel
+ * is eventually dummy, the existence of grouped rel indicates that
+ * aggregates can be pushed down to it). If both were grouped, then
+ * grouping of one side would change the occurrence of the other side's
+ * aggregate transient states on the input of the final aggregation. This
+ * can be handled by adjusting the transient states, but it's not worth
+ * the effort because it's hard to find a use case for this kind of join.
+ *
+ * XXX If the join of two grouped rels is implemented someday, note that
+ * both rels can have aggregates, so it'd be hard to join grouped rel to
+ * non-grouped here: 1) such a "mixed join" would require a special
+ * target, 2) both AGGSPLIT_FINAL_DESERIAL and AGGSPLIT_SIMPLE aggregates
+ * could appear in the target of the final aggregation node, originating
+ * from the grouped and the non-grouped input rel respectively.
+ */
+ if (rel1_grouped && rel2_grouped)
+ return;
+
+ if (rel1_grouped_useful)
+ make_join_rel_common(root, rel1_grouped, rel2, agg_info, NULL);
+ else if (rel2_grouped_useful)
+ make_join_rel_common(root, rel1, rel2_grouped, agg_info, NULL);
+}
+
/*
* make_join_rel
* Find or create a join RelOptInfo that represents the join of
@@ -744,14 +829,83 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* (The join rel may already contain paths generated from other
* pairs of rels that add up to the same set of base rels.)
*
+ * In addition to creating an ordinary join relation, try to create a
+ * grouped one. There are two strategies to achieve that: join a grouped
+ * relation to plain one, or join two plain relations and apply partial
+ * aggregation to the result.
+ *
* NB: will return NULL if attempted join is not valid. This can happen when
* working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
+ * turned into joins. Besides that, NULL is also returned if caller is
+ * interested in a grouped relation but it could not be created.
+ *
+ * Only the plain relation is returned; if grouped relation exists, it can be
+ * retrieved using find_grouped_rel().
*/
RelOptInfo *
make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
{
- return make_join_rel_common(root, rel1, rel2);
+ Relids joinrelids;
+ RelAggInfo *agg_info = NULL;
+ RelOptInfo *joinrel,
+ *joinrel_plain;
+
+ /* 1) form the plain join. */
+ joinrel = make_join_rel_common(root, rel1, rel2, NULL, NULL);
+ joinrel_plain = joinrel;
+
+ if (joinrel_plain == NULL)
+ return joinrel_plain;
+
+ /*
+ * We're done if there are no grouping expressions nor aggregates.
+ */
+ if (root->grouped_var_list == NIL)
+ return joinrel_plain;
+
+ joinrelids = bms_union(rel1->relids, rel2->relids);
+ joinrel = find_grouped_rel(root, joinrelids, &agg_info);
+
+ if (joinrel != NULL)
+ {
+ /*
+ * If the same grouped joinrel was already formed, just with the base
+ * rels divided between rel1 and rel2 in a different way, the matching
+ * agg_info should already be there.
+ */
+ Assert(agg_info != NULL);
+ }
+ else
+ {
+ /*
+ * agg_info must be created from scratch.
+ */
+ agg_info = create_rel_agg_info(root, joinrel_plain);
+
+ /* Cannot we build grouped join? */
+ if (agg_info == NULL)
+ return joinrel_plain;
+
+ /*
+ * The number of aggregate input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ agg_info->input_rows = joinrel_plain->rows;
+ }
+
+ /*
+ * 2) join two plain rels and aggregate the join paths. Aggregate
+ * push-down only makes sense if the join is not the top-level one.
+ */
+ if (bms_nonempty_difference(root->all_baserels, joinrelids))
+ make_join_rel_common(root, rel1, rel2, agg_info, joinrel_plain);
+
+ /*
+ * 3) combine plain and grouped relations.
+ */
+ make_join_rel_combined(root, rel1, rel2, agg_info);
+
+ return joinrel_plain;
}
/*
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 73da0c2d8e..9f97a2dd8f 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -48,6 +48,8 @@ typedef struct PostponedQual
} PostponedQual;
+static void create_aggregate_grouped_var_infos(PlannerInfo *root);
+static void create_grouping_expr_grouped_var_infos(PlannerInfo *root);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
@@ -273,6 +275,261 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
}
}
+/*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate as well as for
+ * each possible grouping expression.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+extern void
+setup_aggregate_pushdown(PlannerInfo *root)
+{
+ ListCell *lc;
+
+ /*
+ * Isn't user interested in the aggregate push-down feature?
+ */
+ if (!enable_agg_pushdown)
+ return;
+
+ /* The feature can only be applied to grouped aggregation. */
+ if (!root->parse->groupClause)
+ return;
+
+ /*
+ * Grouping sets require multiple different groupings but the base
+ * relation can only generate one.
+ */
+ if (root->parse->groupingSets)
+ return;
+
+ /*
+ * SRF is not allowed in the aggregate argument and we don't even want it
+ * in the GROUP BY clause, so forbid it in general. It needs to be
+ * analyzed if evaluation of a GROUP BY clause containing SRF below the
+ * query targetlist would be correct. Currently it does not seem to be an
+ * important use case.
+ */
+ if (root->parse->hasTargetSRFs)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_aggregate_grouped_var_infos(root);
+
+ /* Isn't there any aggregate to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Create GroupedVarInfo per grouping expression. */
+ create_grouping_expr_grouped_var_infos(root);
+
+ /* Isn't there any useful grouping expression for aggregate push-down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /*
+ * Now that we know that grouping can be pushed down, search for the
+ * maximum sortgroupref. The base relations may need it if extra grouping
+ * expressions get added to them.
+ */
+ Assert(root->max_sortgroupref == 0);
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc);
+
+ if (te->ressortgroupref > root->max_sortgroupref)
+ root->max_sortgroupref = te->ressortgroupref;
+ }
+}
+
+/*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ */
+static void
+create_aggregate_grouped_var_infos(PlannerInfo *root)
+{
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES |
+ PVC_INCLUDE_WINDOWFUNCS);
+
+ /*
+ * Although GroupingFunc is related to root->parse->groupingSets, this
+ * field does not necessarily reflect its presence.
+ */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, GroupingFunc))
+ return;
+ }
+
+ /*
+ * Aggregates within the HAVING clause need to be processed in the same
+ * way as those in the main targetlist.
+ *
+ * Note that the contained aggregates will be pushed down, but the
+ * containing HAVING clause must be ignored until the aggregation is
+ * finalized.
+ */
+ if (root->parse->havingQual != NULL)
+ {
+ List *having_exprs;
+
+ having_exprs = pull_var_clause((Node *) root->parse->havingQual,
+ PVC_INCLUDE_AGGREGATES);
+ if (having_exprs != NIL)
+ tlist_exprs = list_concat(tlist_exprs, having_exprs);
+ }
+
+ if (tlist_exprs == NIL)
+ return;
+
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ /*
+ * tlist_exprs may also contain Vars or WindowFuncs, but we only need
+ * Aggrefs.
+ */
+ if (IsA(expr, Var) ||IsA(expr, WindowFunc))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct)
+ {
+ /*
+ * Aggregation push-down is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ gvi = lfirst_node(GroupedVarInfo, lc2);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ gvi = makeNode(GroupedVarInfo);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ gvi->gv_eval_at = NULL;
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+}
+
+/*
+ * Create GroupedVarInfo for each expression usable as grouping key.
+ *
+ * In addition to the expressions of the query targetlist, group_pathkeys is
+ * also considered the source of grouping expressions. That increases the
+ * chance to get the relation output grouped.
+ */
+static void
+create_grouping_expr_grouped_var_infos(PlannerInfo *root)
+{
+ ListCell *l1,
+ *l2;
+ List *exprs = NIL;
+ List *sortgrouprefs = NIL;
+
+ /*
+ * Make sure GroupedVarInfo exists for each expression usable as grouping
+ * key.
+ */
+ foreach(l1, root->parse->groupClause)
+ {
+ SortGroupClause *sgClause;
+ TargetEntry *te;
+ Index sortgroupref;
+
+ sgClause = lfirst_node(SortGroupClause, l1);
+ te = get_sortgroupclause_tle(sgClause, root->processed_tlist);
+ sortgroupref = te->ressortgroupref;
+
+ Assert(sortgroupref > 0);
+
+ /*
+ * Non-zero sortgroupref does not necessarily imply grouping
+ * expression: data can also be sorted by aggregate.
+ */
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ /*
+ * The aggregate push-down feature currently supports only plain Vars
+ * as grouping expressions.
+ */
+ if (!IsA(te->expr, Var))
+ {
+ root->grouped_var_list = NIL;
+ return;
+ }
+
+ exprs = lappend(exprs, te->expr);
+ sortgrouprefs = lappend_int(sortgrouprefs, sortgroupref);
+ }
+
+ /*
+ * Construct GroupedVarInfo for each expression.
+ */
+ forboth(l1, exprs, l2, sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, l1);
+ int sortgroupref = lfirst_int(l2);
+ GroupedVarInfo *gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvexpr = (Expr *) copyObject(var);
+ gvi->sortgroupref = sortgroupref;
+
+ /* Find out where the expression should be evaluated. */
+ gvi->gv_eval_at = bms_make_singleton(var->varno);
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+}
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c
index 9381939c82..c4d07dfac4 100644
--- a/src/backend/optimizer/plan/planagg.c
+++ b/src/backend/optimizer/plan/planagg.c
@@ -442,7 +442,7 @@ build_minmax_path(PlannerInfo *root, MinMaxAggInfo *mminfo,
subroot->tuple_fraction = 1.0;
subroot->limit_tuples = 1.0;
- final_rel = query_planner(subroot, minmax_qp_callback, NULL);
+ final_rel = query_planner(subroot, minmax_qp_callback, NULL, NULL);
/*
* Since we didn't go through subquery_planner() to handle the subquery,
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 0b9999c8a6..4103a393e6 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -45,6 +45,9 @@
* qp_callback is a function to compute query_pathkeys once it's safe to do so
* qp_extra is optional extra data to pass to qp_callback
*
+ * If final_rel_grouped_p is valid, relation containing grouped paths may be
+ * saved to *final_rel_grouped_p.
+ *
* Note: the PlannerInfo node also includes a query_pathkeys field, which
* tells query_planner the sort order that is desired in the final output
* plan. This value is *not* available at call time, but is computed by
@@ -53,7 +56,8 @@
*/
RelOptInfo *
query_planner(PlannerInfo *root,
- query_pathkeys_callback qp_callback, void *qp_extra)
+ query_pathkeys_callback qp_callback, void *qp_extra,
+ RelOptInfo **final_rel_grouped_p)
{
Query *parse = root->parse;
List *joinlist;
@@ -66,6 +70,8 @@ query_planner(PlannerInfo *root,
* here.
*/
root->join_rel_list = makeNode(RelInfoList);
+ root->grouped_rel_list = makeNode(RelInfoList);
+ root->agg_info_list = makeNode(RelInfoList);
root->join_rel_level = NULL;
root->join_cur_level = 0;
root->canon_pathkeys = NIL;
@@ -74,6 +80,7 @@ query_planner(PlannerInfo *root,
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
@@ -254,6 +261,16 @@ query_planner(PlannerInfo *root,
*/
extract_restriction_or_clauses(root);
+ /*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, setup
+ * root->grouped_var_list.
+ *
+ * The base relations should be fully initialized now, so that we have
+ * enough info to decide whether grouping is possible.
+ */
+ setup_aggregate_pushdown(root);
+
/*
* Now expand appendrels by adding "otherrels" for their children. We
* delay this to the end so that we have as much information as possible
@@ -274,5 +291,9 @@ query_planner(PlannerInfo *root,
final_rel->cheapest_total_path->param_info != NULL)
elog(ERROR, "failed to construct the join relation");
+ if (final_rel_grouped_p)
+ *final_rel_grouped_p = find_grouped_rel(root, final_rel->relids,
+ NULL);
+
return final_rel;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 401299e542..b99b979d95 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -151,6 +151,7 @@ static double get_number_of_groups(PlannerInfo *root,
List *target_list);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
PathTarget *target,
bool target_parallel_safe,
const AggClauseCosts *agg_costs,
@@ -164,6 +165,7 @@ static RelOptInfo *make_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
Node *havingQual);
static void create_ordinary_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
RelOptInfo *grouped_rel,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd,
@@ -623,6 +625,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
memset(root->upper_rels, 0, sizeof(root->upper_rels));
memset(root->upper_targets, 0, sizeof(root->upper_targets));
root->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
root->grouping_map = NULL;
root->minmax_aggs = NIL;
root->qual_security_level = 0;
@@ -1943,6 +1946,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
List *activeWindows = NIL;
grouping_sets_data *gset_data = NULL;
standard_qp_extra qp_extra;
+ RelOptInfo *current_rel_grouped = NULL;
/* A recursive query should always have setOperations */
Assert(!root->hasRecursion);
@@ -2045,7 +2049,9 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
* We also generate (in standard_qp_callback) pathkey representations
* of the query's sort clause, distinct clause, etc.
*/
- current_rel = query_planner(root, standard_qp_callback, &qp_extra);
+ current_rel = query_planner(root,
+ standard_qp_callback, &qp_extra,
+ ¤t_rel_grouped);
/*
* Convert the query's result tlist into PathTarget format.
@@ -2190,6 +2196,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
{
current_rel = create_grouping_paths(root,
current_rel,
+ current_rel_grouped,
grouping_target,
grouping_target_parallel_safe,
&agg_costs,
@@ -3798,6 +3805,7 @@ get_number_of_groups(PlannerInfo *root,
static RelOptInfo *
create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
PathTarget *target,
bool target_parallel_safe,
const AggClauseCosts *agg_costs,
@@ -3888,7 +3896,8 @@ create_grouping_paths(PlannerInfo *root,
else
extra.patype = PARTITIONWISE_AGGREGATE_NONE;
- create_ordinary_grouping_paths(root, input_rel, grouped_rel,
+ create_ordinary_grouping_paths(root, input_rel, input_rel_grouped,
+ grouped_rel,
agg_costs, gd, &extra,
&partially_grouped_rel);
}
@@ -4046,6 +4055,7 @@ create_degenerate_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
*/
static void
create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
RelOptInfo *grouped_rel,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd,
@@ -4093,13 +4103,23 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
if ((extra->flags & GROUPING_CAN_PARTIAL_AGG) != 0)
{
bool force_rel_creation;
+ bool have_agg_pushdown_paths;
/*
- * If we're doing partitionwise aggregation at this level, force
- * creation of a partially_grouped_rel so we can add partitionwise
- * paths to it.
+ * Check if the aggregate push-down feature succeeded to generate any
+ * paths. Dummy relation can appear here because grouped paths are not
+ * guaranteed to exist for a relation.
*/
- force_rel_creation = (patype == PARTITIONWISE_AGGREGATE_PARTIAL);
+ have_agg_pushdown_paths = input_rel_grouped != NULL &&
+ !IS_DUMMY_REL(input_rel_grouped);
+
+ /*
+ * If we're doing partitionwise aggregation at this level or if
+ * aggregate push-down succeeded to create some paths, force creation
+ * of a partially_grouped_rel so we can add the related paths to it.
+ */
+ force_rel_creation = patype == PARTITIONWISE_AGGREGATE_PARTIAL ||
+ have_agg_pushdown_paths;
partially_grouped_rel =
create_partial_grouping_paths(root,
@@ -4108,6 +4128,23 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
gd,
extra,
force_rel_creation);
+
+ /*
+ * No further processing is needed for paths provided by the aggregate
+ * push-down feature. Simply add them to the partially grouped
+ * relation.
+ */
+ if (have_agg_pushdown_paths)
+ {
+ ListCell *lc;
+
+ foreach(lc, input_rel_grouped->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ add_path(partially_grouped_rel, path);
+ }
+ }
}
/* Set out parameter. */
@@ -4132,10 +4169,14 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
/* Gather any partially grouped partial paths. */
if (partially_grouped_rel && partially_grouped_rel->partial_pathlist)
- {
gather_grouping_paths(root, partially_grouped_rel);
+
+ /*
+ * The non-partial paths can come either from the Gather above or from
+ * aggregate push-down.
+ */
+ if (partially_grouped_rel && partially_grouped_rel->pathlist)
set_cheapest(partially_grouped_rel);
- }
/*
* Estimate number of groups.
@@ -7296,6 +7337,7 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
/* Create grouping paths for this child relation. */
create_ordinary_grouping_paths(root, child_input_rel,
+ NULL,
child_grouped_rel,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index dc11f098e0..3810345b3d 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -2380,6 +2380,39 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, Aggref))
+ {
+ Aggref *aggref = castNode(Aggref, node);
+
+ /*
+ * The upper plan targetlist can contain Aggref whose value has
+ * already been evaluated by the subplan. However this can only happen
+ * with specific value of aggsplit.
+ */
+ if (aggref->aggsplit == AGGSPLIT_INITIAL_SERIAL)
+ {
+ /* See if the Aggref has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ }
+
+ /* No referent found for Aggref */
+ elog(ERROR, "Aggref not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 67eeba938d..6cb33445e2 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -891,6 +891,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
memset(subroot->upper_rels, 0, sizeof(subroot->upper_rels));
memset(subroot->upper_targets, 0, sizeof(subroot->upper_targets));
subroot->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
subroot->grouping_map = NULL;
subroot->minmax_aggs = NIL;
subroot->qual_security_level = 0;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index d884d2bb00..8019ad7bd2 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2552,8 +2552,7 @@ create_projection_path(PlannerInfo *root,
pathnode->path.pathtype = T_Result;
pathnode->path.parent = rel;
pathnode->path.pathtarget = target;
- /* For now, assume we are above any joins, so no parameterization */
- pathnode->path.param_info = NULL;
+ pathnode->path.param_info = subpath->param_info;
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe &&
@@ -2992,6 +2991,146 @@ create_agg_path(PlannerInfo *root,
return pathnode;
}
+/*
+ * Apply AGG_SORTED aggregation path to subpath if it's suitably sorted.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+AggPath *
+create_agg_sorted_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ RelAggInfo *agg_info)
+{
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ ListCell *lc1;
+ List *key_subset = NIL;
+ AggPath *result = NULL;
+
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, (Node *) agg_exprs, aggsplit, &agg_costs);
+
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until the
+ * final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath, target,
+ AGG_SORTED, aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+
+ /* The agg path should require no fewer parameters than the plain one. */
+ result->path.param_info = subpath->param_info;
+
+ return result;
+}
+
+/*
+ * Apply AGG_HASHED aggregation to subpath.
+ */
+AggPath *
+create_agg_hashed_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, RelAggInfo *agg_info)
+{
+ bool can_hash;
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ double hashaggtablesize;
+ Query *parse = root->parse;
+ AggPath *result = NULL;
+
+ /* Do not try to create hash table for each parameter value. */
+ Assert(subpath->param_info == NULL);
+
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, agg_exprs, aggsplit, &agg_costs);
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until
+ * the final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath,
+ target,
+ AGG_HASHED,
+ aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+ }
+ }
+
+ return result;
+}
+
/*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index c238dd6538..10becc09e9 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -17,18 +17,24 @@
#include <limits.h>
#include "miscadmin.h"
+#include "catalog/pg_class_d.h"
+#include "catalog/pg_constraint.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
+#include "optimizer/optimizer.h"
#include "optimizer/inherit.h"
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/placeholder.h"
#include "optimizer/plancat.h"
+#include "optimizer/planner.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+#include "parser/parse_oper.h"
#include "partitioning/partbounds.h"
#include "utils/hsearch.h"
+#include "utils/selfuncs.h"
typedef struct RelInfoEntry
@@ -63,6 +69,9 @@ static void build_child_join_reltarget(PlannerInfo *root,
RelOptInfo *childrel,
int nappinfos,
AppendRelInfo **appinfos);
+static bool init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p);
/*
@@ -351,6 +360,102 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
return rel;
}
+/*
+ * build_simple_grouped_rel
+ * Construct a new RelOptInfo for a grouped base relation out of an
+ * existing non-grouped relation. On success, pointer to the corresponding
+ * RelAggInfo is stored in *agg_info_p in addition to returning the grouped
+ * relation.
+ */
+RelOptInfo *
+build_simple_grouped_rel(PlannerInfo *root, int relid,
+ RelAggInfo **agg_info_p)
+{
+ RangeTblEntry *rte;
+ RelOptInfo *rel_plain,
+ *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /* Isn't there any grouping expression to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return NULL;
+
+ rel_plain = root->simple_rel_array[relid];
+
+ /* Caller should only pass rti that represents base relation. */
+ Assert(rel_plain != NULL);
+
+ /*
+ * Not all RTE kinds are supported when grouping is considered.
+ *
+ * TODO Consider relaxing some of these restrictions.
+ */
+ rte = root->simple_rte_array[rel_plain->relid];
+ if (rte->rtekind != RTE_RELATION ||
+ rte->relkind == RELKIND_FOREIGN_TABLE ||
+ rte->tablesample != NULL)
+ return NULL;
+
+ /*
+ * Grouped append relation is not supported yet.
+ */
+ if (rte->inh)
+ return NULL;
+
+ /*
+ * Currently we do not support child relations ("other rels").
+ */
+ if (rel_plain->reloptkind != RELOPT_BASEREL)
+ return NULL;
+
+ /*
+ * Prepare the information we need for aggregation of the rel contents.
+ */
+ agg_info = create_rel_agg_info(root, rel_plain);
+ if (agg_info == NULL)
+ return NULL;
+
+ /*
+ * TODO Consider if 1) a flat copy is o.k., 2) it's safer in terms of
+ * adding new fields to RelOptInfo) to copy everything and then reset some
+ * fields, or to zero the structure and copy individual fields.
+ */
+ rel_grouped = makeNode(RelOptInfo);
+ memcpy(rel_grouped, rel_plain, sizeof(RelOptInfo));
+
+ /*
+ * Note on consider_startup: while the AGG_HASHED strategy needs the whole
+ * relation, AGG_SORTED does not. Therefore we do not force
+ * consider_startup to false.
+ */
+
+ /*
+ * Set the appropriate target for grouped paths.
+ *
+ * reltarget should match the target of partially aggregated paths.
+ */
+ rel_grouped->reltarget = agg_info->target;
+
+ /*
+ * Grouped paths must not be mixed with the plain ones.
+ */
+ rel_grouped->pathlist = NIL;
+ rel_grouped->partial_pathlist = NIL;
+ rel_grouped->cheapest_startup_path = NULL;
+ rel_grouped->cheapest_total_path = NULL;
+ rel_grouped->cheapest_unique_path = NULL;
+ rel_grouped->cheapest_parameterized_paths = NIL;
+
+ /*
+ * The number of aggregation input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ agg_info->input_rows = rel_plain->rows;
+
+ *agg_info_p = agg_info;
+ return rel_grouped;
+}
+
/*
* find_base_rel
* Find a base or other relation entry, which must already exist.
@@ -462,8 +567,12 @@ find_rel_info(RelInfoList *list, Relids relids)
void *item = lfirst(l);
Relids item_relids;
- Assert(IsA(item, RelOptInfo));
- item_relids = ((RelOptInfo *) item)->relids;
+ Assert(IsA(item, RelOptInfo) ||IsA(item, RelAggInfo));
+
+ if (IsA(item, RelOptInfo))
+ item_relids = ((RelOptInfo *) item)->relids;
+ else if (IsA(item, RelAggInfo))
+ item_relids = ((RelAggInfo *) item)->relids;
if (bms_equal(item_relids, relids))
return item;
@@ -492,7 +601,7 @@ find_join_rel(PlannerInfo *root, Relids relids)
static void
add_rel_info(RelInfoList *list, void *data)
{
- Assert(IsA(data, RelOptInfo));
+ Assert(IsA(data, RelOptInfo) ||IsA(data, RelAggInfo));
/* GEQO requires us to append the new joinrel to the end of the list! */
list->items = lappend(list->items, data);
@@ -504,7 +613,11 @@ add_rel_info(RelInfoList *list, void *data)
RelInfoEntry *hentry;
bool found;
- relids = ((RelOptInfo *) data)->relids;
+ if (IsA(data, RelOptInfo))
+ relids = ((RelOptInfo *) data)->relids;
+ else if (IsA(data, RelAggInfo))
+ relids = ((RelAggInfo *) data)->relids;
+
hentry = (RelInfoEntry *) hash_search(list->hash,
&relids,
HASH_ENTER,
@@ -525,6 +638,57 @@ add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
add_rel_info(root->join_rel_list, joinrel);
}
+/*
+ * add_grouped_rel
+ * Add grouped base or join relation to the list of grouped relations in
+ * the given PlannerInfo. Also add the corresponding RelAggInfo to
+ * agg_info_list.
+ */
+void
+add_grouped_rel(PlannerInfo *root, RelOptInfo *rel, RelAggInfo *agg_info)
+{
+ add_rel_info(root->grouped_rel_list, rel);
+ add_rel_info(root->agg_info_list, rel);
+}
+
+/*
+ * find_grouped_rel
+ * Returns grouped relation entry (base or join relation) corresponding to
+ * 'relids' or NULL if none exists.
+ *
+ * If agg_info_p is a valid pointer, then pointer to RelAggInfo that
+ * corresponds to the relation returned is assigned to *agg_info_p.
+ */
+RelOptInfo *
+find_grouped_rel(PlannerInfo *root, Relids relids, RelAggInfo **agg_info_p)
+{
+ RelOptInfo *rel;
+
+ rel = (RelOptInfo *) find_rel_info(root->grouped_rel_list, relids);
+ if (rel == NULL)
+ {
+ if (agg_info_p)
+ *agg_info_p = NULL;
+
+ return NULL;
+ }
+
+ /* Is caller interested in RelAggInfo? */
+ if (agg_info_p)
+ {
+ RelAggInfo *agg_info;
+
+ agg_info = (RelAggInfo *) find_rel_info(root->agg_info_list, relids);
+
+ /* The relation exists, so the agg_info should be there too. */
+ Assert(agg_info != NULL);
+
+ *agg_info_p = agg_info;
+ }
+
+ return rel;
+}
+
/*
* set_foreign_rel_properties
* Set up foreign-join fields if outer and inner relation are foreign
@@ -587,6 +751,7 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* 'restrictlist_ptr': result variable. If not NULL, *restrictlist_ptr
* receives the list of RestrictInfo nodes that apply to this
* particular pair of joinable relations.
+ * 'agg_info' indicates that grouped join relation should be created.
*
* restrictlist_ptr makes the routine's API a little grotty, but it saves
* duplicated calculation of the restrictlist...
@@ -597,10 +762,12 @@ build_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr)
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info)
{
RelOptInfo *joinrel;
List *restrictlist;
+ bool grouped = agg_info != NULL;
/* This function should be used only for join between parents. */
Assert(!IS_OTHER_REL(outer_rel) && !IS_OTHER_REL(inner_rel));
@@ -608,7 +775,8 @@ build_join_rel(PlannerInfo *root,
/*
* See if we already have a joinrel for this set of base rels.
*/
- joinrel = find_join_rel(root, joinrelids);
+ joinrel = !grouped ? find_join_rel(root, joinrelids) :
+ find_grouped_rel(root, joinrelids, NULL);
if (joinrel)
{
@@ -700,9 +868,21 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ if (!grouped)
+ {
+ joinrel->reltarget = create_empty_pathtarget();
+ build_joinrel_tlist(root, joinrel, outer_rel);
+ build_joinrel_tlist(root, joinrel, inner_rel);
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ }
+ else
+ {
+ /*
+ * The target for grouped join should already have its cost and width
+ * computed, see create_rel_agg_info().
+ */
+ joinrel->reltarget = agg_info->target;
+ }
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -734,49 +914,73 @@ build_join_rel(PlannerInfo *root,
joinrel->has_eclass_joins = has_relevant_eclass_joinclause(root, joinrel);
/* Store the partition information. */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- sjinfo->jointype);
-
- /*
- * Set estimates of the joinrel's size.
- */
- set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
- sjinfo, restrictlist);
-
- /*
- * Set the consider_parallel flag if this joinrel could potentially be
- * scanned within a parallel worker. If this flag is false for either
- * inner_rel or outer_rel, then it must be false for the joinrel also.
- * Even if both are true, there might be parallel-restricted expressions
- * in the targetlist or quals.
- *
- * Note that if there are more than two rels in this relation, they could
- * be divided between inner_rel and outer_rel in any arbitrary way. We
- * assume this doesn't matter, because we should hit all the same baserels
- * and joinclauses while building up to this joinrel no matter which we
- * take; therefore, we should make the same decision here however we get
- * here.
- */
- if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
- is_parallel_safe(root, (Node *) restrictlist) &&
- is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
- joinrel->consider_parallel = true;
+ if (!grouped)
+ build_joinrel_partition_info(joinrel, outer_rel, inner_rel,
+ restrictlist, sjinfo->jointype);
/* Add the joinrel to the PlannerInfo. */
- add_join_rel(root, joinrel);
+ if (!grouped)
+ add_join_rel(root, joinrel);
+ else
+ add_grouped_rel(root, joinrel, agg_info);
/*
- * Also, if dynamic-programming join search is active, add the new joinrel
- * to the appropriate sublist. Note: you might think the Assert on number
- * of members should be for equality, but some of the level 1 rels might
- * have been joinrels already, so we can only assert <=.
+ * Also, if dynamic-programming join search is active, add the new
+ * joinrelset to the appropriate sublist. Note: you might think the
+ * Assert on number of members should be for equality, but some of the
+ * level 1 rels might have been joinrels already, so we can only assert
+ * <=.
+ *
+ * Do noting for grouped relation as it's stored aside from
+ * join_rel_level.
*/
- if (root->join_rel_level)
+ if (root->join_rel_level && !grouped)
{
Assert(root->join_cur_level > 0);
- Assert(root->join_cur_level <= bms_num_members(joinrel->relids));
+ Assert(root->join_cur_level <= bms_num_members(joinrelids));
root->join_rel_level[root->join_cur_level] =
- lappend(root->join_rel_level[root->join_cur_level], joinrel);
+ lappend(root->join_rel_level[root->join_cur_level],
+ joinrel);
+ }
+
+ /* Set estimates of the joinrel's size. */
+ if (!grouped)
+ {
+ set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
+ sjinfo, restrictlist);
+
+ /*
+ * Set the consider_parallel flag if this joinrel could potentially be
+ * scanned within a parallel worker. If this flag is false for either
+ * inner_rel or outer_rel, then it must be false for the joinrel also.
+ * Even if both are true, there might be parallel-restricted
+ * expressions in the targetlist or quals.
+ *
+ * Note that if there are more than two rels in this relation, they
+ * could be divided between inner_rel and outer_rel in any arbitrary
+ * way. We assume this doesn't matter, because we should hit all the
+ * same baserels and joinclauses while building up to this joinrel no
+ * matter which we take; therefore, we should make the same decision
+ * here however we get here.
+ */
+ if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
+ is_parallel_safe(root, (Node *) restrictlist) &&
+ is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
+ joinrel->consider_parallel = true;
+ }
+ else
+ {
+ /*
+ * Grouping essentially changes the number of rows.
+ *
+ * XXX We do not distinguish whether two plain rels are joined and the
+ * result is aggregated, or the aggregation has been already applied
+ * to one of the input rels. Is this worth extra effort, e.g.
+ * maintaining a separate RelOptInfo for each case (one difficulty
+ * that would introduce is construction of AppendPath)?
+ */
+ joinrel->rows = estimate_num_groups(root, agg_info->group_exprs,
+ agg_info->input_rows, NULL);
}
return joinrel;
@@ -1820,3 +2024,625 @@ build_child_join_reltarget(PlannerInfo *root,
childrel->reltarget->cost.per_tuple = parentrel->reltarget->cost.per_tuple;
childrel->reltarget->width = parentrel->reltarget->width;
}
+
+/*
+ * Check if the relation can produce grouped paths and return the information
+ * it'll need for it. The passed relation is the non-grouped one which has the
+ * reltarget already constructed.
+ */
+RelAggInfo *
+create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel)
+{
+ List *gvis;
+ List *aggregates = NIL;
+ bool found_other_rel_agg;
+ ListCell *lc;
+ RelAggInfo *result;
+ PathTarget *agg_input;
+ PathTarget *target = NULL;
+ List *grp_exprs_extra = NIL;
+ List *group_clauses_final;
+ int i;
+
+ /*
+ * The function shouldn't have been called if there's no opportunity for
+ * aggregation push-down.
+ */
+ Assert(root->grouped_var_list != NIL);
+
+ result = makeNode(RelAggInfo);
+
+ /*
+ * The current implementation of aggregation push-down cannot handle
+ * PlaceHolderVar (PHV).
+ *
+ * If we knew that the PHV should be evaluated in this target (and of
+ * course, if its expression matched some Aggref argument), we'd just let
+ * init_grouping_targets add that Aggref. On the other hand, if we knew
+ * that the PHV is evaluated below the current rel, we could ignore it
+ * because the referencing Aggref would take care of propagation of the
+ * value to upper joins.
+ *
+ * The problem is that the same PHV can be evaluated in the target of the
+ * current rel or in that of lower rel --- depending on the input paths.
+ * For example, consider rel->relids = {A, B, C} and if ph_eval_at = {B,
+ * C}. Path "A JOIN (B JOIN C)" implies that the PHV is evaluated by the
+ * "(B JOIN C)", while path "(A JOIN B) JOIN C" evaluates the PHV itself.
+ */
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *expr = lfirst(lc);
+
+ if (IsA(expr, PlaceHolderVar))
+ return NULL;
+ }
+
+ if (IS_SIMPLE_REL(rel))
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return NULL;
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the Agg plan
+ * would receive different input at the base rel level.
+ *
+ * XXX For RELOPT_JOINREL, do not return if all the joins that can set any
+ * entry of the grouped target (do we need to postpone this check until
+ * the grouped target is available, and init_grouping_targets take care?)
+ * of this rel to NULL are provably below rel. (It's ok if rel is one of
+ * these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return NULL;
+
+ /*
+ * Use equivalence classes to generate additional grouping expressions for
+ * the current rel. Without these we might not be able to apply
+ * aggregation to the relation result set.
+ *
+ * It's important that create_grouping_expr_grouped_var_infos has
+ * processed the explicit grouping columns by now. If the grouping clause
+ * contains multiple expressions belonging to the same EC, the original
+ * (i.e. not derived) one should be preferred when we build grouping
+ * target for a relation. Otherwise we have a problem when trying to match
+ * target entries to grouping clauses during plan creation, see
+ * get_grouping_expression().
+ */
+ gvis = list_copy(root->grouped_var_list);
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+ int relid = -1;
+
+ /* Only interested in grouping expressions. */
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ while ((relid = bms_next_member(rel->relids, relid)) >= 0)
+ {
+ GroupedVarInfo *gvi_trans;
+
+ gvi_trans = translate_expression_to_rels(root, gvi, relid);
+ if (gvi_trans != NULL)
+ gvis = lappend(gvis, gvi_trans);
+ }
+ }
+
+ /*
+ * Check if some aggregates or grouping expressions can be evaluated in
+ * this relation's target, and collect all vars referenced by these
+ * aggregates / grouping expressions;
+ */
+ found_other_rel_agg = false;
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which includes
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * init_grouping_targets will handle plain Var grouping
+ * expressions because it needs to look them up in
+ * grouped_var_list anyway.
+ */
+ if (IsA(gvi->gvexpr, Var))
+ continue;
+
+ /*
+ * Currently, GroupedVarInfo only handles Vars and Aggrefs.
+ */
+ Assert(IsA(gvi->gvexpr, Aggref));
+
+ /* We only derive grouped expressions using ECs, not aggregates */
+ Assert(!gvi->derived);
+
+ gvi->agg_partial = (Aggref *) copyObject(gvi->gvexpr);
+ mark_partial_aggref(gvi->agg_partial, AGGSPLIT_INITIAL_SERIAL);
+
+ /*
+ * Accept the aggregate.
+ */
+ aggregates = lappend(aggregates, gvi);
+ }
+ else if (IsA(gvi->gvexpr, Aggref))
+ {
+ /*
+ * Remember that there is at least one aggregate expression that
+ * needs something else than this rel.
+ */
+ found_other_rel_agg = true;
+
+ /*
+ * This condition effectively terminates creation of the
+ * RelAggInfo, so there's no reason to check the next
+ * GroupedVarInfo.
+ */
+ break;
+ }
+ }
+
+ /*
+ * Grouping makes little sense w/o aggregate function and w/o grouping
+ * expressions.
+ */
+ if (aggregates == NIL)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Give up if some other aggregate(s) need relations other than the
+ * current one.
+ *
+ * If the aggregate needs the current rel plus anything else, then the
+ * problem is that grouping of the current relation could make some input
+ * variables unavailable for the "higher aggregate", and it'd also
+ * decrease the number of input rows the "higher aggregate" receives.
+ *
+ * If the aggregate does not even need the current rel, then neither the
+ * current rel nor anything else should be grouped because we do not
+ * support join of two grouped relations.
+ */
+ if (found_other_rel_agg)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Create target for grouped paths as well as one for the input paths of
+ * the aggregation paths.
+ */
+ target = create_empty_pathtarget();
+ agg_input = create_empty_pathtarget();
+
+ /*
+ * Cannot suitable targets for the aggregation push-down be derived?
+ */
+ if (!init_grouping_targets(root, rel, target, agg_input, gvis,
+ &grp_exprs_extra))
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ list_free(gvis);
+
+ /*
+ * Aggregation push-down makes no sense w/o grouping expressions.
+ */
+ if ((list_length(target->exprs) + list_length(grp_exprs_extra)) == 0)
+ return NULL;
+
+ group_clauses_final = root->parse->groupClause;
+
+ /*
+ * If the aggregation target should have extra grouping expressions (in
+ * order to emit input vars for join conditions), add them now. This step
+ * includes assignment of tleSortGroupRef's which we can generate now.
+ */
+ if (list_length(grp_exprs_extra) > 0)
+ {
+ Index sortgroupref;
+
+ /*
+ * We'll have to add some clauses, but query group clause must be
+ * preserved.
+ */
+ group_clauses_final = list_copy(group_clauses_final);
+
+ /*
+ * Always start at root->max_sortgroupref. The extra grouping
+ * expressions aren't used during the final aggregation, so the
+ * sortgroupref values don't need to be unique across the query. Thus
+ * we don't have to increase root->max_sortgroupref, which makes
+ * recognition of the extra grouping expressions pretty easy.
+ */
+ sortgroupref = root->max_sortgroupref;
+
+ /*
+ * Generate the SortGroupClause's and add the expressions to the
+ * target.
+ */
+ foreach(lc, grp_exprs_extra)
+ {
+ Var *var = lfirst_node(Var, lc);
+ SortGroupClause *cl = makeNode(SortGroupClause);
+
+ /*
+ * Initialize the SortGroupClause.
+ *
+ * As the final aggregation will not use this grouping expression,
+ * we don't care whether sortop is < or >. The value of
+ * nulls_first should not matter for the same reason.
+ */
+ cl->tleSortGroupRef = ++sortgroupref;
+ get_sort_group_operators(var->vartype,
+ false, true, false,
+ &cl->sortop, &cl->eqop, NULL,
+ &cl->hashable);
+ group_clauses_final = lappend(group_clauses_final, cl);
+ add_column_to_pathtarget(target, (Expr *) var,
+ cl->tleSortGroupRef);
+
+ /*
+ * The aggregation input target must emit this var too.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) var,
+ cl->tleSortGroupRef);
+ }
+ }
+
+ /*
+ * Add aggregates to the grouping target.
+ */
+ add_aggregates_to_target(root, target, aggregates);
+
+ /*
+ * Build a list of grouping expressions and a list of the corresponding
+ * SortGroupClauses.
+ */
+ i = 0;
+ foreach(lc, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(lc);
+
+ if (IsA(texpr, Aggref))
+ {
+ /*
+ * Once we see Aggref, no grouping expressions should follow.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides being an aggregate, the target expression should have no
+ * other reason then being a column of a relation functionally
+ * dependent on the GROUP BY clause. So it's not actually a grouping
+ * column.
+ */
+ if (sortgroupref == 0)
+ continue;
+
+ /*
+ * group_clause_final contains the "local" clauses, so this search
+ * should succeed.
+ */
+ cl = get_sortgroupref_clause(sortgroupref, group_clauses_final);
+
+ result->group_clauses = list_append_unique(result->group_clauses,
+ cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ result->group_exprs = list_append_unique(result->group_exprs,
+ texpr);
+ }
+
+ /*
+ * Since neither target nor agg_input is supposed to be identical to the
+ * source reltarget, compute the width and cost again.
+ *
+ * target does not yet contain aggregates, but these will be accounted by
+ * AggPath.
+ */
+ set_pathtarget_cost_width(root, target);
+ set_pathtarget_cost_width(root, agg_input);
+
+ result->relids = bms_copy(rel->relids);
+ result->target = target;
+ result->agg_input = agg_input;
+
+ /* Finally collect the aggregates. */
+ while (lc != NULL)
+ {
+ Aggref *aggref = lfirst_node(Aggref, lc);
+
+ /*
+ * Partial aggregation is what the grouped paths should do.
+ */
+ result->agg_exprs = lappend(result->agg_exprs, aggref);
+ lc = lnext(lc);
+ }
+
+ /* The "input_rows" field should be set by caller. */
+ return result;
+}
+
+/*
+ * Initialize target for grouped paths (target) as well as a target for paths
+ * that generate input for aggregation (agg_input).
+ *
+ * group_exprs_extra_p receives a list of Var nodes for which we need to
+ * construct SortGroupClause. Those vars will then be used as additional
+ * grouping expressions, for the sake of join clauses.
+ *
+ * gvis a list of GroupedVarInfo's possibly useful for rel.
+ *
+ * Return true iff the targets could be initialized.
+ */
+static bool
+init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p)
+{
+ ListCell *lc1,
+ *lc2;
+ List *unresolved = NIL;
+ List *unresolved_sortgrouprefs = NIL;
+
+ foreach(lc1, rel->reltarget->exprs)
+ {
+ Var *tvar;
+ bool is_grouping;
+ Index sortgroupref = 0;
+ bool derived = false;
+ bool needed_by_aggregate;
+
+ /*
+ * Given that PlaceHolderVar currently prevents us from doing
+ * aggregation push-down, the source target cannot contain anything
+ * more complex than a Var.
+ */
+ tvar = lfirst_node(Var, lc1);
+
+ is_grouping = is_grouping_expression(gvis, (Expr *) tvar,
+ &sortgroupref, &derived);
+
+ /*
+ * Derived grouping expressions should not be referenced by the query
+ * targetlist, so let them fall into vars_unresolved. It'll be checked
+ * later if the current targetlist needs them. For example, we should
+ * not automatically use Var as a grouping expression if the only
+ * reason for it to be in the plain relation target is that it's
+ * referenced by aggregate argument, and it happens to be in the same
+ * EC as any grouping expression.
+ */
+ if (is_grouping && !derived)
+ {
+ Assert(sortgroupref > 0);
+
+ /*
+ * It's o.k. to use the target expression for grouping.
+ */
+ add_column_to_pathtarget(target, (Expr *) tvar, sortgroupref);
+
+ /*
+ * As for agg_input, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) tvar, sortgroupref);
+
+ /* Process the next expression. */
+ continue;
+ }
+
+ /*
+ * Is this Var needed in the query targetlist for anything else than
+ * aggregate input?
+ */
+ needed_by_aggregate = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc2);
+ ListCell *lc3;
+ List *vars;
+
+ if (!IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (!bms_is_member(tvar->varno, gvi->gv_eval_at))
+ continue;
+
+ /*
+ * XXX Consider some sort of caching.
+ */
+ vars = pull_var_clause((Node *) gvi->gvexpr, PVC_RECURSE_AGGREGATES);
+ foreach(lc3, vars)
+ {
+ Var *var = lfirst_node(Var, lc3);
+
+ if (equal(var, tvar))
+ {
+ needed_by_aggregate = true;
+ break;
+ }
+ }
+ list_free(vars);
+ if (needed_by_aggregate)
+ break;
+ }
+
+ if (needed_by_aggregate)
+ {
+ bool found = false;
+
+ foreach(lc2, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc2);
+
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ if (equal(te->expr, tvar))
+ {
+ found = true;
+ break;
+ }
+ }
+
+ /*
+ * If it's only Aggref input, add it to the aggregation input
+ * target and that's it.
+ */
+ if (!found)
+ {
+ add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
+ continue;
+ }
+ }
+
+ /*
+ * Further investigation involves dependency check, for which we need
+ * to have all the (plain-var) grouping expressions gathered.
+ */
+ unresolved = lappend(unresolved, tvar);
+ unresolved_sortgrouprefs = lappend_int(unresolved_sortgrouprefs,
+ sortgroupref);
+ }
+
+ /*
+ * Check for other possible reasons for the var to be in the plain target.
+ */
+ forboth(lc1, unresolved, lc2, unresolved_sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, lc1);
+ Index sortgroupref = lfirst_int(lc2);
+ RangeTblEntry *rte;
+ List *deps = NIL;
+ Relids relids_subtract;
+ int ndx;
+ RelOptInfo *baserel;
+
+ rte = root->simple_rte_array[var->varno];
+
+ /*
+ * Check if the Var can be in the grouping key even though it's not
+ * mentioned by the GROUP BY clause (and could not be derived using
+ * ECs).
+ */
+ if (sortgroupref == 0 &&
+ check_functional_grouping(rte->relid, var->varno,
+ var->varlevelsup,
+ target->exprs, &deps))
+ {
+ /*
+ * The var shouldn't be actually used as a grouping key (instead,
+ * the one this depends on will be), so sortgroupref should not be
+ * important.
+ */
+ add_new_column_to_pathtarget(target, (Expr *) var);
+ add_new_column_to_pathtarget(agg_input, (Expr *) var);
+
+ /*
+ * The var may or may not be present in generic grouping
+ * expression(s) in addition, but this is handled elsewhere.
+ */
+ continue;
+ }
+
+ /*
+ * Isn't the expression needed by joins above the current rel?
+ *
+ * The relids we're not interested in do include 0, which is the
+ * top-level targetlist. The only reason for relids to contain 0
+ * should be that arg_var is referenced either by aggregate or by
+ * grouping expression, but right now we're interested in the *other*
+ * reasons. (As soon aggregation is pushed down, the aggregates in the
+ * query targetlist no longer need direct reference to arg_var
+ * anyway.)
+ */
+ relids_subtract = bms_copy(rel->relids);
+ bms_add_member(relids_subtract, 0);
+
+ baserel = find_base_rel(root, var->varno);
+ ndx = var->varattno - baserel->min_attr;
+ if (bms_nonempty_difference(baserel->attr_needed[ndx],
+ relids_subtract))
+ {
+ /*
+ * The variable is needed by a join involving this relation. That
+ * case includes variable that is referenced by a generic grouping
+ * expression.
+ *
+ * The only way to bring this var to the aggregation output is to
+ * add it to the grouping expressions too.
+ */
+ if (sortgroupref > 0)
+ {
+ /*
+ * The var could be recognized as a potentially useful
+ * grouping expression at the top of the loop, so we can add
+ * it to the grouping target, as well as to the agg_input.
+ */
+ add_column_to_pathtarget(target, (Expr *) var, sortgroupref);
+ add_column_to_pathtarget(agg_input, (Expr *) var, sortgroupref);
+ }
+ else
+ {
+ /*
+ * Since root->parse->groupClause is not supposed to contain
+ * this expression, we need to construct special
+ * SortGroupClause. Its tleSortGroupRef needs to be unique
+ * within target_agg, so postpone creation of the
+ * SortGroupRefs until we're done with the iteration of
+ * rel->reltarget->exprs.
+ */
+ *group_exprs_extra_p = lappend(*group_exprs_extra_p, var);
+ }
+ }
+ else
+ {
+ /*
+ * As long as the query is semantically correct, arriving here
+ * means that the var is referenced by a generic grouping
+ * expression but not referenced by any join.
+ *
+ * create_rel_agg_info() should add this variable to "agg_input"
+ * target and also add the whole generic expression to "target",
+ * but that's subject to future enhancement of the aggregate
+ * push-down feature.
+ */
+ return false;
+ }
+ }
+
+ return true;
+}
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
index 41402821a1..c7105dfc2a 100644
--- a/src/backend/optimizer/util/tlist.c
+++ b/src/backend/optimizer/util/tlist.c
@@ -826,6 +826,62 @@ apply_pathtarget_labeling_to_tlist(List *tlist, PathTarget *target)
}
}
+/*
+ * For each aggregate grouping expression or aggregate to the grouped target.
+ *
+ * Caller passes the expressions in the form of GroupedVarInfos so that we
+ * don't have to look for gvid.
+ */
+void
+add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions)
+{
+ ListCell *lc;
+
+ /* Create the vars and add them to the target. */
+ foreach(lc, expressions)
+ {
+ GroupedVarInfo *gvi;
+
+ gvi = lfirst_node(GroupedVarInfo, lc);
+ add_column_to_pathtarget(target, (Expr *) gvi->agg_partial,
+ gvi->sortgroupref);
+ }
+}
+
+/*
+ * Find out if expr can be used as grouping expression in reltarget.
+ *
+ * sortgroupref and is_derived reflect the ->sortgroupref and ->derived fields
+ * of the corresponding GroupedVarInfo.
+ */
+bool
+is_grouping_expression(List *gvis, Expr *expr, Index *sortgroupref,
+ bool *is_derived)
+{
+ ListCell *lc;
+
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (equal(gvi->gvexpr, expr))
+ {
+ Assert(gvi->sortgroupref > 0);
+
+ *sortgroupref = gvi->sortgroupref;
+ *is_derived = gvi->derived;
+ return true;
+ }
+ }
+
+ /* The expression cannot be used as grouping key. */
+ return false;
+}
+
/*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index fc463601ff..8c1929c758 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -1021,6 +1021,15 @@ static struct config_bool ConfigureNamesBool[] =
false,
NULL, NULL, NULL
},
+ {
+ {"enable_agg_pushdown", PGC_USERSET, QUERY_TUNING_METHOD,
+ gettext_noop("Enables aggregation push-down."),
+ NULL
+ },
+ &enable_agg_pushdown,
+ false,
+ NULL, NULL, NULL
+ },
{
{"enable_parallel_append", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables the planner's use of parallel append plans."),
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 11027cdb10..25d058ef2a 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -224,6 +224,7 @@ typedef enum NodeTag
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_RelAggInfo,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
@@ -268,6 +269,7 @@ typedef enum NodeTag
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 38dc186623..988d6220c8 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -240,6 +240,23 @@ struct PlannerInfo
*/
struct RelInfoList *join_rel_list; /* list of join-relation RelOptInfos */
+ /*
+ * grouped_rel_list is a list of RelOptInfos that represent grouped
+ * relations, both base relations and joins. Unlike join_rel_list, base
+ * relations are accepted because grouped base relation is of rather
+ * limited scope, i.e. it's only needed during join search. Thus it does
+ * not deserve separate storage like simple_rel_array.
+ */
+ struct RelInfoList *grouped_rel_list; /* list of grouped relation
+ * RelOptInfos */
+
+ /*
+ * agg_info_list contains one instance of RelAggInfo per an item of
+ * grouped_rel_list.
+ */
+ struct RelInfoList *agg_info_list; /* list of grouped relation
+ * RelAggInfos */
+
/*
* When doing a dynamic-programming-style join search, join_rel_level[k]
* is a list of all join-relation RelOptInfos of level k, and
@@ -285,6 +302,8 @@ struct PlannerInfo
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
@@ -316,6 +335,12 @@ struct PlannerInfo
*/
List *processed_tlist;
+ /*
+ * The maximum ressortgroupref among target entries in processed_list.
+ * Useful when adding extra grouping expressions for partial aggregation.
+ */
+ int max_sortgroupref;
+
/* Fields filled during create_plan() for use in setrefs.c */
AttrNumber *grouping_map; /* for GroupingFunc fixup */
List *minmax_aggs; /* List of MinMaxAggInfos */
@@ -754,6 +779,60 @@ typedef struct RelInfoList
struct HTAB *hash;
} RelInfoList;
+/*
+ * RelAggInfo
+ * Information needed to create grouped paths for base rels and joins.
+ *
+ * "relids" is the set of base-relation identifiers, just like with
+ * RelOptInfo.
+ *
+ * "target" will be used as pathtarget if partial aggregation is applied to
+ * base relation or join. The same target will also --- if the relation is a
+ * join --- be used to joinin grouped path to a non-grouped one. This target
+ * can contain plain-Var grouping expressions and Aggref nodes.
+ *
+ * Note: There's a convention that Aggref expressions are supposed to follow
+ * the other expressions of the target. Iterations of ->exprs may rely on this
+ * arrangement.
+ *
+ * "agg_input" contains Vars used either as grouping expressions or aggregate
+ * arguments. Paths providing the aggregation plan with input data should use
+ * this target. The only difference from reltarget of the non-grouped relation
+ * is that some items can have sortgroupref initialized.
+ *
+ * "input_rows" is the estimated number of input rows for AggPath. It's
+ * actually just a workspace for users of the structure, i.e. not initialized
+ * when instance of the structure is created.
+ *
+ * "group_clauses" and "group_exprs" are lists of SortGroupClause and the
+ * corresponding grouping expressions respectively.
+ *
+ * "agg_exprs" is a list of Aggref nodes for the aggregation of the relation's
+ * paths.
+ *
+ * "rel_grouped" is the relation containing the partially aggregated paths.
+ */
+typedef struct RelAggInfo
+{
+ NodeTag type;
+
+ Relids relids; /* Base rels contained in this grouped rel. */
+
+ struct PathTarget *target; /* Target for grouped paths. */
+
+ struct PathTarget *agg_input; /* pathtarget of paths that generate input
+ * for aggregation paths. */
+
+ double input_rows;
+
+ List *group_clauses;
+ List *group_exprs;
+
+ List *agg_exprs; /* Aggref expressions. */
+
+ RelOptInfo *rel_grouped; /* Grouped relation. */
+} RelAggInfo;
+
/*
* IndexOptInfo
* Per-index information for planning/optimization
@@ -2272,6 +2351,26 @@ typedef struct PlaceHolderInfo
int32 ph_width; /* estimated attribute width */
} PlaceHolderInfo;
+/*
+ * GroupedVarInfo exists for each expression that can be used as an aggregate
+ * or grouping expression evaluated below a join.
+ */
+typedef struct GroupedVarInfo
+{
+ NodeTag type;
+
+ Expr *gvexpr; /* the represented expression. */
+ Aggref *agg_partial; /* if gvexpr is aggregate, agg_partial is the
+ * corresponding partial aggregate */
+ Index sortgroupref; /* If gvexpr is a grouping expression, this is
+ * the tleSortGroupRef of the corresponding
+ * SortGroupClause. */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+ bool derived; /* derived from another GroupedVarInfo using
+ * equeivalence classes? */
+} GroupedVarInfo;
+
/*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
index 2f9aeec4a7..90f6995dc2 100644
--- a/src/include/optimizer/clauses.h
+++ b/src/include/optimizer/clauses.h
@@ -55,4 +55,6 @@ extern void CommuteOpExpr(OpExpr *clause);
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
+extern GroupedVarInfo *translate_expression_to_rels(PlannerInfo *root,
+ GroupedVarInfo *gvi, Index relid);
#endif /* CLAUSES_H */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 182ffeef4b..2d05ff5a15 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -210,6 +210,14 @@ extern AggPath *create_agg_path(PlannerInfo *root,
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+extern AggPath *create_agg_sorted_path(PlannerInfo *root,
+ RelOptInfo *rel,
+ Path *subpath,
+ RelAggInfo *agg_info);
+extern AggPath *create_agg_hashed_path(PlannerInfo *root,
+ RelOptInfo *rel,
+ Path *subpath,
+ RelAggInfo *agg_info);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
@@ -281,14 +289,21 @@ extern void setup_append_rel_array(PlannerInfo *root);
extern void expand_planner_arrays(PlannerInfo *root, int add_size);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
+extern RelOptInfo *build_simple_grouped_rel(PlannerInfo *root, int relid,
+ RelAggInfo **agg_info_p);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
+extern void add_grouped_rel(PlannerInfo *root, RelOptInfo *rel,
+ RelAggInfo *agg_info);
+extern RelOptInfo *find_grouped_rel(PlannerInfo *root, Relids relids,
+ RelAggInfo **agg_info_p);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr);
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info);
extern Relids min_join_parameterization(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
@@ -314,5 +329,5 @@ extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo, JoinType jointype);
-
+extern RelAggInfo *create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index 7345137d1d..3006ebf5aa 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -21,6 +21,7 @@
* allpaths.c
*/
extern PGDLLIMPORT bool enable_geqo;
+extern PGDLLIMPORT bool enable_agg_pushdown;
extern PGDLLIMPORT int geqo_threshold;
extern PGDLLIMPORT int min_parallel_table_scan_size;
extern PGDLLIMPORT int min_parallel_index_scan_size;
@@ -52,8 +53,14 @@ extern RelOptInfo *make_one_rel(PlannerInfo *root, List *joinlist);
extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
List *initial_rels);
+
extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
+extern void generate_grouping_paths(PlannerInfo *root,
+ RelOptInfo *rel_grouped,
+ RelOptInfo *rel_plain,
+ RelAggInfo *agg_info);
+
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages, int max_workers);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index e7aaddd50d..761f610a96 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -28,7 +28,8 @@ typedef void (*query_pathkeys_callback) (PlannerInfo *root, void *extra);
* prototypes for plan/planmain.c
*/
extern RelOptInfo *query_planner(PlannerInfo *root,
- query_pathkeys_callback qp_callback, void *qp_extra);
+ query_pathkeys_callback qp_callback, void *qp_extra,
+ RelOptInfo **final_rel_grouped_p);
/*
* prototypes for plan/planagg.c
@@ -69,6 +70,7 @@ extern void add_other_rels_to_query(PlannerInfo *root);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
Relids where_needed, bool create_new_ph);
+extern void setup_aggregate_pushdown(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
index 8a2378f968..e48f37bac2 100644
--- a/src/include/optimizer/tlist.h
+++ b/src/include/optimizer/tlist.h
@@ -50,8 +50,14 @@ extern void split_pathtarget_at_srfs(PlannerInfo *root,
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+/* TODO Find the best location (position and in some cases even file) for the
+ * following ones. */
+extern void add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions);
+extern bool is_grouping_expression(List *gvis, Expr *expr,
+ Index *sortgroupref, bool *is_derived);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
-
#endif /* TLIST_H */
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
new file mode 100644
index 0000000000..6cf2ed9c20
--- /dev/null
+++ b/src/test/regress/sql/agg_pushdown.sql
@@ -0,0 +1,117 @@
+BEGIN;
+
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+
+CREATE INDEX ON agg_pushdown_child1(parent);
+
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+COMMIT;
+ANALYZE;
+
+SET enable_agg_pushdown TO on;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_seqscan TO on;
+
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
new file mode 100644
index 0000000000..b3a97f86d6
--- /dev/null
+++ b/src/test/regress/expected/agg_pushdown.out
@@ -0,0 +1,217 @@
+BEGIN;
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+CREATE INDEX ON agg_pushdown_child1(parent);
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+COMMIT;
+ANALYZE;
+SET enable_agg_pushdown TO on;
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+--------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(10 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(11 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (p.i = c1.parent)
+ -> Sort
+ Sort Key: p.i
+ -> Seq Scan on agg_pushdown_parent p
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(12 rows)
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(8 rows)
+
+SET enable_seqscan TO on;
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.x = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(11 rows)
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: ((k = c1.j) AND (parent = c1.parent))
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(13 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+----------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Hash
+ -> Seq Scan on agg_pushdown_child2 c2
+(15 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: ((c1.j = c2.k) AND (c1.parent = c2.parent))
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(13 rows)
+
diff --git a/src/test/regress/expected/sysviews.out b/src/test/regress/expected/sysviews.out
index a1c90eb905..55cfd3fff7 100644
--- a/src/test/regress/expected/sysviews.out
+++ b/src/test/regress/expected/sysviews.out
@@ -72,6 +72,7 @@ select count(*) >= 0 as ok from pg_prepared_xacts;
select name, setting from pg_settings where name like 'enable%';
name | setting
--------------------------------+---------
+ enable_agg_pushdown | off
enable_bitmapscan | on
enable_gathermerge | on
enable_hashagg | on
@@ -89,7 +90,7 @@ select name, setting from pg_settings where name like 'enable%';
enable_seqscan | on
enable_sort | on
enable_tidscan | on
-(17 rows)
+(18 rows)
-- Test that the pg_timezone_names and pg_timezone_abbrevs views are
-- more-or-less working. We can't test their contents in any great detail
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index a39ca1012a..4f0ba643db 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -140,6 +140,7 @@ test: amutils
test: stats_ext
test: select_parallel
test: write_parallel
+test: agg_pushdown
test: publication
test: subscription
test: select_views
--
2.16.4
Re: [HACKERS] WIP: Aggregation push-down
On Fri, Jul 12, 2019 at 4:42 PM Antonin Houska <ah@cybertec.at> wrote:
Richard Guo <riguo@pivotal.io> wrote:
I didn't fully follow the whole thread and mainly looked into the
latest
patch set. So what are the considerations for abandoning the
aggmultifn
concept?Originally the function was there to support join where both relations are
grouped. My doubts about usefulness of such join started at [1]. (See the
thread referenced from [2].)In my opinion, aggmultifn would enable us to do a lot more
types of transformation. For example, consider the query below:select sum(foo.c) from foo join bar on foo.b = bar.b group by foo.a,
bar.a;With the latest patch, the plan looks like:
Finalize HashAggregate <------ sum(psum)
Group Key: foo.a, bar.a
-> Hash Join
Hash Cond: (bar.b = foo.b)
-> Seq Scan on bar
-> Hash
-> Partial HashAggregate <------ sum(foo.c) as
psum
Group Key: foo.a, foo.b
-> Seq Scan on fooIf we have aggmultifn, we can perform the query this way:
Finalize HashAggregate <------ sum(foo.c)*cnt
Group Key: foo.a, bar.a
-> Hash Join
Hash Cond: (foo.b = bar.b)
-> Seq Scan on foo
-> Hash
-> Partial HashAggregate <------ count(*) as cnt
Group Key: bar.a, bar.b
-> Seq Scan on barPerhaps, but it that would require changes to nodeAgg.c, which I want to
avoid
for now.And this way:
Finalize HashAggregate <------ sum(psum)*cnt
Group Key: foo.a, bar.a
-> Hash Join
Hash Cond: (foo.b = bar.b)
-> Partial HashAggregate <------ sum(foo.c) as
psum
Group Key: foo.a, foo.b
-> Seq Scan on foo
-> Hash
-> Partial HashAggregate <------ count(*) as cnt
Group Key: bar.a, bar.b
-> Seq Scan on barThis looks like my idea presented in [1], for which there seems to be no
common use case.My another question is in function add_grouped_path(), when creating
sorted aggregation path on top of subpath. If the subpath is not
sorted,
then the sorted aggregation path would not be generated. Why not in
this
case we create a sort path on top of subpath first and then create
group
aggregation path on top of the sort path?I see no reason not to do it. (If you want to try, just go ahead.) However
the
current patch version brings only the basic functionality and I'm not
going to
add new functionality (such as parallel aggregation, partitioned tables or
postgres_fdw) until the open design questions are resolved. Otherwise
there's
a risk that the additional work will be wasted due to major rework of the
core
functionality.Core dump when running one query in agg_pushdown.sql
Thanks for the report! Fixed in the new version.
Another core dump for query below:
select sum(t1.s1) from t1, t2, t3, t4 where t1.j1 = t2.j2 group by t1.g1,
t2.g2;
This is due to a small mistake:
diff --git a/src/backend/optimizer/util/relnode.c
b/src/backend/optimizer/util/relnode.c
index 10becc0..9c2deed 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -648,7 +648,7 @@ void
add_grouped_rel(PlannerInfo *root, RelOptInfo *rel, RelAggInfo *agg_info)
{
add_rel_info(root->grouped_rel_list, rel);
- add_rel_info(root->agg_info_list, rel);
+ add_rel_info(root->agg_info_list, agg_info);
}Thanks
Richard
Re: [HACKERS] WIP: Aggregation push-down
Richard Guo <riguo@pivotal.io> wrote:
Another core dump for query below:
select sum(t1.s1) from t1, t2, t3, t4 where t1.j1 = t2.j2 group by t1.g1, t2.g2;
This is due to a small mistake:
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c index 10becc0..9c2deed 100644 --- a/src/backend/optimizer/util/relnode.c +++ b/src/backend/optimizer/util/relnode.c @@ -648,7 +648,7 @@ void add_grouped_rel(PlannerInfo *root, RelOptInfo *rel, RelAggInfo *agg_info) { add_rel_info(root->grouped_rel_list, rel); - add_rel_info(root->agg_info_list, rel); + add_rel_info(root->agg_info_list, agg_info); }
Thanks! Yes, this is what I had to fix.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
Attachments:
v14-0001-Introduce-RelInfoList-structure.patchtext/x-diffDownload
From b2672f2b06f521fdc11f7e7293ab669ca582061b Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Wed, 17 Jul 2019 16:31:32 +0200
Subject: [PATCH 1/3] Introduce RelInfoList structure.
---
contrib/postgres_fdw/postgres_fdw.c | 3 +-
src/backend/nodes/outfuncs.c | 11 +++
src/backend/optimizer/geqo/geqo_eval.c | 12 +--
src/backend/optimizer/plan/planmain.c | 3 +-
src/backend/optimizer/util/relnode.c | 157 ++++++++++++++++++++-------------
src/include/nodes/nodes.h | 1 +
src/include/nodes/pathnodes.h | 28 ++++--
7 files changed, 136 insertions(+), 79 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 033aeb2556..90414f1168 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -5205,7 +5205,8 @@ foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
*/
Assert(fpinfo->relation_index == 0); /* shouldn't be set yet */
fpinfo->relation_index =
- list_length(root->parse->rtable) + list_length(root->join_rel_list);
+ list_length(root->parse->rtable) +
+ list_length(root->join_rel_list->items);
return true;
}
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8e31fae47f..01745ff879 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2279,6 +2279,14 @@ _outRelOptInfo(StringInfo str, const RelOptInfo *node)
}
static void
+_outRelInfoList(StringInfo str, const RelInfoList *node)
+{
+ WRITE_NODE_TYPE("RELOPTINFOLIST");
+
+ WRITE_NODE_FIELD(items);
+}
+
+static void
_outIndexOptInfo(StringInfo str, const IndexOptInfo *node)
{
WRITE_NODE_TYPE("INDEXOPTINFO");
@@ -4052,6 +4060,9 @@ outNode(StringInfo str, const void *obj)
case T_RelOptInfo:
_outRelOptInfo(str, obj);
break;
+ case T_RelInfoList:
+ _outRelInfoList(str, obj);
+ break;
case T_IndexOptInfo:
_outIndexOptInfo(str, obj);
break;
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index 7b67a29c88..5fca9814a2 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -92,11 +92,11 @@ geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
*
* join_rel_level[] shouldn't be in use, so just Assert it isn't.
*/
- savelength = list_length(root->join_rel_list);
- savehash = root->join_rel_hash;
+ savelength = list_length(root->join_rel_list->items);
+ savehash = root->join_rel_list->hash;
Assert(root->join_rel_level == NULL);
- root->join_rel_hash = NULL;
+ root->join_rel_list->hash = NULL;
/* construct the best path for the given combination of relations */
joinrel = gimme_tree(root, tour, num_gene);
@@ -121,9 +121,9 @@ geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
* Restore join_rel_list to its former state, and put back original
* hashtable if any.
*/
- root->join_rel_list = list_truncate(root->join_rel_list,
- savelength);
- root->join_rel_hash = savehash;
+ root->join_rel_list->items = list_truncate(root->join_rel_list->items,
+ savelength);
+ root->join_rel_list->hash = savehash;
/* release all the memory acquired within gimme_tree */
MemoryContextSwitchTo(oldcxt);
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 2dbf1db844..0b9999c8a6 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -65,8 +65,7 @@ query_planner(PlannerInfo *root,
* NOTE: append_rel_list was set up by subquery_planner, so do not touch
* here.
*/
- root->join_rel_list = NIL;
- root->join_rel_hash = NULL;
+ root->join_rel_list = makeNode(RelInfoList);
root->join_rel_level = NULL;
root->join_cur_level = 0;
root->canon_pathkeys = NIL;
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 6054bd2b53..c238dd6538 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -31,11 +31,11 @@
#include "utils/hsearch.h"
-typedef struct JoinHashEntry
+typedef struct RelInfoEntry
{
- Relids join_relids; /* hash key --- MUST BE FIRST */
- RelOptInfo *join_rel;
-} JoinHashEntry;
+ Relids relids; /* hash key --- MUST BE FIRST */
+ void *data;
+} RelInfoEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *input_rel);
@@ -375,11 +375,11 @@ find_base_rel(PlannerInfo *root, int relid)
}
/*
- * build_join_rel_hash
- * Construct the auxiliary hash table for join relations.
+ * build_rel_hash
+ * Construct the auxiliary hash table for relation specific data.
*/
static void
-build_join_rel_hash(PlannerInfo *root)
+build_rel_hash(RelInfoList *list)
{
HTAB *hashtab;
HASHCTL hash_ctl;
@@ -388,47 +388,50 @@ build_join_rel_hash(PlannerInfo *root)
/* Create the hash table */
MemSet(&hash_ctl, 0, sizeof(hash_ctl));
hash_ctl.keysize = sizeof(Relids);
- hash_ctl.entrysize = sizeof(JoinHashEntry);
+ hash_ctl.entrysize = sizeof(RelInfoEntry);
hash_ctl.hash = bitmap_hash;
hash_ctl.match = bitmap_match;
hash_ctl.hcxt = CurrentMemoryContext;
- hashtab = hash_create("JoinRelHashTable",
+ hashtab = hash_create("RelHashTable",
256L,
&hash_ctl,
HASH_ELEM | HASH_FUNCTION | HASH_COMPARE | HASH_CONTEXT);
/* Insert all the already-existing joinrels */
- foreach(l, root->join_rel_list)
+ foreach(l, list->items)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
- JoinHashEntry *hentry;
+ void *item = lfirst(l);
+ RelInfoEntry *hentry;
bool found;
+ Relids relids;
- hentry = (JoinHashEntry *) hash_search(hashtab,
- &(rel->relids),
- HASH_ENTER,
- &found);
+ Assert(IsA(item, RelOptInfo));
+ relids = ((RelOptInfo *) item)->relids;
+
+ hentry = (RelInfoEntry *) hash_search(hashtab,
+ &relids,
+ HASH_ENTER,
+ &found);
Assert(!found);
- hentry->join_rel = rel;
+ hentry->data = item;
}
- root->join_rel_hash = hashtab;
+ list->hash = hashtab;
}
/*
- * find_join_rel
- * Returns relation entry corresponding to 'relids' (a set of RT indexes),
- * or NULL if none exists. This is for join relations.
+ * find_rel_info
+ * Find a base or join relation entry.
*/
-RelOptInfo *
-find_join_rel(PlannerInfo *root, Relids relids)
+static void *
+find_rel_info(RelInfoList *list, Relids relids)
{
/*
* Switch to using hash lookup when list grows "too long". The threshold
* is arbitrary and is known only here.
*/
- if (!root->join_rel_hash && list_length(root->join_rel_list) > 32)
- build_join_rel_hash(root);
+ if (!list->hash && list_length(list->items) > 32)
+ build_rel_hash(list);
/*
* Use either hashtable lookup or linear search, as appropriate.
@@ -438,28 +441,32 @@ find_join_rel(PlannerInfo *root, Relids relids)
* so would force relids out of a register and thus probably slow down the
* list-search case.
*/
- if (root->join_rel_hash)
+ if (list->hash)
{
Relids hashkey = relids;
- JoinHashEntry *hentry;
+ RelInfoEntry *hentry;
- hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
- &hashkey,
- HASH_FIND,
- NULL);
+ hentry = (RelInfoEntry *) hash_search(list->hash,
+ &hashkey,
+ HASH_FIND,
+ NULL);
if (hentry)
- return hentry->join_rel;
+ return hentry->data;
}
else
{
ListCell *l;
- foreach(l, root->join_rel_list)
+ foreach(l, list->items)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
+ void *item = lfirst(l);
+ Relids item_relids;
- if (bms_equal(rel->relids, relids))
- return rel;
+ Assert(IsA(item, RelOptInfo));
+ item_relids = ((RelOptInfo *) item)->relids;
+
+ if (bms_equal(item_relids, relids))
+ return item;
}
}
@@ -467,6 +474,58 @@ find_join_rel(PlannerInfo *root, Relids relids)
}
/*
+ * find_join_rel
+ * Returns relation entry corresponding to 'relids' (a set of RT indexes),
+ * or NULL if none exists. This is for join relations.
+ */
+RelOptInfo *
+find_join_rel(PlannerInfo *root, Relids relids)
+{
+ return (RelOptInfo *) find_rel_info(root->join_rel_list, relids);
+}
+
+/*
+ * add_rel_info
+ * Add relation specific info to a list, and also add it to the auxiliary
+ * hashtable if there is one.
+ */
+static void
+add_rel_info(RelInfoList *list, void *data)
+{
+ Assert(IsA(data, RelOptInfo));
+
+ /* GEQO requires us to append the new joinrel to the end of the list! */
+ list->items = lappend(list->items, data);
+
+ /* store it into the auxiliary hashtable if there is one. */
+ if (list->hash)
+ {
+ Relids relids;
+ RelInfoEntry *hentry;
+ bool found;
+
+ relids = ((RelOptInfo *) data)->relids;
+ hentry = (RelInfoEntry *) hash_search(list->hash,
+ &relids,
+ HASH_ENTER,
+ &found);
+ Assert(!found);
+ hentry->data = data;
+ }
+}
+
+/*
+ * add_join_rel
+ * Add given join relation to the list of join relations in the given
+ * PlannerInfo.
+ */
+static void
+add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
+{
+ add_rel_info(root->join_rel_list, joinrel);
+}
+
+/*
* set_foreign_rel_properties
* Set up foreign-join fields if outer and inner relation are foreign
* tables (or joins) belonging to the same server and assigned to the same
@@ -517,32 +576,6 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
}
/*
- * add_join_rel
- * Add given join relation to the list of join relations in the given
- * PlannerInfo. Also add it to the auxiliary hashtable if there is one.
- */
-static void
-add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
-{
- /* GEQO requires us to append the new joinrel to the end of the list! */
- root->join_rel_list = lappend(root->join_rel_list, joinrel);
-
- /* store it into the auxiliary hashtable if there is one. */
- if (root->join_rel_hash)
- {
- JoinHashEntry *hentry;
- bool found;
-
- hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
- &(joinrel->relids),
- HASH_ENTER,
- &found);
- Assert(!found);
- hentry->join_rel = joinrel;
- }
-}
-
-/*
* build_join_rel
* Returns relation entry corresponding to the union of two given rels,
* creating a new relation entry if none already exists.
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 4e2fb39105..11027cdb10 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -220,6 +220,7 @@ typedef enum NodeTag
T_PlannerInfo,
T_PlannerGlobal,
T_RelOptInfo,
+ T_RelInfoList,
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 441e64eca9..38dc186623 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -236,15 +236,9 @@ struct PlannerInfo
/*
* join_rel_list is a list of all join-relation RelOptInfos we have
- * considered in this planning run. For small problems we just scan the
- * list to do lookups, but when there are many join relations we build a
- * hash table for faster lookups. The hash table is present and valid
- * when join_rel_hash is not NULL. Note that we still maintain the list
- * even when using the hash table for lookups; this simplifies life for
- * GEQO.
+ * considered in this planning run.
*/
- List *join_rel_list; /* list of join-relation RelOptInfos */
- struct HTAB *join_rel_hash; /* optional hashtable for join relations */
+ struct RelInfoList *join_rel_list; /* list of join-relation RelOptInfos */
/*
* When doing a dynamic-programming-style join search, join_rel_level[k]
@@ -743,6 +737,24 @@ typedef struct RelOptInfo
(rel)->part_rels && (rel)->partexprs && (rel)->nullable_partexprs)
/*
+ * RelInfoList
+ * A list to store relation specific info and to retrieve it by relids.
+ *
+ * For small problems we just scan the list to do lookups, but when there are
+ * many relations we build a hash table for faster lookups. The hash table is
+ * present and valid when rel_hash is not NULL. Note that we still maintain
+ * the list even when using the hash table for lookups; this simplifies life
+ * for GEQO.
+ */
+typedef struct RelInfoList
+{
+ NodeTag type;
+
+ List *items;
+ struct HTAB *hash;
+} RelInfoList;
+
+/*
* IndexOptInfo
* Per-index information for planning/optimization
*
--
2.13.7
v14-0002-Introduce-make_join_rel_common-function.patchtext/x-diffDownload
From f1d25f8389b545a9cc62ced05bdc53f0b7d21238 Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Wed, 17 Jul 2019 16:31:32 +0200
Subject: [PATCH 2/3] Introduce make_join_rel_common function.
This diff tries to make the following patch a bit easier to read.
---
src/backend/optimizer/path/joinrels.c | 39 ++++++++++++++++++++++-------------
1 file changed, 25 insertions(+), 14 deletions(-)
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 6a480ab764..f680658cf7 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -36,6 +36,8 @@ static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
+static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1,
+ RelOptInfo *rel2);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist);
@@ -665,21 +667,12 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
return true;
}
-
/*
- * make_join_rel
- * Find or create a join RelOptInfo that represents the join of
- * the two given rels, and add to it path information for paths
- * created with the two rels as outer and inner rel.
- * (The join rel may already contain paths generated from other
- * pairs of rels that add up to the same set of base rels.)
- *
- * NB: will return NULL if attempted join is not valid. This can happen
- * when working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
- */
-RelOptInfo *
-make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+ * make_join_rel_common
+ * The workhorse of make_join_rel().
+ */
+static RelOptInfo *
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -762,6 +755,24 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
}
/*
+ * make_join_rel
+ * Find or create a join RelOptInfo that represents the join of
+ * the two given rels, and add to it path information for paths
+ * created with the two rels as outer and inner rel.
+ * (The join rel may already contain paths generated from other
+ * pairs of rels that add up to the same set of base rels.)
+ *
+ * NB: will return NULL if attempted join is not valid. This can happen when
+ * working with outer joins, or with IN or EXISTS clauses that have been
+ * turned into joins.
+ */
+RelOptInfo *
+make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+{
+ return make_join_rel_common(root, rel1, rel2);
+}
+
+/*
* populate_joinrel_with_paths
* Add paths to the given joinrel for given pair of joining relations. The
* SpecialJoinInfo provides details about the join and the restrictlist
--
2.13.7
v14-0003-Aggregate-push-down-basic-functionality.patchtext/x-diffDownload
From dc4398c257c6c1a6fd0afead3ea401a985cd6622 Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Wed, 17 Jul 2019 16:31:32 +0200
Subject: [PATCH 3/3] Aggregate push-down - basic functionality.
---
src/backend/nodes/copyfuncs.c | 21 +-
src/backend/nodes/outfuncs.c | 36 ++
src/backend/optimizer/README | 87 +++
src/backend/optimizer/path/allpaths.c | 112 ++++
src/backend/optimizer/path/costsize.c | 17 +-
src/backend/optimizer/path/equivclass.c | 134 +++++
src/backend/optimizer/path/joinrels.c | 170 +++++-
src/backend/optimizer/plan/initsplan.c | 257 ++++++++
src/backend/optimizer/plan/planagg.c | 2 +-
src/backend/optimizer/plan/planmain.c | 23 +-
src/backend/optimizer/plan/planner.c | 58 +-
src/backend/optimizer/plan/setrefs.c | 33 ++
src/backend/optimizer/prep/prepjointree.c | 1 +
src/backend/optimizer/util/pathnode.c | 143 ++++-
src/backend/optimizer/util/relnode.c | 916 +++++++++++++++++++++++++++--
src/backend/optimizer/util/tlist.c | 56 ++
src/backend/utils/misc/guc.c | 9 +
src/include/nodes/nodes.h | 2 +
src/include/nodes/pathnodes.h | 99 ++++
src/include/optimizer/clauses.h | 2 +
src/include/optimizer/pathnode.h | 19 +-
src/include/optimizer/paths.h | 7 +
src/include/optimizer/planmain.h | 4 +-
src/include/optimizer/tlist.h | 8 +-
src/test/regress/expected/agg_pushdown.out | 217 +++++++
src/test/regress/expected/sysviews.out | 3 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/agg_pushdown.sql | 117 ++++
28 files changed, 2480 insertions(+), 74 deletions(-)
create mode 100644 src/test/regress/expected/agg_pushdown.out
create mode 100644 src/test/regress/sql/agg_pushdown.sql
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 6414aded0e..097872f851 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2204,8 +2204,8 @@ _copyOnConflictExpr(const OnConflictExpr *from)
/* ****************************************************************
* pathnodes.h copy functions
*
- * We don't support copying RelOptInfo, IndexOptInfo, or Path nodes.
- * There are some subsidiary structs that are useful to copy, though.
+ * We don't support copying RelOptInfo, IndexOptInfo, RelAggInfo or Path
+ * nodes. There are some subsidiary structs that are useful to copy, though.
* ****************************************************************
*/
@@ -2346,6 +2346,20 @@ _copyPlaceHolderInfo(const PlaceHolderInfo *from)
return newnode;
}
+static GroupedVarInfo *
+_copyGroupedVarInfo(const GroupedVarInfo *from)
+{
+ GroupedVarInfo *newnode = makeNode(GroupedVarInfo);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gv_eval_at);
+ COPY_SCALAR_FIELD(derived);
+
+ return newnode;
+}
+
/* ****************************************************************
* parsenodes.h copy functions
* ****************************************************************
@@ -5082,6 +5096,9 @@ copyObjectImpl(const void *from)
case T_PlaceHolderInfo:
retval = _copyPlaceHolderInfo(from);
break;
+ case T_GroupedVarInfo:
+ retval = _copyGroupedVarInfo(from);
+ break;
/*
* VALUE NODES
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 01745ff879..49ee37fc7b 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2190,6 +2190,8 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_BITMAPSET_FIELD(all_baserels);
WRITE_BITMAPSET_FIELD(nullable_baserels);
WRITE_NODE_FIELD(join_rel_list);
+ WRITE_NODE_FIELD(grouped_rel_list);
+ WRITE_NODE_FIELD(agg_info_list);
WRITE_INT_FIELD(join_cur_level);
WRITE_NODE_FIELD(init_plans);
WRITE_NODE_FIELD(cte_plan_ids);
@@ -2203,6 +2205,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(append_rel_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
@@ -2210,6 +2213,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(distinct_pathkeys);
WRITE_NODE_FIELD(sort_pathkeys);
WRITE_NODE_FIELD(processed_tlist);
+ WRITE_INT_FIELD(max_sortgroupref);
WRITE_NODE_FIELD(minmax_aggs);
WRITE_FLOAT_FIELD(total_table_pages, "%.0f");
WRITE_FLOAT_FIELD(tuple_fraction, "%.4f");
@@ -2429,6 +2433,20 @@ _outParamPathInfo(StringInfo str, const ParamPathInfo *node)
}
static void
+_outRelAggInfo(StringInfo str, const RelAggInfo *node)
+{
+ WRITE_NODE_TYPE("RELAGGINFO");
+
+ WRITE_BITMAPSET_FIELD(relids);
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(agg_input);
+ WRITE_FLOAT_FIELD(input_rows, "%.0f");
+ WRITE_NODE_FIELD(group_clauses);
+ WRITE_NODE_FIELD(group_exprs);
+ WRITE_NODE_FIELD(agg_exprs);
+}
+
+static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
WRITE_NODE_TYPE("RESTRICTINFO");
@@ -2528,6 +2546,18 @@ _outPlaceHolderInfo(StringInfo str, const PlaceHolderInfo *node)
}
static void
+_outGroupedVarInfo(StringInfo str, const GroupedVarInfo *node)
+{
+ WRITE_NODE_TYPE("GROUPEDVARINFO");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_BITMAPSET_FIELD(gv_eval_at);
+ WRITE_BOOL_FIELD(derived);
+}
+
+static void
_outMinMaxAggInfo(StringInfo str, const MinMaxAggInfo *node)
{
WRITE_NODE_TYPE("MINMAXAGGINFO");
@@ -4084,6 +4114,9 @@ outNode(StringInfo str, const void *obj)
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_RelAggInfo:
+ _outRelAggInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
@@ -4102,6 +4135,9 @@ outNode(StringInfo str, const void *obj)
case T_PlaceHolderInfo:
_outPlaceHolderInfo(str, obj);
break;
+ case T_GroupedVarInfo:
+ _outGroupedVarInfo(str, obj);
+ break;
case T_MinMaxAggInfo:
_outMinMaxAggInfo(str, obj);
break;
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
index 89ce373d5e..4633fb8768 100644
--- a/src/backend/optimizer/README
+++ b/src/backend/optimizer/README
@@ -1127,3 +1127,90 @@ breaking down aggregation or grouping over a partitioned relation into
aggregation or grouping over its partitions is called partitionwise
aggregation. Especially when the partition keys match the GROUP BY clause,
this can be significantly faster than the regular method.
+
+Aggregate push-down
+-------------------
+
+The obvious way to evaluate aggregates is to evaluate the FROM clause of the
+SQL query (this is what query_planner does) and use the resuing paths as the
+input of Agg node. However, if the groups are large enough, it may be more
+efficient to apply the partial aggregation to the output of base relation
+scan, and finalize it when we have all relations of the query joined:
+
+ EXPLAIN
+ SELECT a.i, avg(b.u)
+ FROM a JOIN b ON b.j = a.i
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+ -> Index Only Scan using a_pkey on a
+ Index Cond: (i = b.j)
+
+Thus the join above the partial aggregate node receives fewer input rows, and
+so the number of outer-to-inner pairs of tuples to be checked can be
+significantly lower, which can in turn lead to considerably lower join cost.
+
+Note that there's often no GROUP BY expression to be used for the partial
+aggregation, so we use equivalence classes to derive grouping expression: in
+the example above, the grouping key "b.j" was derived from "a.i".
+
+Furthermore, extra grouping columns can be added to the partial Agg node if a
+join clause above that node references a column which is not in the query
+GROUP BY clause and which could not be derived using equivalence class.
+
+ EXPLAIN
+ SELECT a.i, avg(b.u)
+ FROM a JOIN b ON b.j = a.x
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (a.x = b.j)
+ -> Seq Scan on a
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+
+Here the partial aggregate uses "b.j" as grouping column although it's not in
+the same equivalence class as "a.i". Note that no column of the {a.x, b.j}
+equivalence class is used as a key for the final aggregation.
+
+Besides base relation, the aggregation can also be pushed down to join:
+
+ EXPLAIN
+ SELECT a.i, avg(b.u + c.v)
+ FROM a JOIN b ON b.j = a.i
+ JOIN c ON c.k = a.i
+ WHERE b.j = c.k GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (b.j = a.i)
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Hash Join
+ Hash Cond: (b.j = c.k)
+ -> Seq Scan on b
+ -> Hash
+ -> Seq Scan on c
+ -> Hash
+ -> Seq Scan on a
+
+Whether the Agg node is created out of base relation or out of join, it's
+added to a separate RelOptInfo that we call "grouped relation". Grouped
+relation can be joined to a non-grouped relation, which results in a grouped
+relation too. Join of two grouped relations does not seem to be very useful
+and is currently not supported.
+
+If query_planner produces a grouped relation that contains valid paths, these
+are simply added to the UPPERREL_PARTIAL_GROUP_AGG relation. Further
+processing of these paths then does not differ from processing of other
+partially grouped paths.
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index e9ee32b7f4..e073732ac8 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -60,6 +60,7 @@ typedef struct pushdown_safety_info
/* These parameters are set by GUC */
bool enable_geqo = false; /* just in case GUC doesn't set it */
+bool enable_agg_pushdown;
int geqo_threshold;
int min_parallel_table_scan_size;
int min_parallel_index_scan_size;
@@ -124,6 +125,9 @@ static void set_result_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
+static void add_grouped_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, AggStrategy aggstrategy,
+ RelAggInfo *agg_info);
static RelOptInfo *make_rel_from_joinlist(PlannerInfo *root, List *joinlist);
static bool subquery_is_pushdown_safe(Query *subquery, Query *topquery,
pushdown_safety_info *safetyInfo);
@@ -2731,6 +2735,80 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
}
/*
+ * generate_grouping_paths
+ * Create partially aggregated paths and add them to grouped relation.
+ *
+ * "rel_plain" is base or join relation whose paths are not grouped.
+ */
+void
+generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
+ RelOptInfo *rel_plain, RelAggInfo *agg_info)
+{
+ ListCell *lc;
+
+ if (IS_DUMMY_REL(rel_plain))
+ {
+ mark_dummy_rel(rel_grouped);
+ return;
+ }
+
+ foreach(lc, rel_plain->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ /*
+ * Since the path originates from the non-grouped relation which is
+ * not aware of the aggregate push-down, we must ensure that it
+ * provides the correct input for aggregation.
+ */
+ path = (Path *) create_projection_path(root, rel_grouped, path,
+ agg_info->agg_input);
+
+ /*
+ * add_grouped_path() will check whether the path has suitable
+ * pathkeys.
+ */
+ add_grouped_path(root, rel_grouped, path, AGG_SORTED, agg_info);
+
+ /*
+ * Repeated creation of hash table (for new parameter values) should
+ * be possible, does not sound like a good idea in terms of
+ * efficiency.
+ */
+ if (path->param_info == NULL)
+ add_grouped_path(root, rel_grouped, path, AGG_HASHED, agg_info);
+ }
+
+ /* Could not generate any grouped paths? */
+ if (rel_grouped->pathlist == NIL)
+ mark_dummy_rel(rel_grouped);
+}
+
+/*
+ * Apply partial aggregation to a subpath and add the AggPath to the pathlist.
+ */
+static void
+add_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ AggStrategy aggstrategy, RelAggInfo *agg_info)
+{
+ Path *agg_path;
+
+
+ if (aggstrategy == AGG_HASHED)
+ agg_path = (Path *) create_agg_hashed_path(root, rel, subpath,
+ agg_info);
+ else if (aggstrategy == AGG_SORTED)
+ agg_path = (Path *) create_agg_sorted_path(root, rel, subpath,
+ agg_info);
+ else
+ elog(ERROR, "unexpected strategy %d", aggstrategy);
+
+ /* Add the grouped path to the list of grouped base paths. */
+ if (agg_path != NULL)
+ add_path(rel, (Path *) agg_path);
+}
+
+/*
* make_rel_from_joinlist
* Build access paths using a "joinlist" to guide the join path search.
*
@@ -2770,6 +2848,34 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
int varno = ((RangeTblRef *) jlnode)->rtindex;
thisrel = find_base_rel(root, varno);
+
+ /*
+ * Create a grouped relation to facilitate aggregate push-down.
+ * This makes no sense if thisrel is the only relation of the
+ * query.
+ */
+ if (bms_nonempty_difference(root->all_baserels, thisrel->relids))
+ {
+ RelOptInfo *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /*
+ * Build grouped relation if thisrel is suitable for partial
+ * aggregation.
+ */
+ rel_grouped = build_simple_grouped_rel(root, varno, &agg_info);
+
+ if (rel_grouped)
+ {
+ /* Make the relation available for joining. */
+ add_grouped_rel(root, rel_grouped, agg_info);
+
+ /* Add the aggregation paths to it. */
+ generate_grouping_paths(root, rel_grouped, thisrel,
+ agg_info);
+ set_cheapest(rel_grouped);
+ }
+ }
}
else if (IsA(jlnode, List))
{
@@ -2871,6 +2977,7 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
for (lev = 2; lev <= levels_needed; lev++)
{
+ RelOptInfo *rel_grouped;
ListCell *lc;
/*
@@ -2907,6 +3014,11 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
+ /* The same for grouped relation if one exists. */
+ rel_grouped = find_grouped_rel(root, rel->relids, NULL);
+ if (rel_grouped)
+ set_cheapest(rel_grouped);
+
#ifdef OPTIMIZER_DEBUG
debug_print_rel(root, rel);
#endif
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 3a9a994733..9d7fa19970 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4414,7 +4414,6 @@ set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel)
0,
JOIN_INNER,
NULL);
-
rel->rows = clamp_row_est(nrows);
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
@@ -5386,11 +5385,11 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
foreach(lc, target->exprs)
{
Node *node = (Node *) lfirst(lc);
+ int32 item_width;
if (IsA(node, Var))
{
Var *var = (Var *) node;
- int32 item_width;
/* We should not see any upper-level Vars here */
Assert(var->varlevelsup == 0);
@@ -5421,6 +5420,20 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
Assert(item_width > 0);
tuple_width += item_width;
}
+ else if (IsA(node, Aggref))
+ {
+ /*
+ * If the target is evaluated by AggPath, it'll care of cost
+ * estimate. If the target is above AggPath (typically target of a
+ * join relation that contains grouped relation), the cost of
+ * Aggref should not be accounted for again.
+ *
+ * On the other hand, width is always needed.
+ */
+ item_width = get_typavgwidth(exprType(node), exprTypmod(node));
+ Assert(item_width > 0);
+ tuple_width += item_width;
+ }
else
{
/*
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 78d076b13c..7852cf60c4 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -2535,6 +2535,140 @@ is_redundant_derived_clause(RestrictInfo *rinfo, List *clauselist)
}
/*
+ * translate_expression_to_rels
+ * If the appropriate equivalence classes exist, replace vars in
+ * gvi->gvexpr with vars whose varno is equal to relid. Return NULL if
+ * translation is not possible or needed.
+ *
+ * Note: Currently we only translate Var expressions. This is subject to
+ * change as the aggregate push-down feature gets enhanced.
+ */
+GroupedVarInfo *
+translate_expression_to_rels(PlannerInfo *root, GroupedVarInfo *gvi,
+ Index relid)
+{
+ Var *var;
+ ListCell *l1;
+ bool found_orig = false;
+ Var *var_translated = NULL;
+ GroupedVarInfo *result;
+
+ /* Can't do anything w/o equivalence classes. */
+ if (root->eq_classes == NIL)
+ return NULL;
+
+ var = castNode(Var, gvi->gvexpr);
+
+ /*
+ * Do we need to translate the var?
+ */
+ if (var->varno == relid)
+ return NULL;
+
+ /*
+ * Find the replacement var.
+ */
+ foreach(l1, root->eq_classes)
+ {
+ EquivalenceClass *ec = lfirst_node(EquivalenceClass, l1);
+ ListCell *l2;
+
+ /* TODO Check if any other EC kind should be ignored. */
+ if (ec->ec_has_volatile || ec->ec_below_outer_join || ec->ec_broken)
+ continue;
+
+ /* Single-element EC can hardly help in translations. */
+ if (list_length(ec->ec_members) == 1)
+ continue;
+
+ /*
+ * Collect all vars of this EC and their varnos.
+ *
+ * ec->ec_relids does not help because we're only interested in a
+ * subset of EC members.
+ */
+ foreach(l2, ec->ec_members)
+ {
+ EquivalenceMember *em = lfirst_node(EquivalenceMember, l2);
+ Var *ec_var;
+
+ /*
+ * The grouping expressions derived here are used to evaluate
+ * possibility to push aggregation down to RELOPT_BASEREL or
+ * RELOPT_JOINREL relations, and to construct reltargets for the
+ * grouped rels. We're not interested at the moment whether the
+ * relations do have children.
+ */
+ if (em->em_is_child)
+ continue;
+
+ if (!IsA(em->em_expr, Var))
+ continue;
+
+ ec_var = castNode(Var, em->em_expr);
+ if (equal(ec_var, var))
+ found_orig = true;
+ else if (ec_var->varno == relid)
+ var_translated = ec_var;
+
+ if (found_orig && var_translated)
+ {
+ /*
+ * The replacement Var must have the same data type, otherwise
+ * the values are not guaranteed to be grouped in the same way
+ * as values of the original Var.
+ */
+ if (ec_var->vartype != var->vartype)
+ return NULL;
+
+ break;
+ }
+ }
+
+ if (found_orig)
+ {
+ /*
+ * The same expression probably does not exist in multiple ECs.
+ */
+ if (var_translated == NULL)
+ {
+ /*
+ * Failed to translate the expression.
+ */
+ return NULL;
+ }
+ else
+ {
+ /* Success. */
+ break;
+ }
+ }
+ else
+ {
+ /*
+ * Vars of the requested relid can be in the next ECs too.
+ */
+ var_translated = NULL;
+ }
+ }
+
+ if (!found_orig)
+ return NULL;
+
+ result = makeNode(GroupedVarInfo);
+ memcpy(result, gvi, sizeof(GroupedVarInfo));
+
+ /*
+ * translate_expression_to_rels_mutator updates gv_eval_at.
+ */
+ result->gv_eval_at = bms_make_singleton(relid);
+ result->gvexpr = (Expr *) var_translated;
+ result->derived = true;
+
+ return result;
+}
+
+/*
* is_redundant_with_indexclauses
* Test whether rinfo is redundant with any clause in the IndexClause
* list. Here, for convenience, we test both simple identity and
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index f680658cf7..d0454646c7 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -22,6 +22,7 @@
#include "partitioning/partbounds.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+#include "utils/selfuncs.h"
static void make_rels_by_clause_joins(PlannerInfo *root,
@@ -37,7 +38,9 @@ static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1,
- RelOptInfo *rel2);
+ RelOptInfo *rel2,
+ RelAggInfo *agg_info,
+ RelOptInfo *rel_agg_input);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist);
@@ -670,9 +673,17 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
/*
* make_join_rel_common
* The workhorse of make_join_rel().
- */
+ *
+ * 'agg_info' contains the reltarget of grouped relation and everything we
+ * need to aggregate the join result. If NULL, then the join relation should
+ * not be grouped.
+ *
+ * 'rel_agg_input' describes the AggPath input relation if the join output
+ * should be aggregated. If NULL is passed, do not aggregate the join output.
+ */
static RelOptInfo *
-make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ RelAggInfo *agg_info, RelOptInfo *rel_agg_input)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -733,7 +744,7 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* goes with this particular joining.
*/
joinrel = build_join_rel(root, joinrelids, rel1, rel2, sjinfo,
- &restrictlist);
+ &restrictlist, agg_info);
/*
* If we've already proven this join is empty, we needn't consider any
@@ -746,8 +757,26 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
}
/* Add paths to the join relation. */
- populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
- restrictlist);
+ if (rel_agg_input == NULL)
+ {
+ /*
+ * Simply join the input relations, whether both are plain or one of
+ * them is grouped.
+ */
+ populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
+ restrictlist);
+ }
+ else
+ {
+ /* The join relation is grouped. */
+ Assert(agg_info != NULL);
+
+ /*
+ * Apply partial aggregation to the paths of rel_agg_input and add the
+ * resulting paths to joinrel.
+ */
+ generate_grouping_paths(root, joinrel, rel_agg_input, agg_info);
+ }
bms_free(joinrelids);
@@ -755,6 +784,62 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
}
/*
+ * make_join_rel_combined
+ * Join grouped relation to non-grouped one.
+ */
+static void
+make_join_rel_combined(PlannerInfo *root, RelOptInfo *rel1,
+ RelOptInfo *rel2,
+ RelAggInfo *agg_info)
+{
+ RelOptInfo *rel1_grouped;
+ RelOptInfo *rel2_grouped;
+ bool rel1_grouped_useful = false;
+ bool rel2_grouped_useful = false;
+
+ /* Retrieve the grouped relations. */
+ rel1_grouped = find_grouped_rel(root, rel1->relids, NULL);
+ rel2_grouped = find_grouped_rel(root, rel2->relids, NULL);
+
+ /*
+ * Dummy rel may indicate a join relation that is able to generate grouped
+ * paths as such (i.e. it has valid agg_info), but for which the path
+ * actually could not be created (e.g. only AGG_HASHED strategy was
+ * possible but work_mem was not sufficient for hash table).
+ */
+ rel1_grouped_useful = rel1_grouped != NULL && !IS_DUMMY_REL(rel1_grouped);
+ rel2_grouped_useful = rel2_grouped != NULL && !IS_DUMMY_REL(rel2_grouped);
+
+ /* Nothing to do if there's no grouped relation. */
+ if (!rel1_grouped_useful && !rel2_grouped_useful)
+ return;
+
+ /*
+ * At maximum one input rel can be grouped (here we don't care if any rel
+ * is eventually dummy, the existence of grouped rel indicates that
+ * aggregates can be pushed down to it). If both were grouped, then
+ * grouping of one side would change the occurrence of the other side's
+ * aggregate transient states on the input of the final aggregation. This
+ * can be handled by adjusting the transient states, but it's not worth
+ * the effort because it's hard to find a use case for this kind of join.
+ *
+ * XXX If the join of two grouped rels is implemented someday, note that
+ * both rels can have aggregates, so it'd be hard to join grouped rel to
+ * non-grouped here: 1) such a "mixed join" would require a special
+ * target, 2) both AGGSPLIT_FINAL_DESERIAL and AGGSPLIT_SIMPLE aggregates
+ * could appear in the target of the final aggregation node, originating
+ * from the grouped and the non-grouped input rel respectively.
+ */
+ if (rel1_grouped && rel2_grouped)
+ return;
+
+ if (rel1_grouped_useful)
+ make_join_rel_common(root, rel1_grouped, rel2, agg_info, NULL);
+ else if (rel2_grouped_useful)
+ make_join_rel_common(root, rel1, rel2_grouped, agg_info, NULL);
+}
+
+/*
* make_join_rel
* Find or create a join RelOptInfo that represents the join of
* the two given rels, and add to it path information for paths
@@ -762,14 +847,83 @@ make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* (The join rel may already contain paths generated from other
* pairs of rels that add up to the same set of base rels.)
*
+ * In addition to creating an ordinary join relation, try to create a
+ * grouped one. There are two strategies to achieve that: join a grouped
+ * relation to plain one, or join two plain relations and apply partial
+ * aggregation to the result.
+ *
* NB: will return NULL if attempted join is not valid. This can happen when
* working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
+ * turned into joins. Besides that, NULL is also returned if caller is
+ * interested in a grouped relation but it could not be created.
+ *
+ * Only the plain relation is returned; if grouped relation exists, it can be
+ * retrieved using find_grouped_rel().
*/
RelOptInfo *
make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
{
- return make_join_rel_common(root, rel1, rel2);
+ Relids joinrelids;
+ RelAggInfo *agg_info = NULL;
+ RelOptInfo *joinrel,
+ *joinrel_plain;
+
+ /* 1) form the plain join. */
+ joinrel = make_join_rel_common(root, rel1, rel2, NULL, NULL);
+ joinrel_plain = joinrel;
+
+ if (joinrel_plain == NULL)
+ return joinrel_plain;
+
+ /*
+ * We're done if there are no grouping expressions nor aggregates.
+ */
+ if (root->grouped_var_list == NIL)
+ return joinrel_plain;
+
+ joinrelids = bms_union(rel1->relids, rel2->relids);
+ joinrel = find_grouped_rel(root, joinrelids, &agg_info);
+
+ if (joinrel != NULL)
+ {
+ /*
+ * If the same grouped joinrel was already formed, just with the base
+ * rels divided between rel1 and rel2 in a different way, the matching
+ * agg_info should already be there.
+ */
+ Assert(agg_info != NULL);
+ }
+ else
+ {
+ /*
+ * agg_info must be created from scratch.
+ */
+ agg_info = create_rel_agg_info(root, joinrel_plain);
+
+ /* Cannot we build grouped join? */
+ if (agg_info == NULL)
+ return joinrel_plain;
+
+ /*
+ * The number of aggregate input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ agg_info->input_rows = joinrel_plain->rows;
+ }
+
+ /*
+ * 2) join two plain rels and aggregate the join paths. Aggregate
+ * push-down only makes sense if the join is not the top-level one.
+ */
+ if (bms_nonempty_difference(root->all_baserels, joinrelids))
+ make_join_rel_common(root, rel1, rel2, agg_info, joinrel_plain);
+
+ /*
+ * 3) combine plain and grouped relations.
+ */
+ make_join_rel_combined(root, rel1, rel2, agg_info);
+
+ return joinrel_plain;
}
/*
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index 73da0c2d8e..9f97a2dd8f 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -48,6 +48,8 @@ typedef struct PostponedQual
} PostponedQual;
+static void create_aggregate_grouped_var_infos(PlannerInfo *root);
+static void create_grouping_expr_grouped_var_infos(PlannerInfo *root);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
@@ -273,6 +275,261 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
}
}
+/*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate as well as for
+ * each possible grouping expression.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+extern void
+setup_aggregate_pushdown(PlannerInfo *root)
+{
+ ListCell *lc;
+
+ /*
+ * Isn't user interested in the aggregate push-down feature?
+ */
+ if (!enable_agg_pushdown)
+ return;
+
+ /* The feature can only be applied to grouped aggregation. */
+ if (!root->parse->groupClause)
+ return;
+
+ /*
+ * Grouping sets require multiple different groupings but the base
+ * relation can only generate one.
+ */
+ if (root->parse->groupingSets)
+ return;
+
+ /*
+ * SRF is not allowed in the aggregate argument and we don't even want it
+ * in the GROUP BY clause, so forbid it in general. It needs to be
+ * analyzed if evaluation of a GROUP BY clause containing SRF below the
+ * query targetlist would be correct. Currently it does not seem to be an
+ * important use case.
+ */
+ if (root->parse->hasTargetSRFs)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_aggregate_grouped_var_infos(root);
+
+ /* Isn't there any aggregate to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Create GroupedVarInfo per grouping expression. */
+ create_grouping_expr_grouped_var_infos(root);
+
+ /* Isn't there any useful grouping expression for aggregate push-down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /*
+ * Now that we know that grouping can be pushed down, search for the
+ * maximum sortgroupref. The base relations may need it if extra grouping
+ * expressions get added to them.
+ */
+ Assert(root->max_sortgroupref == 0);
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc);
+
+ if (te->ressortgroupref > root->max_sortgroupref)
+ root->max_sortgroupref = te->ressortgroupref;
+ }
+}
+
+/*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ */
+static void
+create_aggregate_grouped_var_infos(PlannerInfo *root)
+{
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES |
+ PVC_INCLUDE_WINDOWFUNCS);
+
+ /*
+ * Although GroupingFunc is related to root->parse->groupingSets, this
+ * field does not necessarily reflect its presence.
+ */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, GroupingFunc))
+ return;
+ }
+
+ /*
+ * Aggregates within the HAVING clause need to be processed in the same
+ * way as those in the main targetlist.
+ *
+ * Note that the contained aggregates will be pushed down, but the
+ * containing HAVING clause must be ignored until the aggregation is
+ * finalized.
+ */
+ if (root->parse->havingQual != NULL)
+ {
+ List *having_exprs;
+
+ having_exprs = pull_var_clause((Node *) root->parse->havingQual,
+ PVC_INCLUDE_AGGREGATES);
+ if (having_exprs != NIL)
+ tlist_exprs = list_concat(tlist_exprs, having_exprs);
+ }
+
+ if (tlist_exprs == NIL)
+ return;
+
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ /*
+ * tlist_exprs may also contain Vars or WindowFuncs, but we only need
+ * Aggrefs.
+ */
+ if (IsA(expr, Var) ||IsA(expr, WindowFunc))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct)
+ {
+ /*
+ * Aggregation push-down is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ gvi = lfirst_node(GroupedVarInfo, lc2);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ gvi = makeNode(GroupedVarInfo);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ gvi->gv_eval_at = NULL;
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+}
+
+/*
+ * Create GroupedVarInfo for each expression usable as grouping key.
+ *
+ * In addition to the expressions of the query targetlist, group_pathkeys is
+ * also considered the source of grouping expressions. That increases the
+ * chance to get the relation output grouped.
+ */
+static void
+create_grouping_expr_grouped_var_infos(PlannerInfo *root)
+{
+ ListCell *l1,
+ *l2;
+ List *exprs = NIL;
+ List *sortgrouprefs = NIL;
+
+ /*
+ * Make sure GroupedVarInfo exists for each expression usable as grouping
+ * key.
+ */
+ foreach(l1, root->parse->groupClause)
+ {
+ SortGroupClause *sgClause;
+ TargetEntry *te;
+ Index sortgroupref;
+
+ sgClause = lfirst_node(SortGroupClause, l1);
+ te = get_sortgroupclause_tle(sgClause, root->processed_tlist);
+ sortgroupref = te->ressortgroupref;
+
+ Assert(sortgroupref > 0);
+
+ /*
+ * Non-zero sortgroupref does not necessarily imply grouping
+ * expression: data can also be sorted by aggregate.
+ */
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ /*
+ * The aggregate push-down feature currently supports only plain Vars
+ * as grouping expressions.
+ */
+ if (!IsA(te->expr, Var))
+ {
+ root->grouped_var_list = NIL;
+ return;
+ }
+
+ exprs = lappend(exprs, te->expr);
+ sortgrouprefs = lappend_int(sortgrouprefs, sortgroupref);
+ }
+
+ /*
+ * Construct GroupedVarInfo for each expression.
+ */
+ forboth(l1, exprs, l2, sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, l1);
+ int sortgroupref = lfirst_int(l2);
+ GroupedVarInfo *gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvexpr = (Expr *) copyObject(var);
+ gvi->sortgroupref = sortgroupref;
+
+ /* Find out where the expression should be evaluated. */
+ gvi->gv_eval_at = bms_make_singleton(var->varno);
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+}
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c
index 9381939c82..c4d07dfac4 100644
--- a/src/backend/optimizer/plan/planagg.c
+++ b/src/backend/optimizer/plan/planagg.c
@@ -442,7 +442,7 @@ build_minmax_path(PlannerInfo *root, MinMaxAggInfo *mminfo,
subroot->tuple_fraction = 1.0;
subroot->limit_tuples = 1.0;
- final_rel = query_planner(subroot, minmax_qp_callback, NULL);
+ final_rel = query_planner(subroot, minmax_qp_callback, NULL, NULL);
/*
* Since we didn't go through subquery_planner() to handle the subquery,
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 0b9999c8a6..4103a393e6 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -45,6 +45,9 @@
* qp_callback is a function to compute query_pathkeys once it's safe to do so
* qp_extra is optional extra data to pass to qp_callback
*
+ * If final_rel_grouped_p is valid, relation containing grouped paths may be
+ * saved to *final_rel_grouped_p.
+ *
* Note: the PlannerInfo node also includes a query_pathkeys field, which
* tells query_planner the sort order that is desired in the final output
* plan. This value is *not* available at call time, but is computed by
@@ -53,7 +56,8 @@
*/
RelOptInfo *
query_planner(PlannerInfo *root,
- query_pathkeys_callback qp_callback, void *qp_extra)
+ query_pathkeys_callback qp_callback, void *qp_extra,
+ RelOptInfo **final_rel_grouped_p)
{
Query *parse = root->parse;
List *joinlist;
@@ -66,6 +70,8 @@ query_planner(PlannerInfo *root,
* here.
*/
root->join_rel_list = makeNode(RelInfoList);
+ root->grouped_rel_list = makeNode(RelInfoList);
+ root->agg_info_list = makeNode(RelInfoList);
root->join_rel_level = NULL;
root->join_cur_level = 0;
root->canon_pathkeys = NIL;
@@ -74,6 +80,7 @@ query_planner(PlannerInfo *root,
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
@@ -255,6 +262,16 @@ query_planner(PlannerInfo *root,
extract_restriction_or_clauses(root);
/*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, setup
+ * root->grouped_var_list.
+ *
+ * The base relations should be fully initialized now, so that we have
+ * enough info to decide whether grouping is possible.
+ */
+ setup_aggregate_pushdown(root);
+
+ /*
* Now expand appendrels by adding "otherrels" for their children. We
* delay this to the end so that we have as much information as possible
* available for each baserel, including all restriction clauses. That
@@ -274,5 +291,9 @@ query_planner(PlannerInfo *root,
final_rel->cheapest_total_path->param_info != NULL)
elog(ERROR, "failed to construct the join relation");
+ if (final_rel_grouped_p)
+ *final_rel_grouped_p = find_grouped_rel(root, final_rel->relids,
+ NULL);
+
return final_rel;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index ca3b7f29e1..aa18fe5694 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -151,6 +151,7 @@ static double get_number_of_groups(PlannerInfo *root,
List *target_list);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
PathTarget *target,
bool target_parallel_safe,
const AggClauseCosts *agg_costs,
@@ -164,6 +165,7 @@ static RelOptInfo *make_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
Node *havingQual);
static void create_ordinary_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
RelOptInfo *grouped_rel,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd,
@@ -623,6 +625,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
memset(root->upper_rels, 0, sizeof(root->upper_rels));
memset(root->upper_targets, 0, sizeof(root->upper_targets));
root->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
root->grouping_map = NULL;
root->minmax_aggs = NIL;
root->qual_security_level = 0;
@@ -1943,6 +1946,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
List *activeWindows = NIL;
grouping_sets_data *gset_data = NULL;
standard_qp_extra qp_extra;
+ RelOptInfo *current_rel_grouped = NULL;
/* A recursive query should always have setOperations */
Assert(!root->hasRecursion);
@@ -2045,7 +2049,9 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
* We also generate (in standard_qp_callback) pathkey representations
* of the query's sort clause, distinct clause, etc.
*/
- current_rel = query_planner(root, standard_qp_callback, &qp_extra);
+ current_rel = query_planner(root,
+ standard_qp_callback, &qp_extra,
+ ¤t_rel_grouped);
/*
* Convert the query's result tlist into PathTarget format.
@@ -2190,6 +2196,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
{
current_rel = create_grouping_paths(root,
current_rel,
+ current_rel_grouped,
grouping_target,
grouping_target_parallel_safe,
&agg_costs,
@@ -3798,6 +3805,7 @@ get_number_of_groups(PlannerInfo *root,
static RelOptInfo *
create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
PathTarget *target,
bool target_parallel_safe,
const AggClauseCosts *agg_costs,
@@ -3888,7 +3896,8 @@ create_grouping_paths(PlannerInfo *root,
else
extra.patype = PARTITIONWISE_AGGREGATE_NONE;
- create_ordinary_grouping_paths(root, input_rel, grouped_rel,
+ create_ordinary_grouping_paths(root, input_rel, input_rel_grouped,
+ grouped_rel,
agg_costs, gd, &extra,
&partially_grouped_rel);
}
@@ -4046,6 +4055,7 @@ create_degenerate_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
*/
static void
create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
RelOptInfo *grouped_rel,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd,
@@ -4093,13 +4103,23 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
if ((extra->flags & GROUPING_CAN_PARTIAL_AGG) != 0)
{
bool force_rel_creation;
+ bool have_agg_pushdown_paths;
/*
- * If we're doing partitionwise aggregation at this level, force
- * creation of a partially_grouped_rel so we can add partitionwise
- * paths to it.
+ * Check if the aggregate push-down feature succeeded to generate any
+ * paths. Dummy relation can appear here because grouped paths are not
+ * guaranteed to exist for a relation.
*/
- force_rel_creation = (patype == PARTITIONWISE_AGGREGATE_PARTIAL);
+ have_agg_pushdown_paths = input_rel_grouped != NULL &&
+ !IS_DUMMY_REL(input_rel_grouped);
+
+ /*
+ * If we're doing partitionwise aggregation at this level or if
+ * aggregate push-down succeeded to create some paths, force creation
+ * of a partially_grouped_rel so we can add the related paths to it.
+ */
+ force_rel_creation = patype == PARTITIONWISE_AGGREGATE_PARTIAL ||
+ have_agg_pushdown_paths;
partially_grouped_rel =
create_partial_grouping_paths(root,
@@ -4108,6 +4128,23 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
gd,
extra,
force_rel_creation);
+
+ /*
+ * No further processing is needed for paths provided by the aggregate
+ * push-down feature. Simply add them to the partially grouped
+ * relation.
+ */
+ if (have_agg_pushdown_paths)
+ {
+ ListCell *lc;
+
+ foreach(lc, input_rel_grouped->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ add_path(partially_grouped_rel, path);
+ }
+ }
}
/* Set out parameter. */
@@ -4132,10 +4169,14 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
/* Gather any partially grouped partial paths. */
if (partially_grouped_rel && partially_grouped_rel->partial_pathlist)
- {
gather_grouping_paths(root, partially_grouped_rel);
+
+ /*
+ * The non-partial paths can come either from the Gather above or from
+ * aggregate push-down.
+ */
+ if (partially_grouped_rel && partially_grouped_rel->pathlist)
set_cheapest(partially_grouped_rel);
- }
/*
* Estimate number of groups.
@@ -7296,6 +7337,7 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
/* Create grouping paths for this child relation. */
create_ordinary_grouping_paths(root, child_input_rel,
+ NULL,
child_grouped_rel,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index dc11f098e0..3810345b3d 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -2380,6 +2380,39 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, Aggref))
+ {
+ Aggref *aggref = castNode(Aggref, node);
+
+ /*
+ * The upper plan targetlist can contain Aggref whose value has
+ * already been evaluated by the subplan. However this can only happen
+ * with specific value of aggsplit.
+ */
+ if (aggref->aggsplit == AGGSPLIT_INITIAL_SERIAL)
+ {
+ /* See if the Aggref has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ }
+
+ /* No referent found for Aggref */
+ elog(ERROR, "Aggref not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index e20bee0b33..a6ca35ad98 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -891,6 +891,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
memset(subroot->upper_rels, 0, sizeof(subroot->upper_rels));
memset(subroot->upper_targets, 0, sizeof(subroot->upper_targets));
subroot->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
subroot->grouping_map = NULL;
subroot->minmax_aggs = NIL;
subroot->qual_security_level = 0;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 0ac73984d2..4126caac8c 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2528,8 +2528,7 @@ create_projection_path(PlannerInfo *root,
pathnode->path.pathtype = T_Result;
pathnode->path.parent = rel;
pathnode->path.pathtarget = target;
- /* For now, assume we are above any joins, so no parameterization */
- pathnode->path.param_info = NULL;
+ pathnode->path.param_info = subpath->param_info;
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe &&
@@ -2969,6 +2968,146 @@ create_agg_path(PlannerInfo *root,
}
/*
+ * Apply AGG_SORTED aggregation path to subpath if it's suitably sorted.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+AggPath *
+create_agg_sorted_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ RelAggInfo *agg_info)
+{
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ ListCell *lc1;
+ List *key_subset = NIL;
+ AggPath *result = NULL;
+
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, (Node *) agg_exprs, aggsplit, &agg_costs);
+
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until the
+ * final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath, target,
+ AGG_SORTED, aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+
+ /* The agg path should require no fewer parameters than the plain one. */
+ result->path.param_info = subpath->param_info;
+
+ return result;
+}
+
+/*
+ * Apply AGG_HASHED aggregation to subpath.
+ */
+AggPath *
+create_agg_hashed_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, RelAggInfo *agg_info)
+{
+ bool can_hash;
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ double hashaggtablesize;
+ Query *parse = root->parse;
+ AggPath *result = NULL;
+
+ /* Do not try to create hash table for each parameter value. */
+ Assert(subpath->param_info == NULL);
+
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, agg_exprs, aggsplit, &agg_costs);
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until
+ * the final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath,
+ target,
+ AGG_HASHED,
+ aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+ }
+ }
+
+ return result;
+}
+
+/*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
*
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index c238dd6538..30ccbde60f 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -17,18 +17,24 @@
#include <limits.h>
#include "miscadmin.h"
+#include "catalog/pg_class_d.h"
+#include "catalog/pg_constraint.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
+#include "optimizer/optimizer.h"
#include "optimizer/inherit.h"
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/placeholder.h"
#include "optimizer/plancat.h"
+#include "optimizer/planner.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+#include "parser/parse_oper.h"
#include "partitioning/partbounds.h"
#include "utils/hsearch.h"
+#include "utils/selfuncs.h"
typedef struct RelInfoEntry
@@ -63,6 +69,9 @@ static void build_child_join_reltarget(PlannerInfo *root,
RelOptInfo *childrel,
int nappinfos,
AppendRelInfo **appinfos);
+static bool init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p);
/*
@@ -352,6 +361,102 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
}
/*
+ * build_simple_grouped_rel
+ * Construct a new RelOptInfo for a grouped base relation out of an
+ * existing non-grouped relation. On success, pointer to the corresponding
+ * RelAggInfo is stored in *agg_info_p in addition to returning the grouped
+ * relation.
+ */
+RelOptInfo *
+build_simple_grouped_rel(PlannerInfo *root, int relid,
+ RelAggInfo **agg_info_p)
+{
+ RangeTblEntry *rte;
+ RelOptInfo *rel_plain,
+ *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /* Isn't there any grouping expression to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return NULL;
+
+ rel_plain = root->simple_rel_array[relid];
+
+ /* Caller should only pass rti that represents base relation. */
+ Assert(rel_plain != NULL);
+
+ /*
+ * Not all RTE kinds are supported when grouping is considered.
+ *
+ * TODO Consider relaxing some of these restrictions.
+ */
+ rte = root->simple_rte_array[rel_plain->relid];
+ if (rte->rtekind != RTE_RELATION ||
+ rte->relkind == RELKIND_FOREIGN_TABLE ||
+ rte->tablesample != NULL)
+ return NULL;
+
+ /*
+ * Grouped append relation is not supported yet.
+ */
+ if (rte->inh)
+ return NULL;
+
+ /*
+ * Currently we do not support child relations ("other rels").
+ */
+ if (rel_plain->reloptkind != RELOPT_BASEREL)
+ return NULL;
+
+ /*
+ * Prepare the information we need for aggregation of the rel contents.
+ */
+ agg_info = create_rel_agg_info(root, rel_plain);
+ if (agg_info == NULL)
+ return NULL;
+
+ /*
+ * TODO Consider if 1) a flat copy is o.k., 2) it's safer in terms of
+ * adding new fields to RelOptInfo) to copy everything and then reset some
+ * fields, or to zero the structure and copy individual fields.
+ */
+ rel_grouped = makeNode(RelOptInfo);
+ memcpy(rel_grouped, rel_plain, sizeof(RelOptInfo));
+
+ /*
+ * Note on consider_startup: while the AGG_HASHED strategy needs the whole
+ * relation, AGG_SORTED does not. Therefore we do not force
+ * consider_startup to false.
+ */
+
+ /*
+ * Set the appropriate target for grouped paths.
+ *
+ * reltarget should match the target of partially aggregated paths.
+ */
+ rel_grouped->reltarget = agg_info->target;
+
+ /*
+ * Grouped paths must not be mixed with the plain ones.
+ */
+ rel_grouped->pathlist = NIL;
+ rel_grouped->partial_pathlist = NIL;
+ rel_grouped->cheapest_startup_path = NULL;
+ rel_grouped->cheapest_total_path = NULL;
+ rel_grouped->cheapest_unique_path = NULL;
+ rel_grouped->cheapest_parameterized_paths = NIL;
+
+ /*
+ * The number of aggregation input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ agg_info->input_rows = rel_plain->rows;
+
+ *agg_info_p = agg_info;
+ return rel_grouped;
+}
+
+/*
* find_base_rel
* Find a base or other relation entry, which must already exist.
*/
@@ -460,10 +565,14 @@ find_rel_info(RelInfoList *list, Relids relids)
foreach(l, list->items)
{
void *item = lfirst(l);
- Relids item_relids;
+ Relids item_relids = NULL;
- Assert(IsA(item, RelOptInfo));
- item_relids = ((RelOptInfo *) item)->relids;
+ Assert(IsA(item, RelOptInfo) ||IsA(item, RelAggInfo));
+
+ if (IsA(item, RelOptInfo))
+ item_relids = ((RelOptInfo *) item)->relids;
+ else if (IsA(item, RelAggInfo))
+ item_relids = ((RelAggInfo *) item)->relids;
if (bms_equal(item_relids, relids))
return item;
@@ -492,7 +601,7 @@ find_join_rel(PlannerInfo *root, Relids relids)
static void
add_rel_info(RelInfoList *list, void *data)
{
- Assert(IsA(data, RelOptInfo));
+ Assert(IsA(data, RelOptInfo) ||IsA(data, RelAggInfo));
/* GEQO requires us to append the new joinrel to the end of the list! */
list->items = lappend(list->items, data);
@@ -504,7 +613,11 @@ add_rel_info(RelInfoList *list, void *data)
RelInfoEntry *hentry;
bool found;
- relids = ((RelOptInfo *) data)->relids;
+ if (IsA(data, RelOptInfo))
+ relids = ((RelOptInfo *) data)->relids;
+ else if (IsA(data, RelAggInfo))
+ relids = ((RelAggInfo *) data)->relids;
+
hentry = (RelInfoEntry *) hash_search(list->hash,
&relids,
HASH_ENTER,
@@ -526,6 +639,57 @@ add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
}
/*
+ * add_grouped_rel
+ * Add grouped base or join relation to the list of grouped relations in
+ * the given PlannerInfo. Also add the corresponding RelAggInfo to
+ * agg_info_list.
+ */
+void
+add_grouped_rel(PlannerInfo *root, RelOptInfo *rel, RelAggInfo *agg_info)
+{
+ add_rel_info(root->grouped_rel_list, rel);
+ add_rel_info(root->agg_info_list, agg_info);
+}
+
+/*
+ * find_grouped_rel
+ * Returns grouped relation entry (base or join relation) corresponding to
+ * 'relids' or NULL if none exists.
+ *
+ * If agg_info_p is a valid pointer, then pointer to RelAggInfo that
+ * corresponds to the relation returned is assigned to *agg_info_p.
+ */
+RelOptInfo *
+find_grouped_rel(PlannerInfo *root, Relids relids, RelAggInfo **agg_info_p)
+{
+ RelOptInfo *rel;
+
+ rel = (RelOptInfo *) find_rel_info(root->grouped_rel_list, relids);
+ if (rel == NULL)
+ {
+ if (agg_info_p)
+ *agg_info_p = NULL;
+
+ return NULL;
+ }
+
+ /* Is caller interested in RelAggInfo? */
+ if (agg_info_p)
+ {
+ RelAggInfo *agg_info;
+
+ agg_info = (RelAggInfo *) find_rel_info(root->agg_info_list, relids);
+
+ /* The relation exists, so the agg_info should be there too. */
+ Assert(agg_info != NULL);
+
+ *agg_info_p = agg_info;
+ }
+
+ return rel;
+}
+
+/*
* set_foreign_rel_properties
* Set up foreign-join fields if outer and inner relation are foreign
* tables (or joins) belonging to the same server and assigned to the same
@@ -587,6 +751,7 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* 'restrictlist_ptr': result variable. If not NULL, *restrictlist_ptr
* receives the list of RestrictInfo nodes that apply to this
* particular pair of joinable relations.
+ * 'agg_info' indicates that grouped join relation should be created.
*
* restrictlist_ptr makes the routine's API a little grotty, but it saves
* duplicated calculation of the restrictlist...
@@ -597,10 +762,12 @@ build_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr)
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info)
{
RelOptInfo *joinrel;
List *restrictlist;
+ bool grouped = agg_info != NULL;
/* This function should be used only for join between parents. */
Assert(!IS_OTHER_REL(outer_rel) && !IS_OTHER_REL(inner_rel));
@@ -608,7 +775,8 @@ build_join_rel(PlannerInfo *root,
/*
* See if we already have a joinrel for this set of base rels.
*/
- joinrel = find_join_rel(root, joinrelids);
+ joinrel = !grouped ? find_join_rel(root, joinrelids) :
+ find_grouped_rel(root, joinrelids, NULL);
if (joinrel)
{
@@ -700,9 +868,21 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ if (!grouped)
+ {
+ joinrel->reltarget = create_empty_pathtarget();
+ build_joinrel_tlist(root, joinrel, outer_rel);
+ build_joinrel_tlist(root, joinrel, inner_rel);
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ }
+ else
+ {
+ /*
+ * The target for grouped join should already have its cost and width
+ * computed, see create_rel_agg_info().
+ */
+ joinrel->reltarget = agg_info->target;
+ }
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -734,49 +914,73 @@ build_join_rel(PlannerInfo *root,
joinrel->has_eclass_joins = has_relevant_eclass_joinclause(root, joinrel);
/* Store the partition information. */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- sjinfo->jointype);
-
- /*
- * Set estimates of the joinrel's size.
- */
- set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
- sjinfo, restrictlist);
-
- /*
- * Set the consider_parallel flag if this joinrel could potentially be
- * scanned within a parallel worker. If this flag is false for either
- * inner_rel or outer_rel, then it must be false for the joinrel also.
- * Even if both are true, there might be parallel-restricted expressions
- * in the targetlist or quals.
- *
- * Note that if there are more than two rels in this relation, they could
- * be divided between inner_rel and outer_rel in any arbitrary way. We
- * assume this doesn't matter, because we should hit all the same baserels
- * and joinclauses while building up to this joinrel no matter which we
- * take; therefore, we should make the same decision here however we get
- * here.
- */
- if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
- is_parallel_safe(root, (Node *) restrictlist) &&
- is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
- joinrel->consider_parallel = true;
+ if (!grouped)
+ build_joinrel_partition_info(joinrel, outer_rel, inner_rel,
+ restrictlist, sjinfo->jointype);
/* Add the joinrel to the PlannerInfo. */
- add_join_rel(root, joinrel);
+ if (!grouped)
+ add_join_rel(root, joinrel);
+ else
+ add_grouped_rel(root, joinrel, agg_info);
/*
- * Also, if dynamic-programming join search is active, add the new joinrel
- * to the appropriate sublist. Note: you might think the Assert on number
- * of members should be for equality, but some of the level 1 rels might
- * have been joinrels already, so we can only assert <=.
+ * Also, if dynamic-programming join search is active, add the new
+ * joinrelset to the appropriate sublist. Note: you might think the
+ * Assert on number of members should be for equality, but some of the
+ * level 1 rels might have been joinrels already, so we can only assert
+ * <=.
+ *
+ * Do noting for grouped relation as it's stored aside from
+ * join_rel_level.
*/
- if (root->join_rel_level)
+ if (root->join_rel_level && !grouped)
{
Assert(root->join_cur_level > 0);
- Assert(root->join_cur_level <= bms_num_members(joinrel->relids));
+ Assert(root->join_cur_level <= bms_num_members(joinrelids));
root->join_rel_level[root->join_cur_level] =
- lappend(root->join_rel_level[root->join_cur_level], joinrel);
+ lappend(root->join_rel_level[root->join_cur_level],
+ joinrel);
+ }
+
+ /* Set estimates of the joinrel's size. */
+ if (!grouped)
+ {
+ set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
+ sjinfo, restrictlist);
+
+ /*
+ * Set the consider_parallel flag if this joinrel could potentially be
+ * scanned within a parallel worker. If this flag is false for either
+ * inner_rel or outer_rel, then it must be false for the joinrel also.
+ * Even if both are true, there might be parallel-restricted
+ * expressions in the targetlist or quals.
+ *
+ * Note that if there are more than two rels in this relation, they
+ * could be divided between inner_rel and outer_rel in any arbitrary
+ * way. We assume this doesn't matter, because we should hit all the
+ * same baserels and joinclauses while building up to this joinrel no
+ * matter which we take; therefore, we should make the same decision
+ * here however we get here.
+ */
+ if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
+ is_parallel_safe(root, (Node *) restrictlist) &&
+ is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
+ joinrel->consider_parallel = true;
+ }
+ else
+ {
+ /*
+ * Grouping essentially changes the number of rows.
+ *
+ * XXX We do not distinguish whether two plain rels are joined and the
+ * result is aggregated, or the aggregation has been already applied
+ * to one of the input rels. Is this worth extra effort, e.g.
+ * maintaining a separate RelOptInfo for each case (one difficulty
+ * that would introduce is construction of AppendPath)?
+ */
+ joinrel->rows = estimate_num_groups(root, agg_info->group_exprs,
+ agg_info->input_rows, NULL);
}
return joinrel;
@@ -1820,3 +2024,625 @@ build_child_join_reltarget(PlannerInfo *root,
childrel->reltarget->cost.per_tuple = parentrel->reltarget->cost.per_tuple;
childrel->reltarget->width = parentrel->reltarget->width;
}
+
+/*
+ * Check if the relation can produce grouped paths and return the information
+ * it'll need for it. The passed relation is the non-grouped one which has the
+ * reltarget already constructed.
+ */
+RelAggInfo *
+create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel)
+{
+ List *gvis;
+ List *aggregates = NIL;
+ bool found_other_rel_agg;
+ ListCell *lc;
+ RelAggInfo *result;
+ PathTarget *agg_input;
+ PathTarget *target = NULL;
+ List *grp_exprs_extra = NIL;
+ List *group_clauses_final;
+ int i;
+
+ /*
+ * The function shouldn't have been called if there's no opportunity for
+ * aggregation push-down.
+ */
+ Assert(root->grouped_var_list != NIL);
+
+ result = makeNode(RelAggInfo);
+
+ /*
+ * The current implementation of aggregation push-down cannot handle
+ * PlaceHolderVar (PHV).
+ *
+ * If we knew that the PHV should be evaluated in this target (and of
+ * course, if its expression matched some Aggref argument), we'd just let
+ * init_grouping_targets add that Aggref. On the other hand, if we knew
+ * that the PHV is evaluated below the current rel, we could ignore it
+ * because the referencing Aggref would take care of propagation of the
+ * value to upper joins.
+ *
+ * The problem is that the same PHV can be evaluated in the target of the
+ * current rel or in that of lower rel --- depending on the input paths.
+ * For example, consider rel->relids = {A, B, C} and if ph_eval_at = {B,
+ * C}. Path "A JOIN (B JOIN C)" implies that the PHV is evaluated by the
+ * "(B JOIN C)", while path "(A JOIN B) JOIN C" evaluates the PHV itself.
+ */
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *expr = lfirst(lc);
+
+ if (IsA(expr, PlaceHolderVar))
+ return NULL;
+ }
+
+ if (IS_SIMPLE_REL(rel))
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return NULL;
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the Agg plan
+ * would receive different input at the base rel level.
+ *
+ * XXX For RELOPT_JOINREL, do not return if all the joins that can set any
+ * entry of the grouped target (do we need to postpone this check until
+ * the grouped target is available, and init_grouping_targets take care?)
+ * of this rel to NULL are provably below rel. (It's ok if rel is one of
+ * these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return NULL;
+
+ /*
+ * Use equivalence classes to generate additional grouping expressions for
+ * the current rel. Without these we might not be able to apply
+ * aggregation to the relation result set.
+ *
+ * It's important that create_grouping_expr_grouped_var_infos has
+ * processed the explicit grouping columns by now. If the grouping clause
+ * contains multiple expressions belonging to the same EC, the original
+ * (i.e. not derived) one should be preferred when we build grouping
+ * target for a relation. Otherwise we have a problem when trying to match
+ * target entries to grouping clauses during plan creation, see
+ * get_grouping_expression().
+ */
+ gvis = list_copy(root->grouped_var_list);
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+ int relid = -1;
+
+ /* Only interested in grouping expressions. */
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ while ((relid = bms_next_member(rel->relids, relid)) >= 0)
+ {
+ GroupedVarInfo *gvi_trans;
+
+ gvi_trans = translate_expression_to_rels(root, gvi, relid);
+ if (gvi_trans != NULL)
+ gvis = lappend(gvis, gvi_trans);
+ }
+ }
+
+ /*
+ * Check if some aggregates or grouping expressions can be evaluated in
+ * this relation's target, and collect all vars referenced by these
+ * aggregates / grouping expressions;
+ */
+ found_other_rel_agg = false;
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which includes
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * init_grouping_targets will handle plain Var grouping
+ * expressions because it needs to look them up in
+ * grouped_var_list anyway.
+ */
+ if (IsA(gvi->gvexpr, Var))
+ continue;
+
+ /*
+ * Currently, GroupedVarInfo only handles Vars and Aggrefs.
+ */
+ Assert(IsA(gvi->gvexpr, Aggref));
+
+ /* We only derive grouped expressions using ECs, not aggregates */
+ Assert(!gvi->derived);
+
+ gvi->agg_partial = (Aggref *) copyObject(gvi->gvexpr);
+ mark_partial_aggref(gvi->agg_partial, AGGSPLIT_INITIAL_SERIAL);
+
+ /*
+ * Accept the aggregate.
+ */
+ aggregates = lappend(aggregates, gvi);
+ }
+ else if (IsA(gvi->gvexpr, Aggref))
+ {
+ /*
+ * Remember that there is at least one aggregate expression that
+ * needs something else than this rel.
+ */
+ found_other_rel_agg = true;
+
+ /*
+ * This condition effectively terminates creation of the
+ * RelAggInfo, so there's no reason to check the next
+ * GroupedVarInfo.
+ */
+ break;
+ }
+ }
+
+ /*
+ * Grouping makes little sense w/o aggregate function and w/o grouping
+ * expressions.
+ */
+ if (aggregates == NIL)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Give up if some other aggregate(s) need relations other than the
+ * current one.
+ *
+ * If the aggregate needs the current rel plus anything else, then the
+ * problem is that grouping of the current relation could make some input
+ * variables unavailable for the "higher aggregate", and it'd also
+ * decrease the number of input rows the "higher aggregate" receives.
+ *
+ * If the aggregate does not even need the current rel, then neither the
+ * current rel nor anything else should be grouped because we do not
+ * support join of two grouped relations.
+ */
+ if (found_other_rel_agg)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Create target for grouped paths as well as one for the input paths of
+ * the aggregation paths.
+ */
+ target = create_empty_pathtarget();
+ agg_input = create_empty_pathtarget();
+
+ /*
+ * Cannot suitable targets for the aggregation push-down be derived?
+ */
+ if (!init_grouping_targets(root, rel, target, agg_input, gvis,
+ &grp_exprs_extra))
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ list_free(gvis);
+
+ /*
+ * Aggregation push-down makes no sense w/o grouping expressions.
+ */
+ if ((list_length(target->exprs) + list_length(grp_exprs_extra)) == 0)
+ return NULL;
+
+ group_clauses_final = root->parse->groupClause;
+
+ /*
+ * If the aggregation target should have extra grouping expressions (in
+ * order to emit input vars for join conditions), add them now. This step
+ * includes assignment of tleSortGroupRef's which we can generate now.
+ */
+ if (list_length(grp_exprs_extra) > 0)
+ {
+ Index sortgroupref;
+
+ /*
+ * We'll have to add some clauses, but query group clause must be
+ * preserved.
+ */
+ group_clauses_final = list_copy(group_clauses_final);
+
+ /*
+ * Always start at root->max_sortgroupref. The extra grouping
+ * expressions aren't used during the final aggregation, so the
+ * sortgroupref values don't need to be unique across the query. Thus
+ * we don't have to increase root->max_sortgroupref, which makes
+ * recognition of the extra grouping expressions pretty easy.
+ */
+ sortgroupref = root->max_sortgroupref;
+
+ /*
+ * Generate the SortGroupClause's and add the expressions to the
+ * target.
+ */
+ foreach(lc, grp_exprs_extra)
+ {
+ Var *var = lfirst_node(Var, lc);
+ SortGroupClause *cl = makeNode(SortGroupClause);
+
+ /*
+ * Initialize the SortGroupClause.
+ *
+ * As the final aggregation will not use this grouping expression,
+ * we don't care whether sortop is < or >. The value of
+ * nulls_first should not matter for the same reason.
+ */
+ cl->tleSortGroupRef = ++sortgroupref;
+ get_sort_group_operators(var->vartype,
+ false, true, false,
+ &cl->sortop, &cl->eqop, NULL,
+ &cl->hashable);
+ group_clauses_final = lappend(group_clauses_final, cl);
+ add_column_to_pathtarget(target, (Expr *) var,
+ cl->tleSortGroupRef);
+
+ /*
+ * The aggregation input target must emit this var too.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) var,
+ cl->tleSortGroupRef);
+ }
+ }
+
+ /*
+ * Add aggregates to the grouping target.
+ */
+ add_aggregates_to_target(root, target, aggregates);
+
+ /*
+ * Build a list of grouping expressions and a list of the corresponding
+ * SortGroupClauses.
+ */
+ i = 0;
+ foreach(lc, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(lc);
+
+ if (IsA(texpr, Aggref))
+ {
+ /*
+ * Once we see Aggref, no grouping expressions should follow.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides being an aggregate, the target expression should have no
+ * other reason then being a column of a relation functionally
+ * dependent on the GROUP BY clause. So it's not actually a grouping
+ * column.
+ */
+ if (sortgroupref == 0)
+ continue;
+
+ /*
+ * group_clause_final contains the "local" clauses, so this search
+ * should succeed.
+ */
+ cl = get_sortgroupref_clause(sortgroupref, group_clauses_final);
+
+ result->group_clauses = list_append_unique(result->group_clauses,
+ cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ result->group_exprs = list_append_unique(result->group_exprs,
+ texpr);
+ }
+
+ /*
+ * Since neither target nor agg_input is supposed to be identical to the
+ * source reltarget, compute the width and cost again.
+ *
+ * target does not yet contain aggregates, but these will be accounted by
+ * AggPath.
+ */
+ set_pathtarget_cost_width(root, target);
+ set_pathtarget_cost_width(root, agg_input);
+
+ result->relids = bms_copy(rel->relids);
+ result->target = target;
+ result->agg_input = agg_input;
+
+ /* Finally collect the aggregates. */
+ while (lc != NULL)
+ {
+ Aggref *aggref = lfirst_node(Aggref, lc);
+
+ /*
+ * Partial aggregation is what the grouped paths should do.
+ */
+ result->agg_exprs = lappend(result->agg_exprs, aggref);
+ lc = lnext(target->exprs, lc);
+ }
+
+ /* The "input_rows" field should be set by caller. */
+ return result;
+}
+
+/*
+ * Initialize target for grouped paths (target) as well as a target for paths
+ * that generate input for aggregation (agg_input).
+ *
+ * group_exprs_extra_p receives a list of Var nodes for which we need to
+ * construct SortGroupClause. Those vars will then be used as additional
+ * grouping expressions, for the sake of join clauses.
+ *
+ * gvis a list of GroupedVarInfo's possibly useful for rel.
+ *
+ * Return true iff the targets could be initialized.
+ */
+static bool
+init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p)
+{
+ ListCell *lc1,
+ *lc2;
+ List *unresolved = NIL;
+ List *unresolved_sortgrouprefs = NIL;
+
+ foreach(lc1, rel->reltarget->exprs)
+ {
+ Var *tvar;
+ bool is_grouping;
+ Index sortgroupref = 0;
+ bool derived = false;
+ bool needed_by_aggregate;
+
+ /*
+ * Given that PlaceHolderVar currently prevents us from doing
+ * aggregation push-down, the source target cannot contain anything
+ * more complex than a Var.
+ */
+ tvar = lfirst_node(Var, lc1);
+
+ is_grouping = is_grouping_expression(gvis, (Expr *) tvar,
+ &sortgroupref, &derived);
+
+ /*
+ * Derived grouping expressions should not be referenced by the query
+ * targetlist, so let them fall into vars_unresolved. It'll be checked
+ * later if the current targetlist needs them. For example, we should
+ * not automatically use Var as a grouping expression if the only
+ * reason for it to be in the plain relation target is that it's
+ * referenced by aggregate argument, and it happens to be in the same
+ * EC as any grouping expression.
+ */
+ if (is_grouping && !derived)
+ {
+ Assert(sortgroupref > 0);
+
+ /*
+ * It's o.k. to use the target expression for grouping.
+ */
+ add_column_to_pathtarget(target, (Expr *) tvar, sortgroupref);
+
+ /*
+ * As for agg_input, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) tvar, sortgroupref);
+
+ /* Process the next expression. */
+ continue;
+ }
+
+ /*
+ * Is this Var needed in the query targetlist for anything else than
+ * aggregate input?
+ */
+ needed_by_aggregate = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc2);
+ ListCell *lc3;
+ List *vars;
+
+ if (!IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (!bms_is_member(tvar->varno, gvi->gv_eval_at))
+ continue;
+
+ /*
+ * XXX Consider some sort of caching.
+ */
+ vars = pull_var_clause((Node *) gvi->gvexpr, PVC_RECURSE_AGGREGATES);
+ foreach(lc3, vars)
+ {
+ Var *var = lfirst_node(Var, lc3);
+
+ if (equal(var, tvar))
+ {
+ needed_by_aggregate = true;
+ break;
+ }
+ }
+ list_free(vars);
+ if (needed_by_aggregate)
+ break;
+ }
+
+ if (needed_by_aggregate)
+ {
+ bool found = false;
+
+ foreach(lc2, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc2);
+
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ if (equal(te->expr, tvar))
+ {
+ found = true;
+ break;
+ }
+ }
+
+ /*
+ * If it's only Aggref input, add it to the aggregation input
+ * target and that's it.
+ */
+ if (!found)
+ {
+ add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
+ continue;
+ }
+ }
+
+ /*
+ * Further investigation involves dependency check, for which we need
+ * to have all the (plain-var) grouping expressions gathered.
+ */
+ unresolved = lappend(unresolved, tvar);
+ unresolved_sortgrouprefs = lappend_int(unresolved_sortgrouprefs,
+ sortgroupref);
+ }
+
+ /*
+ * Check for other possible reasons for the var to be in the plain target.
+ */
+ forboth(lc1, unresolved, lc2, unresolved_sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, lc1);
+ Index sortgroupref = lfirst_int(lc2);
+ RangeTblEntry *rte;
+ List *deps = NIL;
+ Relids relids_subtract;
+ int ndx;
+ RelOptInfo *baserel;
+
+ rte = root->simple_rte_array[var->varno];
+
+ /*
+ * Check if the Var can be in the grouping key even though it's not
+ * mentioned by the GROUP BY clause (and could not be derived using
+ * ECs).
+ */
+ if (sortgroupref == 0 &&
+ check_functional_grouping(rte->relid, var->varno,
+ var->varlevelsup,
+ target->exprs, &deps))
+ {
+ /*
+ * The var shouldn't be actually used as a grouping key (instead,
+ * the one this depends on will be), so sortgroupref should not be
+ * important.
+ */
+ add_new_column_to_pathtarget(target, (Expr *) var);
+ add_new_column_to_pathtarget(agg_input, (Expr *) var);
+
+ /*
+ * The var may or may not be present in generic grouping
+ * expression(s) in addition, but this is handled elsewhere.
+ */
+ continue;
+ }
+
+ /*
+ * Isn't the expression needed by joins above the current rel?
+ *
+ * The relids we're not interested in do include 0, which is the
+ * top-level targetlist. The only reason for relids to contain 0
+ * should be that arg_var is referenced either by aggregate or by
+ * grouping expression, but right now we're interested in the *other*
+ * reasons. (As soon aggregation is pushed down, the aggregates in the
+ * query targetlist no longer need direct reference to arg_var
+ * anyway.)
+ */
+ relids_subtract = bms_copy(rel->relids);
+ bms_add_member(relids_subtract, 0);
+
+ baserel = find_base_rel(root, var->varno);
+ ndx = var->varattno - baserel->min_attr;
+ if (bms_nonempty_difference(baserel->attr_needed[ndx],
+ relids_subtract))
+ {
+ /*
+ * The variable is needed by a join involving this relation. That
+ * case includes variable that is referenced by a generic grouping
+ * expression.
+ *
+ * The only way to bring this var to the aggregation output is to
+ * add it to the grouping expressions too.
+ */
+ if (sortgroupref > 0)
+ {
+ /*
+ * The var could be recognized as a potentially useful
+ * grouping expression at the top of the loop, so we can add
+ * it to the grouping target, as well as to the agg_input.
+ */
+ add_column_to_pathtarget(target, (Expr *) var, sortgroupref);
+ add_column_to_pathtarget(agg_input, (Expr *) var, sortgroupref);
+ }
+ else
+ {
+ /*
+ * Since root->parse->groupClause is not supposed to contain
+ * this expression, we need to construct special
+ * SortGroupClause. Its tleSortGroupRef needs to be unique
+ * within target_agg, so postpone creation of the
+ * SortGroupRefs until we're done with the iteration of
+ * rel->reltarget->exprs.
+ */
+ *group_exprs_extra_p = lappend(*group_exprs_extra_p, var);
+ }
+ }
+ else
+ {
+ /*
+ * As long as the query is semantically correct, arriving here
+ * means that the var is referenced by a generic grouping
+ * expression but not referenced by any join.
+ *
+ * create_rel_agg_info() should add this variable to "agg_input"
+ * target and also add the whole generic expression to "target",
+ * but that's subject to future enhancement of the aggregate
+ * push-down feature.
+ */
+ return false;
+ }
+ }
+
+ return true;
+}
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
index 7ccb10e4e1..01dc70b27f 100644
--- a/src/backend/optimizer/util/tlist.c
+++ b/src/backend/optimizer/util/tlist.c
@@ -827,6 +827,62 @@ apply_pathtarget_labeling_to_tlist(List *tlist, PathTarget *target)
}
/*
+ * For each aggregate grouping expression or aggregate to the grouped target.
+ *
+ * Caller passes the expressions in the form of GroupedVarInfos so that we
+ * don't have to look for gvid.
+ */
+void
+add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions)
+{
+ ListCell *lc;
+
+ /* Create the vars and add them to the target. */
+ foreach(lc, expressions)
+ {
+ GroupedVarInfo *gvi;
+
+ gvi = lfirst_node(GroupedVarInfo, lc);
+ add_column_to_pathtarget(target, (Expr *) gvi->agg_partial,
+ gvi->sortgroupref);
+ }
+}
+
+/*
+ * Find out if expr can be used as grouping expression in reltarget.
+ *
+ * sortgroupref and is_derived reflect the ->sortgroupref and ->derived fields
+ * of the corresponding GroupedVarInfo.
+ */
+bool
+is_grouping_expression(List *gvis, Expr *expr, Index *sortgroupref,
+ bool *is_derived)
+{
+ ListCell *lc;
+
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (equal(gvi->gvexpr, expr))
+ {
+ Assert(gvi->sortgroupref > 0);
+
+ *sortgroupref = gvi->sortgroupref;
+ *is_derived = gvi->derived;
+ return true;
+ }
+ }
+
+ /* The expression cannot be used as grouping key. */
+ return false;
+}
+
+/*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
*
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index fc463601ff..8c1929c758 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -1022,6 +1022,15 @@ static struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
{
+ {"enable_agg_pushdown", PGC_USERSET, QUERY_TUNING_METHOD,
+ gettext_noop("Enables aggregation push-down."),
+ NULL
+ },
+ &enable_agg_pushdown,
+ false,
+ NULL, NULL, NULL
+ },
+ {
{"enable_parallel_append", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables the planner's use of parallel append plans."),
NULL,
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 11027cdb10..25d058ef2a 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -224,6 +224,7 @@ typedef enum NodeTag
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_RelAggInfo,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
@@ -268,6 +269,7 @@ typedef enum NodeTag
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 38dc186623..988d6220c8 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -241,6 +241,23 @@ struct PlannerInfo
struct RelInfoList *join_rel_list; /* list of join-relation RelOptInfos */
/*
+ * grouped_rel_list is a list of RelOptInfos that represent grouped
+ * relations, both base relations and joins. Unlike join_rel_list, base
+ * relations are accepted because grouped base relation is of rather
+ * limited scope, i.e. it's only needed during join search. Thus it does
+ * not deserve separate storage like simple_rel_array.
+ */
+ struct RelInfoList *grouped_rel_list; /* list of grouped relation
+ * RelOptInfos */
+
+ /*
+ * agg_info_list contains one instance of RelAggInfo per an item of
+ * grouped_rel_list.
+ */
+ struct RelInfoList *agg_info_list; /* list of grouped relation
+ * RelAggInfos */
+
+ /*
* When doing a dynamic-programming-style join search, join_rel_level[k]
* is a list of all join-relation RelOptInfos of level k, and
* join_cur_level is the current level. New join-relation RelOptInfos are
@@ -285,6 +302,8 @@ struct PlannerInfo
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
@@ -316,6 +335,12 @@ struct PlannerInfo
*/
List *processed_tlist;
+ /*
+ * The maximum ressortgroupref among target entries in processed_list.
+ * Useful when adding extra grouping expressions for partial aggregation.
+ */
+ int max_sortgroupref;
+
/* Fields filled during create_plan() for use in setrefs.c */
AttrNumber *grouping_map; /* for GroupingFunc fixup */
List *minmax_aggs; /* List of MinMaxAggInfos */
@@ -755,6 +780,60 @@ typedef struct RelInfoList
} RelInfoList;
/*
+ * RelAggInfo
+ * Information needed to create grouped paths for base rels and joins.
+ *
+ * "relids" is the set of base-relation identifiers, just like with
+ * RelOptInfo.
+ *
+ * "target" will be used as pathtarget if partial aggregation is applied to
+ * base relation or join. The same target will also --- if the relation is a
+ * join --- be used to joinin grouped path to a non-grouped one. This target
+ * can contain plain-Var grouping expressions and Aggref nodes.
+ *
+ * Note: There's a convention that Aggref expressions are supposed to follow
+ * the other expressions of the target. Iterations of ->exprs may rely on this
+ * arrangement.
+ *
+ * "agg_input" contains Vars used either as grouping expressions or aggregate
+ * arguments. Paths providing the aggregation plan with input data should use
+ * this target. The only difference from reltarget of the non-grouped relation
+ * is that some items can have sortgroupref initialized.
+ *
+ * "input_rows" is the estimated number of input rows for AggPath. It's
+ * actually just a workspace for users of the structure, i.e. not initialized
+ * when instance of the structure is created.
+ *
+ * "group_clauses" and "group_exprs" are lists of SortGroupClause and the
+ * corresponding grouping expressions respectively.
+ *
+ * "agg_exprs" is a list of Aggref nodes for the aggregation of the relation's
+ * paths.
+ *
+ * "rel_grouped" is the relation containing the partially aggregated paths.
+ */
+typedef struct RelAggInfo
+{
+ NodeTag type;
+
+ Relids relids; /* Base rels contained in this grouped rel. */
+
+ struct PathTarget *target; /* Target for grouped paths. */
+
+ struct PathTarget *agg_input; /* pathtarget of paths that generate input
+ * for aggregation paths. */
+
+ double input_rows;
+
+ List *group_clauses;
+ List *group_exprs;
+
+ List *agg_exprs; /* Aggref expressions. */
+
+ RelOptInfo *rel_grouped; /* Grouped relation. */
+} RelAggInfo;
+
+/*
* IndexOptInfo
* Per-index information for planning/optimization
*
@@ -2273,6 +2352,26 @@ typedef struct PlaceHolderInfo
} PlaceHolderInfo;
/*
+ * GroupedVarInfo exists for each expression that can be used as an aggregate
+ * or grouping expression evaluated below a join.
+ */
+typedef struct GroupedVarInfo
+{
+ NodeTag type;
+
+ Expr *gvexpr; /* the represented expression. */
+ Aggref *agg_partial; /* if gvexpr is aggregate, agg_partial is the
+ * corresponding partial aggregate */
+ Index sortgroupref; /* If gvexpr is a grouping expression, this is
+ * the tleSortGroupRef of the corresponding
+ * SortGroupClause. */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+ bool derived; /* derived from another GroupedVarInfo using
+ * equeivalence classes? */
+} GroupedVarInfo;
+
+/*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
* path, the list is stored into root->minmax_aggs for use during setrefs.c.
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
index 2f9aeec4a7..90f6995dc2 100644
--- a/src/include/optimizer/clauses.h
+++ b/src/include/optimizer/clauses.h
@@ -55,4 +55,6 @@ extern void CommuteOpExpr(OpExpr *clause);
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
+extern GroupedVarInfo *translate_expression_to_rels(PlannerInfo *root,
+ GroupedVarInfo *gvi, Index relid);
#endif /* CLAUSES_H */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 182ffeef4b..2d05ff5a15 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -210,6 +210,14 @@ extern AggPath *create_agg_path(PlannerInfo *root,
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+extern AggPath *create_agg_sorted_path(PlannerInfo *root,
+ RelOptInfo *rel,
+ Path *subpath,
+ RelAggInfo *agg_info);
+extern AggPath *create_agg_hashed_path(PlannerInfo *root,
+ RelOptInfo *rel,
+ Path *subpath,
+ RelAggInfo *agg_info);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
@@ -281,14 +289,21 @@ extern void setup_append_rel_array(PlannerInfo *root);
extern void expand_planner_arrays(PlannerInfo *root, int add_size);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
+extern RelOptInfo *build_simple_grouped_rel(PlannerInfo *root, int relid,
+ RelAggInfo **agg_info_p);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
+extern void add_grouped_rel(PlannerInfo *root, RelOptInfo *rel,
+ RelAggInfo *agg_info);
+extern RelOptInfo *find_grouped_rel(PlannerInfo *root, Relids relids,
+ RelAggInfo **agg_info_p);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr);
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info);
extern Relids min_join_parameterization(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
@@ -314,5 +329,5 @@ extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo, JoinType jointype);
-
+extern RelAggInfo *create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index 7345137d1d..3006ebf5aa 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -21,6 +21,7 @@
* allpaths.c
*/
extern PGDLLIMPORT bool enable_geqo;
+extern PGDLLIMPORT bool enable_agg_pushdown;
extern PGDLLIMPORT int geqo_threshold;
extern PGDLLIMPORT int min_parallel_table_scan_size;
extern PGDLLIMPORT int min_parallel_index_scan_size;
@@ -52,8 +53,14 @@ extern RelOptInfo *make_one_rel(PlannerInfo *root, List *joinlist);
extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
List *initial_rels);
+
extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
+extern void generate_grouping_paths(PlannerInfo *root,
+ RelOptInfo *rel_grouped,
+ RelOptInfo *rel_plain,
+ RelAggInfo *agg_info);
+
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages, int max_workers);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index e7aaddd50d..761f610a96 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -28,7 +28,8 @@ typedef void (*query_pathkeys_callback) (PlannerInfo *root, void *extra);
* prototypes for plan/planmain.c
*/
extern RelOptInfo *query_planner(PlannerInfo *root,
- query_pathkeys_callback qp_callback, void *qp_extra);
+ query_pathkeys_callback qp_callback, void *qp_extra,
+ RelOptInfo **final_rel_grouped_p);
/*
* prototypes for plan/planagg.c
@@ -69,6 +70,7 @@ extern void add_other_rels_to_query(PlannerInfo *root);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
Relids where_needed, bool create_new_ph);
+extern void setup_aggregate_pushdown(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
index 8a2378f968..e48f37bac2 100644
--- a/src/include/optimizer/tlist.h
+++ b/src/include/optimizer/tlist.h
@@ -50,8 +50,14 @@ extern void split_pathtarget_at_srfs(PlannerInfo *root,
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+/* TODO Find the best location (position and in some cases even file) for the
+ * following ones. */
+extern void add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions);
+extern bool is_grouping_expression(List *gvis, Expr *expr,
+ Index *sortgroupref, bool *is_derived);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
-
#endif /* TLIST_H */
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
new file mode 100644
index 0000000000..b3a97f86d6
--- /dev/null
+++ b/src/test/regress/expected/agg_pushdown.out
@@ -0,0 +1,217 @@
+BEGIN;
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+CREATE INDEX ON agg_pushdown_child1(parent);
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+COMMIT;
+ANALYZE;
+SET enable_agg_pushdown TO on;
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+--------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(10 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(11 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (p.i = c1.parent)
+ -> Sort
+ Sort Key: p.i
+ -> Seq Scan on agg_pushdown_parent p
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(12 rows)
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(8 rows)
+
+SET enable_seqscan TO on;
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.x = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(11 rows)
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: ((k = c1.j) AND (parent = c1.parent))
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(13 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+----------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Hash
+ -> Seq Scan on agg_pushdown_child2 c2
+(15 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: ((c1.j = c2.k) AND (c1.parent = c2.parent))
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(13 rows)
+
diff --git a/src/test/regress/expected/sysviews.out b/src/test/regress/expected/sysviews.out
index a1c90eb905..55cfd3fff7 100644
--- a/src/test/regress/expected/sysviews.out
+++ b/src/test/regress/expected/sysviews.out
@@ -72,6 +72,7 @@ select count(*) >= 0 as ok from pg_prepared_xacts;
select name, setting from pg_settings where name like 'enable%';
name | setting
--------------------------------+---------
+ enable_agg_pushdown | off
enable_bitmapscan | on
enable_gathermerge | on
enable_hashagg | on
@@ -89,7 +90,7 @@ select name, setting from pg_settings where name like 'enable%';
enable_seqscan | on
enable_sort | on
enable_tidscan | on
-(17 rows)
+(18 rows)
-- Test that the pg_timezone_names and pg_timezone_abbrevs views are
-- more-or-less working. We can't test their contents in any great detail
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index a39ca1012a..4f0ba643db 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -140,6 +140,7 @@ test: amutils
test: stats_ext
test: select_parallel
test: write_parallel
+test: agg_pushdown
test: publication
test: subscription
test: select_views
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
new file mode 100644
index 0000000000..6cf2ed9c20
--- /dev/null
+++ b/src/test/regress/sql/agg_pushdown.sql
@@ -0,0 +1,117 @@
+BEGIN;
+
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+
+CREATE INDEX ON agg_pushdown_child1(parent);
+
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+COMMIT;
+ANALYZE;
+
+SET enable_agg_pushdown TO on;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_seqscan TO on;
+
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
--
2.13.7
Re: [HACKERS] WIP: Aggregation push-down
This stuff seems very useful. How come it sits unreviewed for so long?
--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] WIP: Aggregation push-down
Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
This stuff seems very useful. How come it sits unreviewed for so long?
I think the review is hard for people who are not interested in the planner
very much. And as for further development, there are a few design decisions
that can hardly be resolved without Tom Lane's comments. Right now I recall
two problems: 1) is the way I currently store RelOptInfo for the grouped
relations correct?, 2) how should we handle types for which logical equality
does not imply physical (byte-wise) equality?
Fortunately it seems now that I'm not the only one who cares about 2), so this
problem might be resolved soon:
/messages/by-id/CAH2-Wzn3Ee49Gmxb7V1VJ3-AC8fWn-Fr8pfWQebHe8rYRxt5OQ@mail.gmail.com
But 1) still remains.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
Re: [HACKERS] WIP: Aggregation push-down
Hi,
I've been looking at the last version (v14) of this patch series,
submitted way back in July and unfortunately quiet since then. Antonin
is probably right one of the reasons for the lack of reviews is that it
requires interest/experience with planner.
Initially it was also a bit hard to understand what are the benefits
(and the patch shifted a bit), which is now mostly addressed by the
README in the last patch. The trouble is that's hidden in the patch and
so not immediately obvious to people considering reviewing it :-( Tom
posted a nice summary in November 2018, but it was perhaps focused on
the internals.
So my first suggestion it to write a short summary with example how it
affects practical queries (plan change, speedup) etc.
My second suggestion is to have meaningful commit messages for each part
of the patch series, explaining why we need that particular part. It
might have been explained somewhere in the thread, but as a reviewer I
really don't want to reverse-engineer the whole thread.
Now, regarding individual parts of the patch:
1) v14-0001-Introduce-RelInfoList-structure.patch
-------------------------------------------------
- I'm not entirely sure why we need this change. We had the list+hash
before, so I assume we do this because we need the output functions?
- The RelInfoList naming is pretty confusing, because it's actually not
a list but a combination of list+hash. And the embedded list is called
items, to make it yet a bit more confusing. I suggest we call this
just RelInfo or RelInfoLookup or something else not including List.
- RelInfoEntry seems severely under-documented, particularly the data
part (which is void* making it even harder to understand what's it for
without having to read the functions). AFAICS it's just simply a link
to the embedded list, but maybe I'm wrong.
- I suggest we move find_join_rel/add_rel_info/add_join_rel in relnode.c
right before build_join_rel. This makes diff clearer/smaller and
visual diffs nicer.
2) v14-0002-Introduce-make_join_rel_common-function.patch
---------------------------------------------------------
I see no point in keeping this change in a separate patch, as it prety
much just renames make_join_rel to make_join_rel_common and then adds
make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
{
return make_join_rel_common(root, rel1, rel2);
}
which seems rather trivial and useless on it's own. I'd just merge it
into 0003 where we use the _common function more extensively.
3) v14-0003-Aggregate-push-down-basic-functionality.patch
---------------------------------------------------------
I haven't done much review on this yet, but I've done some testing and
benchmarking so let me share at least that. The queries I used were of
the type I mentioned earlier in this thread - essentially a star schema,
i.e. fact table referencing dimensions, with aggregation per columns in
the dimension. So something like
SELECT d.c, sum(f) FROM fact JOIN dimension d ON (d.id = f.d_id)
GROUP BY d.c;
where "fact" table is much much larger than the dimension.
Attached is a script I used for testing with a bunch of queries and a
number of parameters (parallelism, size of dimension, size of fact, ...)
and a spreadsheed summarizing the results.
Overall, the results seem pretty good - in many cases the queries get
about 2x faster and I haven't seen any regressions. That's pretty nice.
One annoying thing is that this only works for non-parallel queries.
That is, I saw no improvements with parallel queries. It was still
faster than the non-parallel query with aggregate pushdown, but the
parallel query did not improve.
An example of timing looks like this:
non-parallel (pushdown = off): 918 ms
non-parallel (pushdown = on): 561 ms
parallel (pushdown = off): 313 ms
parallel (pushdown = on): 313 ms
The non-parallel query gets faster because after enabling push-down the
plan changes (and we get partial aggregate below the join). With
parallel query that does not happen, the plans stay the same.
I'm not claiming this is a bug - we end up picking the fastest execution
plan (i.e. we don't make a mistake by e.g. switching to the non-parallel
one with pushdown).
My understanding is that the aggregate pushdown can't (currently?) be
used with queries that use partial aggregate because of parallelism.
That is we can either end up with a plan like this:
Finalize Aggregate
-> Join
-> Partial Aggregate
-> ...
or a parallel plan like this:
Finalize Aggregate
-> Gather
-> Partial Aggregate
-> Join
-> ...
-> ...
but we currently don't support a combination of both
Finalize Aggregate
-> Gather
-> Join
-> Partial Aggregate
-> ...
I wonder if that's actually even possible and what would it take to make
it work?
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] WIP: Aggregation push-down
Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
Hi,
I've been looking at the last version (v14) of this patch series,
submitted way back in July and unfortunately quiet since then. Antonin
is probably right one of the reasons for the lack of reviews is that it
requires interest/experience with planner.Initially it was also a bit hard to understand what are the benefits
(and the patch shifted a bit), which is now mostly addressed by the
README in the last patch. The trouble is that's hidden in the patch and
so not immediately obvious to people considering reviewing it :-( Tom
posted a nice summary in November 2018, but it was perhaps focused on
the internals.So my first suggestion it to write a short summary with example how it
affects practical queries (plan change, speedup) etc.
I think README plus regression test output should be enough for someone who is
about to review patch as complex as this.
My second suggestion is to have meaningful commit messages for each part
of the patch series, explaining why we need that particular part. It
might have been explained somewhere in the thread, but as a reviewer I
really don't want to reverse-engineer the whole thread.
ok, done.
Now, regarding individual parts of the patch:
1) v14-0001-Introduce-RelInfoList-structure.patch
-------------------------------------------------- I'm not entirely sure why we need this change. We had the list+hash
before, so I assume we do this because we need the output functions?
I believe that this is what Tom proposed in [1]/messages/by-id/9726.1542577439@sss.pgh.pa.us, see "Maybe an appropriate
preliminary patch is to refactor the support code ..." near the end of that
message. The point is that now we need two lists: one for "plain" relations
and one for grouped ones.
- The RelInfoList naming is pretty confusing, because it's actually not
a list but a combination of list+hash. And the embedded list is called
items, to make it yet a bit more confusing. I suggest we call this
just RelInfo or RelInfoLookup or something else not including List.
I think it can be considered a list by the caller of add_join_rel() etc. The
hash table is hidden from user and is empty until the list becomes long
enough. The word "list" in the structure name may just indicate that new
elements can be added to the end, which shouldn't be expected if the structure
was an array.
I searched a bit in tools/pgindent/typedefs.list and found at least a few
structures that also do have "list" in the name but are not lists internally:
CatCList, FuncCandidateList, MCVList.
Actually I'm not opposed to renaming the structure, but don't have better idea
right now. As for *Lookup: following is the only use of such a structure name
in PG code and it's not something to store data in:
typedef enum
{
IDENTIFIER_LOOKUP_NORMAL, /* normal processing of var names */
IDENTIFIER_LOOKUP_DECLARE, /* In DECLARE --- don't look up names */
IDENTIFIER_LOOKUP_EXPR /* In SQL expression --- special case */
} IdentifierLookup;
- RelInfoEntry seems severely under-documented, particularly the data
part (which is void* making it even harder to understand what's it for
without having to read the functions). AFAICS it's just simply a link
to the embedded list, but maybe I'm wrong.
This is just JoinHashEntry (which was also undocumented) renamed. I've added a
short comment now.
- I suggest we move find_join_rel/add_rel_info/add_join_rel in relnode.c
right before build_join_rel. This makes diff clearer/smaller and
visual diffs nicer.
Hm, it might improve readability of the diff, but this API is exactly where I
still need feedback from Tom. I'm not eager to optimize the diff as long as
there's a risk that these functions will be removed or renamed.
2) v14-0002-Introduce-make_join_rel_common-function.patch
---------------------------------------------------------I see no point in keeping this change in a separate patch, as it prety
much just renames make_join_rel to make_join_rel_common and then addsmake_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
{
return make_join_rel_common(root, rel1, rel2);
}which seems rather trivial and useless on it's own. I'd just merge it
into 0003 where we use the _common function more extensively.
ok. I thought that this improves readability of the diffs, but it doesn't look
that bad if this is included in 0003.
3) v14-0003-Aggregate-push-down-basic-functionality.patch
---------------------------------------------------------I haven't done much review on this yet, but I've done some testing and
benchmarking so let me share at least that. The queries I used were of
the type I mentioned earlier in this thread - essentially a star schema,
i.e. fact table referencing dimensions, with aggregation per columns in
the dimension. So something likeSELECT d.c, sum(f) FROM fact JOIN dimension d ON (d.id = f.d_id)
GROUP BY d.c;where "fact" table is much much larger than the dimension.
Attached is a script I used for testing with a bunch of queries and a
number of parameters (parallelism, size of dimension, size of fact, ...)
and a spreadsheed summarizing the results.Overall, the results seem pretty good - in many cases the queries get
about 2x faster and I haven't seen any regressions. That's pretty nice.
Thanks for such an elaborate testing! The ultimate goal of this patch is to
improve sharding (using postgres_fdw), but it's nice to see significant
improvement even for local queries.
One annoying thing is that this only works for non-parallel queries.
That is, I saw no improvements with parallel queries. It was still
faster than the non-parallel query with aggregate pushdown, but the
parallel query did not improve.
An example of timing looks like this:
non-parallel (pushdown = off): 918 ms
non-parallel (pushdown = on): 561 ms
parallel (pushdown = off): 313 ms
parallel (pushdown = on): 313 msThe non-parallel query gets faster because after enabling push-down the
plan changes (and we get partial aggregate below the join). With
parallel query that does not happen, the plans stay the same.I'm not claiming this is a bug - we end up picking the fastest execution
plan (i.e. we don't make a mistake by e.g. switching to the non-parallel
one with pushdown).My understanding is that the aggregate pushdown can't (currently?) be
used with queries that use partial aggregate because of parallelism.
That is we can either end up with a plan like this:Finalize Aggregate
-> Join
-> Partial Aggregate
-> ...or a parallel plan like this:
Finalize Aggregate
-> Gather
-> Partial Aggregate
-> Join
-> ...
-> ...but we currently don't support a combination of both
Finalize Aggregate
-> Gather
-> Join
-> Partial Aggregate
-> ...I wonder if that's actually even possible and what would it take to make
it work?
I had a prototype of the feature that does affect parallel queries (as well as
partitioned tables and postgres_fdw), but didn't include these parts in the
recent patch versions. The point is that the API to store RelOptInfos can
still change and in such a case I'd have to rebase too much code.
v15 is attached. Actually there are no significant changes relative to v14,
but v14 does not apply to the current master:
error: patch failed: src/test/regress/serial_schedule:140
error: src/test/regress/serial_schedule: patch does not apply
This is interesting because no problem is currently reported at
http://commitfest.cputube.org/
[1]: /messages/by-id/9726.1542577439@sss.pgh.pa.us
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
Attachments:
v15-0001-Introduce-RelInfoList-structure.patchtext/x-diffDownload
From 4ea0789e84a26eb339b751e4c5bd67f410f8a512 Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Thu, 16 Jan 2020 16:02:02 +0100
Subject: [PATCH 1/2] Introduce RelInfoList structure.
This patch puts join_rel_list and join_rel_hash fields of PlannerInfo
structure into a new structure RelInfoList. It also adjusts add_join_rel() and
find_join_rel() functions so they only call add_rel_info() and find_rel_inf()
respectively. Thus it'll be easier to add a new list and accessor functions
that we'll need for grouped relation.
---
contrib/postgres_fdw/postgres_fdw.c | 3 +-
src/backend/nodes/outfuncs.c | 11 ++
src/backend/optimizer/geqo/geqo_eval.c | 12 +-
src/backend/optimizer/plan/planmain.c | 3 +-
src/backend/optimizer/util/relnode.c | 161 +++++++++++++++----------
src/include/nodes/nodes.h | 1 +
src/include/nodes/pathnodes.h | 28 +++--
7 files changed, 140 insertions(+), 79 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 2175dff824..07c70843e8 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -5251,7 +5251,8 @@ foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
*/
Assert(fpinfo->relation_index == 0); /* shouldn't be set yet */
fpinfo->relation_index =
- list_length(root->parse->rtable) + list_length(root->join_rel_list);
+ list_length(root->parse->rtable) +
+ list_length(root->join_rel_list->items);
return true;
}
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index d76fae44b8..e90b47a7a5 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2288,6 +2288,14 @@ _outRelOptInfo(StringInfo str, const RelOptInfo *node)
WRITE_NODE_FIELD(partitioned_child_rels);
}
+static void
+_outRelInfoList(StringInfo str, const RelInfoList *node)
+{
+ WRITE_NODE_TYPE("RELOPTINFOLIST");
+
+ WRITE_NODE_FIELD(items);
+}
+
static void
_outIndexOptInfo(StringInfo str, const IndexOptInfo *node)
{
@@ -4077,6 +4085,9 @@ outNode(StringInfo str, const void *obj)
case T_RelOptInfo:
_outRelOptInfo(str, obj);
break;
+ case T_RelInfoList:
+ _outRelInfoList(str, obj);
+ break;
case T_IndexOptInfo:
_outIndexOptInfo(str, obj);
break;
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index 6d897936d7..5f3209ac67 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -92,11 +92,11 @@ geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
*
* join_rel_level[] shouldn't be in use, so just Assert it isn't.
*/
- savelength = list_length(root->join_rel_list);
- savehash = root->join_rel_hash;
+ savelength = list_length(root->join_rel_list->items);
+ savehash = root->join_rel_list->hash;
Assert(root->join_rel_level == NULL);
- root->join_rel_hash = NULL;
+ root->join_rel_list->hash = NULL;
/* construct the best path for the given combination of relations */
joinrel = gimme_tree(root, tour, num_gene);
@@ -121,9 +121,9 @@ geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
* Restore join_rel_list to its former state, and put back original
* hashtable if any.
*/
- root->join_rel_list = list_truncate(root->join_rel_list,
- savelength);
- root->join_rel_hash = savehash;
+ root->join_rel_list->items = list_truncate(root->join_rel_list->items,
+ savelength);
+ root->join_rel_list->hash = savehash;
/* release all the memory acquired within gimme_tree */
MemoryContextSwitchTo(oldcxt);
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 62dfc6d44a..5fa33ec200 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -65,8 +65,7 @@ query_planner(PlannerInfo *root,
* NOTE: append_rel_list was set up by subquery_planner, so do not touch
* here.
*/
- root->join_rel_list = NIL;
- root->join_rel_hash = NULL;
+ root->join_rel_list = makeNode(RelInfoList);
root->join_rel_level = NULL;
root->join_cur_level = 0;
root->canon_pathkeys = NIL;
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 374f93890b..ccd2a8bd33 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -31,11 +31,15 @@
#include "utils/hsearch.h"
-typedef struct JoinHashEntry
+/*
+ * An entry of a hash table that we use to make lookup for RelOptInfo
+ * structures more efficient.
+ */
+typedef struct RelInfoEntry
{
- Relids join_relids; /* hash key --- MUST BE FIRST */
- RelOptInfo *join_rel;
-} JoinHashEntry;
+ Relids relids; /* hash key --- MUST BE FIRST */
+ void *data;
+} RelInfoEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *input_rel);
@@ -379,11 +383,11 @@ find_base_rel(PlannerInfo *root, int relid)
}
/*
- * build_join_rel_hash
- * Construct the auxiliary hash table for join relations.
+ * build_rel_hash
+ * Construct the auxiliary hash table for relation specific data.
*/
static void
-build_join_rel_hash(PlannerInfo *root)
+build_rel_hash(RelInfoList *list)
{
HTAB *hashtab;
HASHCTL hash_ctl;
@@ -392,47 +396,50 @@ build_join_rel_hash(PlannerInfo *root)
/* Create the hash table */
MemSet(&hash_ctl, 0, sizeof(hash_ctl));
hash_ctl.keysize = sizeof(Relids);
- hash_ctl.entrysize = sizeof(JoinHashEntry);
+ hash_ctl.entrysize = sizeof(RelInfoEntry);
hash_ctl.hash = bitmap_hash;
hash_ctl.match = bitmap_match;
hash_ctl.hcxt = CurrentMemoryContext;
- hashtab = hash_create("JoinRelHashTable",
+ hashtab = hash_create("RelHashTable",
256L,
&hash_ctl,
HASH_ELEM | HASH_FUNCTION | HASH_COMPARE | HASH_CONTEXT);
/* Insert all the already-existing joinrels */
- foreach(l, root->join_rel_list)
+ foreach(l, list->items)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
- JoinHashEntry *hentry;
+ void *item = lfirst(l);
+ RelInfoEntry *hentry;
bool found;
+ Relids relids;
- hentry = (JoinHashEntry *) hash_search(hashtab,
- &(rel->relids),
- HASH_ENTER,
- &found);
+ Assert(IsA(item, RelOptInfo));
+ relids = ((RelOptInfo *) item)->relids;
+
+ hentry = (RelInfoEntry *) hash_search(hashtab,
+ &relids,
+ HASH_ENTER,
+ &found);
Assert(!found);
- hentry->join_rel = rel;
+ hentry->data = item;
}
- root->join_rel_hash = hashtab;
+ list->hash = hashtab;
}
/*
- * find_join_rel
- * Returns relation entry corresponding to 'relids' (a set of RT indexes),
- * or NULL if none exists. This is for join relations.
+ * find_rel_info
+ * Find a base or join relation entry.
*/
-RelOptInfo *
-find_join_rel(PlannerInfo *root, Relids relids)
+static void *
+find_rel_info(RelInfoList *list, Relids relids)
{
/*
* Switch to using hash lookup when list grows "too long". The threshold
* is arbitrary and is known only here.
*/
- if (!root->join_rel_hash && list_length(root->join_rel_list) > 32)
- build_join_rel_hash(root);
+ if (!list->hash && list_length(list->items) > 32)
+ build_rel_hash(list);
/*
* Use either hashtable lookup or linear search, as appropriate.
@@ -442,34 +449,90 @@ find_join_rel(PlannerInfo *root, Relids relids)
* so would force relids out of a register and thus probably slow down the
* list-search case.
*/
- if (root->join_rel_hash)
+ if (list->hash)
{
Relids hashkey = relids;
- JoinHashEntry *hentry;
+ RelInfoEntry *hentry;
- hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
- &hashkey,
- HASH_FIND,
- NULL);
+ hentry = (RelInfoEntry *) hash_search(list->hash,
+ &hashkey,
+ HASH_FIND,
+ NULL);
if (hentry)
- return hentry->join_rel;
+ return hentry->data;
}
else
{
ListCell *l;
- foreach(l, root->join_rel_list)
+ foreach(l, list->items)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
+ void *item = lfirst(l);
+ Relids item_relids;
- if (bms_equal(rel->relids, relids))
- return rel;
+ Assert(IsA(item, RelOptInfo));
+ item_relids = ((RelOptInfo *) item)->relids;
+
+ if (bms_equal(item_relids, relids))
+ return item;
}
}
return NULL;
}
+/*
+ * find_join_rel
+ * Returns relation entry corresponding to 'relids' (a set of RT indexes),
+ * or NULL if none exists. This is for join relations.
+ */
+RelOptInfo *
+find_join_rel(PlannerInfo *root, Relids relids)
+{
+ return (RelOptInfo *) find_rel_info(root->join_rel_list, relids);
+}
+
+/*
+ * add_rel_info
+ * Add relation specific info to a list, and also add it to the auxiliary
+ * hashtable if there is one.
+ */
+static void
+add_rel_info(RelInfoList *list, void *data)
+{
+ Assert(IsA(data, RelOptInfo));
+
+ /* GEQO requires us to append the new joinrel to the end of the list! */
+ list->items = lappend(list->items, data);
+
+ /* store it into the auxiliary hashtable if there is one. */
+ if (list->hash)
+ {
+ Relids relids;
+ RelInfoEntry *hentry;
+ bool found;
+
+ relids = ((RelOptInfo *) data)->relids;
+ hentry = (RelInfoEntry *) hash_search(list->hash,
+ &relids,
+ HASH_ENTER,
+ &found);
+ Assert(!found);
+ hentry->data = data;
+ }
+}
+
+/*
+ * add_join_rel
+ * Add given join relation to the list of join relations in the given
+ * PlannerInfo.
+ */
+static void
+add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
+{
+ add_rel_info(root->join_rel_list, joinrel);
+}
+
/*
* set_foreign_rel_properties
* Set up foreign-join fields if outer and inner relation are foreign
@@ -520,32 +583,6 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
}
}
-/*
- * add_join_rel
- * Add given join relation to the list of join relations in the given
- * PlannerInfo. Also add it to the auxiliary hashtable if there is one.
- */
-static void
-add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
-{
- /* GEQO requires us to append the new joinrel to the end of the list! */
- root->join_rel_list = lappend(root->join_rel_list, joinrel);
-
- /* store it into the auxiliary hashtable if there is one. */
- if (root->join_rel_hash)
- {
- JoinHashEntry *hentry;
- bool found;
-
- hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
- &(joinrel->relids),
- HASH_ENTER,
- &found);
- Assert(!found);
- hentry->join_rel = joinrel;
- }
-}
-
/*
* build_join_rel
* Returns relation entry corresponding to the union of two given rels,
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index baced7eec0..316c1ecbb9 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -220,6 +220,7 @@ typedef enum NodeTag
T_PlannerInfo,
T_PlannerGlobal,
T_RelOptInfo,
+ T_RelInfoList,
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 3d3be197e0..aca7993af3 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -236,15 +236,9 @@ struct PlannerInfo
/*
* join_rel_list is a list of all join-relation RelOptInfos we have
- * considered in this planning run. For small problems we just scan the
- * list to do lookups, but when there are many join relations we build a
- * hash table for faster lookups. The hash table is present and valid
- * when join_rel_hash is not NULL. Note that we still maintain the list
- * even when using the hash table for lookups; this simplifies life for
- * GEQO.
+ * considered in this planning run.
*/
- List *join_rel_list; /* list of join-relation RelOptInfos */
- struct HTAB *join_rel_hash; /* optional hashtable for join relations */
+ struct RelInfoList *join_rel_list; /* list of join-relation RelOptInfos */
/*
* When doing a dynamic-programming-style join search, join_rel_level[k]
@@ -748,6 +742,24 @@ typedef struct RelOptInfo
((rel)->part_scheme && (rel)->boundinfo && (rel)->nparts > 0 && \
(rel)->part_rels && (rel)->partexprs && (rel)->nullable_partexprs)
+/*
+ * RelInfoList
+ * A list to store relation specific info and to retrieve it by relids.
+ *
+ * For small problems we just scan the list to do lookups, but when there are
+ * many relations we build a hash table for faster lookups. The hash table is
+ * present and valid when rel_hash is not NULL. Note that we still maintain
+ * the list even when using the hash table for lookups; this simplifies life
+ * for GEQO.
+ */
+typedef struct RelInfoList
+{
+ NodeTag type;
+
+ List *items;
+ struct HTAB *hash;
+} RelInfoList;
+
/*
* IndexOptInfo
* Per-index information for planning/optimization
--
2.20.1
v15-0002-Aggregate-push-down-basic-functionality.patchtext/x-diffDownload
From a2ee1ca8b95216461426a3e0f002adb6fe3bb1ce Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Thu, 16 Jan 2020 16:02:02 +0100
Subject: [PATCH 2/2] Aggregate push-down - basic functionality.
With this patch, partial aggregation can be applied to a base relation or to a
join, and the resulting "grouped" relations can be joined to other "plain"
relations. Once all tables are joined, the aggregation is finalized. See
README for more information.
The next patches will enable the aggregate push-down feature for parallel
query processing, for partitioned tables and for foreign tables.
---
src/backend/nodes/copyfuncs.c | 21 +-
src/backend/nodes/outfuncs.c | 36 +
src/backend/optimizer/README | 87 ++
src/backend/optimizer/path/allpaths.c | 112 +++
src/backend/optimizer/path/costsize.c | 17 +-
src/backend/optimizer/path/equivclass.c | 134 +++
src/backend/optimizer/path/joinrels.c | 195 ++++-
src/backend/optimizer/plan/initsplan.c | 257 ++++++
src/backend/optimizer/plan/planagg.c | 2 +-
src/backend/optimizer/plan/planmain.c | 23 +-
src/backend/optimizer/plan/planner.c | 58 +-
src/backend/optimizer/plan/setrefs.c | 33 +
src/backend/optimizer/prep/prepjointree.c | 1 +
src/backend/optimizer/util/pathnode.c | 143 +++-
src/backend/optimizer/util/relnode.c | 916 ++++++++++++++++++++-
src/backend/optimizer/util/tlist.c | 56 ++
src/backend/utils/misc/guc.c | 9 +
src/include/nodes/nodes.h | 2 +
src/include/nodes/pathnodes.h | 99 +++
src/include/optimizer/clauses.h | 2 +
src/include/optimizer/pathnode.h | 19 +-
src/include/optimizer/paths.h | 7 +
src/include/optimizer/planmain.h | 4 +-
src/include/optimizer/tlist.h | 8 +-
src/test/regress/expected/agg_pushdown.out | 217 +++++
src/test/regress/expected/sysviews.out | 3 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/agg_pushdown.sql | 117 +++
28 files changed, 2498 insertions(+), 81 deletions(-)
create mode 100644 src/test/regress/expected/agg_pushdown.out
create mode 100644 src/test/regress/sql/agg_pushdown.sql
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 54ad62bb7f..c7712166f0 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2211,8 +2211,8 @@ _copyOnConflictExpr(const OnConflictExpr *from)
/* ****************************************************************
* pathnodes.h copy functions
*
- * We don't support copying RelOptInfo, IndexOptInfo, or Path nodes.
- * There are some subsidiary structs that are useful to copy, though.
+ * We don't support copying RelOptInfo, IndexOptInfo, RelAggInfo or Path
+ * nodes. There are some subsidiary structs that are useful to copy, though.
* ****************************************************************
*/
@@ -2355,6 +2355,20 @@ _copyPlaceHolderInfo(const PlaceHolderInfo *from)
return newnode;
}
+static GroupedVarInfo *
+_copyGroupedVarInfo(const GroupedVarInfo *from)
+{
+ GroupedVarInfo *newnode = makeNode(GroupedVarInfo);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gv_eval_at);
+ COPY_SCALAR_FIELD(derived);
+
+ return newnode;
+}
+
/* ****************************************************************
* parsenodes.h copy functions
* ****************************************************************
@@ -5107,6 +5121,9 @@ copyObjectImpl(const void *from)
case T_PlaceHolderInfo:
retval = _copyPlaceHolderInfo(from);
break;
+ case T_GroupedVarInfo:
+ retval = _copyGroupedVarInfo(from);
+ break;
/*
* VALUE NODES
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index e90b47a7a5..575f0cbf58 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2198,6 +2198,8 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_BITMAPSET_FIELD(all_baserels);
WRITE_BITMAPSET_FIELD(nullable_baserels);
WRITE_NODE_FIELD(join_rel_list);
+ WRITE_NODE_FIELD(grouped_rel_list);
+ WRITE_NODE_FIELD(agg_info_list);
WRITE_INT_FIELD(join_cur_level);
WRITE_NODE_FIELD(init_plans);
WRITE_NODE_FIELD(cte_plan_ids);
@@ -2212,6 +2214,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(append_rel_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
@@ -2219,6 +2222,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(distinct_pathkeys);
WRITE_NODE_FIELD(sort_pathkeys);
WRITE_NODE_FIELD(processed_tlist);
+ WRITE_INT_FIELD(max_sortgroupref);
WRITE_NODE_FIELD(minmax_aggs);
WRITE_FLOAT_FIELD(total_table_pages, "%.0f");
WRITE_FLOAT_FIELD(tuple_fraction, "%.4f");
@@ -2438,6 +2442,20 @@ _outParamPathInfo(StringInfo str, const ParamPathInfo *node)
WRITE_NODE_FIELD(ppi_clauses);
}
+static void
+_outRelAggInfo(StringInfo str, const RelAggInfo *node)
+{
+ WRITE_NODE_TYPE("RELAGGINFO");
+
+ WRITE_BITMAPSET_FIELD(relids);
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(agg_input);
+ WRITE_FLOAT_FIELD(input_rows, "%.0f");
+ WRITE_NODE_FIELD(group_clauses);
+ WRITE_NODE_FIELD(group_exprs);
+ WRITE_NODE_FIELD(agg_exprs);
+}
+
static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
@@ -2539,6 +2557,18 @@ _outPlaceHolderInfo(StringInfo str, const PlaceHolderInfo *node)
WRITE_INT_FIELD(ph_width);
}
+static void
+_outGroupedVarInfo(StringInfo str, const GroupedVarInfo *node)
+{
+ WRITE_NODE_TYPE("GROUPEDVARINFO");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_BITMAPSET_FIELD(gv_eval_at);
+ WRITE_BOOL_FIELD(derived);
+}
+
static void
_outMinMaxAggInfo(StringInfo str, const MinMaxAggInfo *node)
{
@@ -4109,6 +4139,9 @@ outNode(StringInfo str, const void *obj)
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_RelAggInfo:
+ _outRelAggInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
@@ -4127,6 +4160,9 @@ outNode(StringInfo str, const void *obj)
case T_PlaceHolderInfo:
_outPlaceHolderInfo(str, obj);
break;
+ case T_GroupedVarInfo:
+ _outGroupedVarInfo(str, obj);
+ break;
case T_MinMaxAggInfo:
_outMinMaxAggInfo(str, obj);
break;
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
index 89ce373d5e..f48e5657e1 100644
--- a/src/backend/optimizer/README
+++ b/src/backend/optimizer/README
@@ -1127,3 +1127,90 @@ breaking down aggregation or grouping over a partitioned relation into
aggregation or grouping over its partitions is called partitionwise
aggregation. Especially when the partition keys match the GROUP BY clause,
this can be significantly faster than the regular method.
+
+Aggregate push-down
+-------------------
+
+The obvious way to evaluate aggregates is to evaluate the FROM clause of the
+SQL query (this is what query_planner does) and use the resulting paths as the
+input of Agg node. However, if the groups are large enough, it may be more
+efficient to apply the partial aggregation to the output of base relation
+scan, and finalize it when we have all relations of the query joined:
+
+ EXPLAIN
+ SELECT a.i, avg(b.u)
+ FROM a JOIN b ON b.j = a.i
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+ -> Index Only Scan using a_pkey on a
+ Index Cond: (i = b.j)
+
+Thus the join above the partial aggregate node receives fewer input rows, and
+so the number of outer-to-inner pairs of tuples to be checked can be
+significantly lower, which can in turn lead to considerably lower join cost.
+
+Note that there's often no GROUP BY expression to be used for the partial
+aggregation, so we use equivalence classes to derive grouping expression: in
+the example above, the grouping key "b.j" was derived from "a.i".
+
+Furthermore, extra grouping columns can be added to the partial Agg node if a
+join clause above that node references a column which is not in the query
+GROUP BY clause and which could not be derived using equivalence class.
+
+ EXPLAIN
+ SELECT a.i, avg(b.u)
+ FROM a JOIN b ON b.j = a.x
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (a.x = b.j)
+ -> Seq Scan on a
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+
+Here the partial aggregate uses "b.j" as grouping column although it's not in
+the same equivalence class as "a.i". Note that no column of the {a.x, b.j}
+equivalence class is used as a key for the final aggregation.
+
+Besides base relation, the aggregation can also be pushed down to join:
+
+ EXPLAIN
+ SELECT a.i, avg(b.u + c.v)
+ FROM a JOIN b ON b.j = a.i
+ JOIN c ON c.k = a.i
+ WHERE b.j = c.k GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (b.j = a.i)
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Hash Join
+ Hash Cond: (b.j = c.k)
+ -> Seq Scan on b
+ -> Hash
+ -> Seq Scan on c
+ -> Hash
+ -> Seq Scan on a
+
+Whether the Agg node is created out of base relation or out of join, it's
+added to a separate RelOptInfo that we call "grouped relation". Grouped
+relation can be joined to a non-grouped relation, which results in a grouped
+relation too. Join of two grouped relations does not seem to be very useful
+and is currently not supported.
+
+If query_planner produces a grouped relation that contains valid paths, these
+are simply added to the UPPERREL_PARTIAL_GROUP_AGG relation. Further
+processing of these paths then does not differ from processing of other
+partially grouped paths.
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 8286d9cf34..4445ec2504 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -60,6 +60,7 @@ typedef struct pushdown_safety_info
/* These parameters are set by GUC */
bool enable_geqo = false; /* just in case GUC doesn't set it */
+bool enable_agg_pushdown;
int geqo_threshold;
int min_parallel_table_scan_size;
int min_parallel_index_scan_size;
@@ -124,6 +125,9 @@ static void set_result_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
+static void add_grouped_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, AggStrategy aggstrategy,
+ RelAggInfo *agg_info);
static RelOptInfo *make_rel_from_joinlist(PlannerInfo *root, List *joinlist);
static bool subquery_is_pushdown_safe(Query *subquery, Query *topquery,
pushdown_safety_info *safetyInfo);
@@ -2727,6 +2731,80 @@ generate_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_rows)
}
}
+/*
+ * generate_grouping_paths
+ * Create partially aggregated paths and add them to grouped relation.
+ *
+ * "rel_plain" is base or join relation whose paths are not grouped.
+ */
+void
+generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
+ RelOptInfo *rel_plain, RelAggInfo *agg_info)
+{
+ ListCell *lc;
+
+ if (IS_DUMMY_REL(rel_plain))
+ {
+ mark_dummy_rel(rel_grouped);
+ return;
+ }
+
+ foreach(lc, rel_plain->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ /*
+ * Since the path originates from the non-grouped relation which is
+ * not aware of the aggregate push-down, we must ensure that it
+ * provides the correct input for aggregation.
+ */
+ path = (Path *) create_projection_path(root, rel_grouped, path,
+ agg_info->agg_input);
+
+ /*
+ * add_grouped_path() will check whether the path has suitable
+ * pathkeys.
+ */
+ add_grouped_path(root, rel_grouped, path, AGG_SORTED, agg_info);
+
+ /*
+ * Repeated creation of hash table (for new parameter values) should
+ * be possible, does not sound like a good idea in terms of
+ * efficiency.
+ */
+ if (path->param_info == NULL)
+ add_grouped_path(root, rel_grouped, path, AGG_HASHED, agg_info);
+ }
+
+ /* Could not generate any grouped paths? */
+ if (rel_grouped->pathlist == NIL)
+ mark_dummy_rel(rel_grouped);
+}
+
+/*
+ * Apply partial aggregation to a subpath and add the AggPath to the pathlist.
+ */
+static void
+add_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ AggStrategy aggstrategy, RelAggInfo *agg_info)
+{
+ Path *agg_path;
+
+
+ if (aggstrategy == AGG_HASHED)
+ agg_path = (Path *) create_agg_hashed_path(root, rel, subpath,
+ agg_info);
+ else if (aggstrategy == AGG_SORTED)
+ agg_path = (Path *) create_agg_sorted_path(root, rel, subpath,
+ agg_info);
+ else
+ elog(ERROR, "unexpected strategy %d", aggstrategy);
+
+ /* Add the grouped path to the list of grouped base paths. */
+ if (agg_path != NULL)
+ add_path(rel, (Path *) agg_path);
+}
+
/*
* make_rel_from_joinlist
* Build access paths using a "joinlist" to guide the join path search.
@@ -2767,6 +2845,34 @@ make_rel_from_joinlist(PlannerInfo *root, List *joinlist)
int varno = ((RangeTblRef *) jlnode)->rtindex;
thisrel = find_base_rel(root, varno);
+
+ /*
+ * Create a grouped relation to facilitate aggregate push-down.
+ * This makes no sense if thisrel is the only relation of the
+ * query.
+ */
+ if (bms_nonempty_difference(root->all_baserels, thisrel->relids))
+ {
+ RelOptInfo *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /*
+ * Build grouped relation if thisrel is suitable for partial
+ * aggregation.
+ */
+ rel_grouped = build_simple_grouped_rel(root, varno, &agg_info);
+
+ if (rel_grouped)
+ {
+ /* Make the relation available for joining. */
+ add_grouped_rel(root, rel_grouped, agg_info);
+
+ /* Add the aggregation paths to it. */
+ generate_grouping_paths(root, rel_grouped, thisrel,
+ agg_info);
+ set_cheapest(rel_grouped);
+ }
+ }
}
else if (IsA(jlnode, List))
{
@@ -2868,6 +2974,7 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
for (lev = 2; lev <= levels_needed; lev++)
{
+ RelOptInfo *rel_grouped;
ListCell *lc;
/*
@@ -2904,6 +3011,11 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
+ /* The same for grouped relation if one exists. */
+ rel_grouped = find_grouped_rel(root, rel->relids, NULL);
+ if (rel_grouped)
+ set_cheapest(rel_grouped);
+
#ifdef OPTIMIZER_DEBUG
debug_print_rel(root, rel);
#endif
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index b5a0033721..6d25c2f581 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4414,7 +4414,6 @@ set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel)
0,
JOIN_INNER,
NULL);
-
rel->rows = clamp_row_est(nrows);
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
@@ -5385,11 +5384,11 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
foreach(lc, target->exprs)
{
Node *node = (Node *) lfirst(lc);
+ int32 item_width;
if (IsA(node, Var))
{
Var *var = (Var *) node;
- int32 item_width;
/* We should not see any upper-level Vars here */
Assert(var->varlevelsup == 0);
@@ -5420,6 +5419,20 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
Assert(item_width > 0);
tuple_width += item_width;
}
+ else if (IsA(node, Aggref))
+ {
+ /*
+ * If the target is evaluated by AggPath, it'll care of cost
+ * estimate. If the target is above AggPath (typically target of a
+ * join relation that contains grouped relation), the cost of
+ * Aggref should not be accounted for again.
+ *
+ * On the other hand, width is always needed.
+ */
+ item_width = get_typavgwidth(exprType(node), exprTypmod(node));
+ Assert(item_width > 0);
+ tuple_width += item_width;
+ }
else
{
/*
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index 4ef12547ee..cd52b7e7ac 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -2813,6 +2813,140 @@ is_redundant_derived_clause(RestrictInfo *rinfo, List *clauselist)
return false;
}
+/*
+ * translate_expression_to_rels
+ * If the appropriate equivalence classes exist, replace vars in
+ * gvi->gvexpr with vars whose varno is equal to relid. Return NULL if
+ * translation is not possible or needed.
+ *
+ * Note: Currently we only translate Var expressions. This is subject to
+ * change as the aggregate push-down feature gets enhanced.
+ */
+GroupedVarInfo *
+translate_expression_to_rels(PlannerInfo *root, GroupedVarInfo *gvi,
+ Index relid)
+{
+ Var *var;
+ ListCell *l1;
+ bool found_orig = false;
+ Var *var_translated = NULL;
+ GroupedVarInfo *result;
+
+ /* Can't do anything w/o equivalence classes. */
+ if (root->eq_classes == NIL)
+ return NULL;
+
+ var = castNode(Var, gvi->gvexpr);
+
+ /*
+ * Do we need to translate the var?
+ */
+ if (var->varno == relid)
+ return NULL;
+
+ /*
+ * Find the replacement var.
+ */
+ foreach(l1, root->eq_classes)
+ {
+ EquivalenceClass *ec = lfirst_node(EquivalenceClass, l1);
+ ListCell *l2;
+
+ /* TODO Check if any other EC kind should be ignored. */
+ if (ec->ec_has_volatile || ec->ec_below_outer_join || ec->ec_broken)
+ continue;
+
+ /* Single-element EC can hardly help in translations. */
+ if (list_length(ec->ec_members) == 1)
+ continue;
+
+ /*
+ * Collect all vars of this EC and their varnos.
+ *
+ * ec->ec_relids does not help because we're only interested in a
+ * subset of EC members.
+ */
+ foreach(l2, ec->ec_members)
+ {
+ EquivalenceMember *em = lfirst_node(EquivalenceMember, l2);
+ Var *ec_var;
+
+ /*
+ * The grouping expressions derived here are used to evaluate
+ * possibility to push aggregation down to RELOPT_BASEREL or
+ * RELOPT_JOINREL relations, and to construct reltargets for the
+ * grouped rels. We're not interested at the moment whether the
+ * relations do have children.
+ */
+ if (em->em_is_child)
+ continue;
+
+ if (!IsA(em->em_expr, Var))
+ continue;
+
+ ec_var = castNode(Var, em->em_expr);
+ if (equal(ec_var, var))
+ found_orig = true;
+ else if (ec_var->varno == relid)
+ var_translated = ec_var;
+
+ if (found_orig && var_translated)
+ {
+ /*
+ * The replacement Var must have the same data type, otherwise
+ * the values are not guaranteed to be grouped in the same way
+ * as values of the original Var.
+ */
+ if (ec_var->vartype != var->vartype)
+ return NULL;
+
+ break;
+ }
+ }
+
+ if (found_orig)
+ {
+ /*
+ * The same expression probably does not exist in multiple ECs.
+ */
+ if (var_translated == NULL)
+ {
+ /*
+ * Failed to translate the expression.
+ */
+ return NULL;
+ }
+ else
+ {
+ /* Success. */
+ break;
+ }
+ }
+ else
+ {
+ /*
+ * Vars of the requested relid can be in the next ECs too.
+ */
+ var_translated = NULL;
+ }
+ }
+
+ if (!found_orig)
+ return NULL;
+
+ result = makeNode(GroupedVarInfo);
+ memcpy(result, gvi, sizeof(GroupedVarInfo));
+
+ /*
+ * translate_expression_to_rels_mutator updates gv_eval_at.
+ */
+ result->gv_eval_at = bms_make_singleton(relid);
+ result->gvexpr = (Expr *) var_translated;
+ result->derived = true;
+
+ return result;
+}
+
/*
* is_redundant_with_indexclauses
* Test whether rinfo is redundant with any clause in the IndexClause
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index a21c295b99..6a56c7cc88 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -22,6 +22,7 @@
#include "partitioning/partbounds.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
+#include "utils/selfuncs.h"
static void make_rels_by_clause_joins(PlannerInfo *root,
@@ -36,6 +37,10 @@ static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
+static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1,
+ RelOptInfo *rel2,
+ RelAggInfo *agg_info,
+ RelOptInfo *rel_agg_input);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist);
@@ -665,21 +670,20 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
return true;
}
-
/*
- * make_join_rel
- * Find or create a join RelOptInfo that represents the join of
- * the two given rels, and add to it path information for paths
- * created with the two rels as outer and inner rel.
- * (The join rel may already contain paths generated from other
- * pairs of rels that add up to the same set of base rels.)
+ * make_join_rel_common
+ * The workhorse of make_join_rel().
+ *
+ * 'agg_info' contains the reltarget of grouped relation and everything we
+ * need to aggregate the join result. If NULL, then the join relation should
+ * not be grouped.
*
- * NB: will return NULL if attempted join is not valid. This can happen
- * when working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
+ * 'rel_agg_input' describes the AggPath input relation if the join output
+ * should be aggregated. If NULL is passed, do not aggregate the join output.
*/
-RelOptInfo *
-make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+static RelOptInfo *
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ RelAggInfo *agg_info, RelOptInfo *rel_agg_input)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -740,7 +744,7 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* goes with this particular joining.
*/
joinrel = build_join_rel(root, joinrelids, rel1, rel2, sjinfo,
- &restrictlist);
+ &restrictlist, agg_info);
/*
* If we've already proven this join is empty, we needn't consider any
@@ -753,14 +757,175 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
}
/* Add paths to the join relation. */
- populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
- restrictlist);
+ if (rel_agg_input == NULL)
+ {
+ /*
+ * Simply join the input relations, whether both are plain or one of
+ * them is grouped.
+ */
+ populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
+ restrictlist);
+ }
+ else
+ {
+ /* The join relation is grouped. */
+ Assert(agg_info != NULL);
+
+ /*
+ * Apply partial aggregation to the paths of rel_agg_input and add the
+ * resulting paths to joinrel.
+ */
+ generate_grouping_paths(root, joinrel, rel_agg_input, agg_info);
+ }
bms_free(joinrelids);
return joinrel;
}
+/*
+ * make_join_rel_combined
+ * Join grouped relation to non-grouped one.
+ */
+static void
+make_join_rel_combined(PlannerInfo *root, RelOptInfo *rel1,
+ RelOptInfo *rel2,
+ RelAggInfo *agg_info)
+{
+ RelOptInfo *rel1_grouped;
+ RelOptInfo *rel2_grouped;
+ bool rel1_grouped_useful = false;
+ bool rel2_grouped_useful = false;
+
+ /* Retrieve the grouped relations. */
+ rel1_grouped = find_grouped_rel(root, rel1->relids, NULL);
+ rel2_grouped = find_grouped_rel(root, rel2->relids, NULL);
+
+ /*
+ * Dummy rel may indicate a join relation that is able to generate grouped
+ * paths as such (i.e. it has valid agg_info), but for which the path
+ * actually could not be created (e.g. only AGG_HASHED strategy was
+ * possible but work_mem was not sufficient for hash table).
+ */
+ rel1_grouped_useful = rel1_grouped != NULL && !IS_DUMMY_REL(rel1_grouped);
+ rel2_grouped_useful = rel2_grouped != NULL && !IS_DUMMY_REL(rel2_grouped);
+
+ /* Nothing to do if there's no grouped relation. */
+ if (!rel1_grouped_useful && !rel2_grouped_useful)
+ return;
+
+ /*
+ * At maximum one input rel can be grouped (here we don't care if any rel
+ * is eventually dummy, the existence of grouped rel indicates that
+ * aggregates can be pushed down to it). If both were grouped, then
+ * grouping of one side would change the occurrence of the other side's
+ * aggregate transient states on the input of the final aggregation. This
+ * can be handled by adjusting the transient states, but it's not worth
+ * the effort because it's hard to find a use case for this kind of join.
+ *
+ * XXX If the join of two grouped rels is implemented someday, note that
+ * both rels can have aggregates, so it'd be hard to join grouped rel to
+ * non-grouped here: 1) such a "mixed join" would require a special
+ * target, 2) both AGGSPLIT_FINAL_DESERIAL and AGGSPLIT_SIMPLE aggregates
+ * could appear in the target of the final aggregation node, originating
+ * from the grouped and the non-grouped input rel respectively.
+ */
+ if (rel1_grouped && rel2_grouped)
+ return;
+
+ if (rel1_grouped_useful)
+ make_join_rel_common(root, rel1_grouped, rel2, agg_info, NULL);
+ else if (rel2_grouped_useful)
+ make_join_rel_common(root, rel1, rel2_grouped, agg_info, NULL);
+}
+
+/*
+ * make_join_rel
+ * Find or create a join RelOptInfo that represents the join of
+ * the two given rels, and add to it path information for paths
+ * created with the two rels as outer and inner rel.
+ * (The join rel may already contain paths generated from other
+ * pairs of rels that add up to the same set of base rels.)
+ *
+ * In addition to creating an ordinary join relation, try to create a
+ * grouped one. There are two strategies to achieve that: join a grouped
+ * relation to plain one, or join two plain relations and apply partial
+ * aggregation to the result.
+ *
+ * NB: will return NULL if attempted join is not valid. This can happen when
+ * working with outer joins, or with IN or EXISTS clauses that have been
+ * turned into joins. Besides that, NULL is also returned if caller is
+ * interested in a grouped relation but it could not be created.
+ *
+ * Only the plain relation is returned; if grouped relation exists, it can be
+ * retrieved using find_grouped_rel().
+ */
+RelOptInfo *
+make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+{
+ Relids joinrelids;
+ RelAggInfo *agg_info = NULL;
+ RelOptInfo *joinrel,
+ *joinrel_plain;
+
+ /* 1) form the plain join. */
+ joinrel = make_join_rel_common(root, rel1, rel2, NULL, NULL);
+ joinrel_plain = joinrel;
+
+ if (joinrel_plain == NULL)
+ return joinrel_plain;
+
+ /*
+ * We're done if there are no grouping expressions nor aggregates.
+ */
+ if (root->grouped_var_list == NIL)
+ return joinrel_plain;
+
+ joinrelids = bms_union(rel1->relids, rel2->relids);
+ joinrel = find_grouped_rel(root, joinrelids, &agg_info);
+
+ if (joinrel != NULL)
+ {
+ /*
+ * If the same grouped joinrel was already formed, just with the base
+ * rels divided between rel1 and rel2 in a different way, the matching
+ * agg_info should already be there.
+ */
+ Assert(agg_info != NULL);
+ }
+ else
+ {
+ /*
+ * agg_info must be created from scratch.
+ */
+ agg_info = create_rel_agg_info(root, joinrel_plain);
+
+ /* Cannot we build grouped join? */
+ if (agg_info == NULL)
+ return joinrel_plain;
+
+ /*
+ * The number of aggregate input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ agg_info->input_rows = joinrel_plain->rows;
+ }
+
+ /*
+ * 2) join two plain rels and aggregate the join paths. Aggregate
+ * push-down only makes sense if the join is not the top-level one.
+ */
+ if (bms_nonempty_difference(root->all_baserels, joinrelids))
+ make_join_rel_common(root, rel1, rel2, agg_info, joinrel_plain);
+
+ /*
+ * 3) combine plain and grouped relations.
+ */
+ make_join_rel_combined(root, rel1, rel2, agg_info);
+
+ return joinrel_plain;
+}
+
/*
* populate_joinrel_with_paths
* Add paths to the given joinrel for given pair of joining relations. The
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index e978b491f6..b55e858246 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -47,6 +47,8 @@ typedef struct PostponedQual
} PostponedQual;
+static void create_aggregate_grouped_var_infos(PlannerInfo *root);
+static void create_grouping_expr_grouped_var_infos(PlannerInfo *root);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
@@ -272,6 +274,261 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
}
}
+/*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate as well as for
+ * each possible grouping expression.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+extern void
+setup_aggregate_pushdown(PlannerInfo *root)
+{
+ ListCell *lc;
+
+ /*
+ * Isn't user interested in the aggregate push-down feature?
+ */
+ if (!enable_agg_pushdown)
+ return;
+
+ /* The feature can only be applied to grouped aggregation. */
+ if (!root->parse->groupClause)
+ return;
+
+ /*
+ * Grouping sets require multiple different groupings but the base
+ * relation can only generate one.
+ */
+ if (root->parse->groupingSets)
+ return;
+
+ /*
+ * SRF is not allowed in the aggregate argument and we don't even want it
+ * in the GROUP BY clause, so forbid it in general. It needs to be
+ * analyzed if evaluation of a GROUP BY clause containing SRF below the
+ * query targetlist would be correct. Currently it does not seem to be an
+ * important use case.
+ */
+ if (root->parse->hasTargetSRFs)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_aggregate_grouped_var_infos(root);
+
+ /* Isn't there any aggregate to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Create GroupedVarInfo per grouping expression. */
+ create_grouping_expr_grouped_var_infos(root);
+
+ /* Isn't there any useful grouping expression for aggregate push-down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /*
+ * Now that we know that grouping can be pushed down, search for the
+ * maximum sortgroupref. The base relations may need it if extra grouping
+ * expressions get added to them.
+ */
+ Assert(root->max_sortgroupref == 0);
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc);
+
+ if (te->ressortgroupref > root->max_sortgroupref)
+ root->max_sortgroupref = te->ressortgroupref;
+ }
+}
+
+/*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ */
+static void
+create_aggregate_grouped_var_infos(PlannerInfo *root)
+{
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES |
+ PVC_INCLUDE_WINDOWFUNCS);
+
+ /*
+ * Although GroupingFunc is related to root->parse->groupingSets, this
+ * field does not necessarily reflect its presence.
+ */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, GroupingFunc))
+ return;
+ }
+
+ /*
+ * Aggregates within the HAVING clause need to be processed in the same
+ * way as those in the main targetlist.
+ *
+ * Note that the contained aggregates will be pushed down, but the
+ * containing HAVING clause must be ignored until the aggregation is
+ * finalized.
+ */
+ if (root->parse->havingQual != NULL)
+ {
+ List *having_exprs;
+
+ having_exprs = pull_var_clause((Node *) root->parse->havingQual,
+ PVC_INCLUDE_AGGREGATES);
+ if (having_exprs != NIL)
+ tlist_exprs = list_concat(tlist_exprs, having_exprs);
+ }
+
+ if (tlist_exprs == NIL)
+ return;
+
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ /*
+ * tlist_exprs may also contain Vars or WindowFuncs, but we only need
+ * Aggrefs.
+ */
+ if (IsA(expr, Var) ||IsA(expr, WindowFunc))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct)
+ {
+ /*
+ * Aggregation push-down is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ gvi = lfirst_node(GroupedVarInfo, lc2);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ gvi = makeNode(GroupedVarInfo);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ gvi->gv_eval_at = NULL;
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+}
+
+/*
+ * Create GroupedVarInfo for each expression usable as grouping key.
+ *
+ * In addition to the expressions of the query targetlist, group_pathkeys is
+ * also considered the source of grouping expressions. That increases the
+ * chance to get the relation output grouped.
+ */
+static void
+create_grouping_expr_grouped_var_infos(PlannerInfo *root)
+{
+ ListCell *l1,
+ *l2;
+ List *exprs = NIL;
+ List *sortgrouprefs = NIL;
+
+ /*
+ * Make sure GroupedVarInfo exists for each expression usable as grouping
+ * key.
+ */
+ foreach(l1, root->parse->groupClause)
+ {
+ SortGroupClause *sgClause;
+ TargetEntry *te;
+ Index sortgroupref;
+
+ sgClause = lfirst_node(SortGroupClause, l1);
+ te = get_sortgroupclause_tle(sgClause, root->processed_tlist);
+ sortgroupref = te->ressortgroupref;
+
+ Assert(sortgroupref > 0);
+
+ /*
+ * Non-zero sortgroupref does not necessarily imply grouping
+ * expression: data can also be sorted by aggregate.
+ */
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ /*
+ * The aggregate push-down feature currently supports only plain Vars
+ * as grouping expressions.
+ */
+ if (!IsA(te->expr, Var))
+ {
+ root->grouped_var_list = NIL;
+ return;
+ }
+
+ exprs = lappend(exprs, te->expr);
+ sortgrouprefs = lappend_int(sortgrouprefs, sortgroupref);
+ }
+
+ /*
+ * Construct GroupedVarInfo for each expression.
+ */
+ forboth(l1, exprs, l2, sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, l1);
+ int sortgroupref = lfirst_int(l2);
+ GroupedVarInfo *gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvexpr = (Expr *) copyObject(var);
+ gvi->sortgroupref = sortgroupref;
+
+ /* Find out where the expression should be evaluated. */
+ gvi->gv_eval_at = bms_make_singleton(var->varno);
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+}
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planagg.c b/src/backend/optimizer/plan/planagg.c
index 8634940efc..64add6718f 100644
--- a/src/backend/optimizer/plan/planagg.c
+++ b/src/backend/optimizer/plan/planagg.c
@@ -441,7 +441,7 @@ build_minmax_path(PlannerInfo *root, MinMaxAggInfo *mminfo,
subroot->tuple_fraction = 1.0;
subroot->limit_tuples = 1.0;
- final_rel = query_planner(subroot, minmax_qp_callback, NULL);
+ final_rel = query_planner(subroot, minmax_qp_callback, NULL, NULL);
/*
* Since we didn't go through subquery_planner() to handle the subquery,
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 5fa33ec200..ae73017db9 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -45,6 +45,9 @@
* qp_callback is a function to compute query_pathkeys once it's safe to do so
* qp_extra is optional extra data to pass to qp_callback
*
+ * If final_rel_grouped_p is valid, relation containing grouped paths may be
+ * saved to *final_rel_grouped_p.
+ *
* Note: the PlannerInfo node also includes a query_pathkeys field, which
* tells query_planner the sort order that is desired in the final output
* plan. This value is *not* available at call time, but is computed by
@@ -53,7 +56,8 @@
*/
RelOptInfo *
query_planner(PlannerInfo *root,
- query_pathkeys_callback qp_callback, void *qp_extra)
+ query_pathkeys_callback qp_callback, void *qp_extra,
+ RelOptInfo **final_rel_grouped_p)
{
Query *parse = root->parse;
List *joinlist;
@@ -66,6 +70,8 @@ query_planner(PlannerInfo *root,
* here.
*/
root->join_rel_list = makeNode(RelInfoList);
+ root->grouped_rel_list = makeNode(RelInfoList);
+ root->agg_info_list = makeNode(RelInfoList);
root->join_rel_level = NULL;
root->join_cur_level = 0;
root->canon_pathkeys = NIL;
@@ -74,6 +80,7 @@ query_planner(PlannerInfo *root,
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
@@ -252,6 +259,16 @@ query_planner(PlannerInfo *root,
*/
extract_restriction_or_clauses(root);
+ /*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, setup
+ * root->grouped_var_list.
+ *
+ * The base relations should be fully initialized now, so that we have
+ * enough info to decide whether grouping is possible.
+ */
+ setup_aggregate_pushdown(root);
+
/*
* Now expand appendrels by adding "otherrels" for their children. We
* delay this to the end so that we have as much information as possible
@@ -272,5 +289,9 @@ query_planner(PlannerInfo *root,
final_rel->cheapest_total_path->param_info != NULL)
elog(ERROR, "failed to construct the join relation");
+ if (final_rel_grouped_p)
+ *final_rel_grouped_p = find_grouped_rel(root, final_rel->relids,
+ NULL);
+
return final_rel;
}
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index d6f2153593..24c3c8d196 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -150,6 +150,7 @@ static double get_number_of_groups(PlannerInfo *root,
List *target_list);
static RelOptInfo *create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
PathTarget *target,
bool target_parallel_safe,
const AggClauseCosts *agg_costs,
@@ -163,6 +164,7 @@ static RelOptInfo *make_grouping_rel(PlannerInfo *root, RelOptInfo *input_rel,
Node *havingQual);
static void create_ordinary_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
RelOptInfo *grouped_rel,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd,
@@ -626,6 +628,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
memset(root->upper_rels, 0, sizeof(root->upper_rels));
memset(root->upper_targets, 0, sizeof(root->upper_targets));
root->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
root->grouping_map = NULL;
root->minmax_aggs = NIL;
root->qual_security_level = 0;
@@ -1954,6 +1957,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
List *activeWindows = NIL;
grouping_sets_data *gset_data = NULL;
standard_qp_extra qp_extra;
+ RelOptInfo *current_rel_grouped = NULL;
/* A recursive query should always have setOperations */
Assert(!root->hasRecursion);
@@ -2056,7 +2060,9 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
* We also generate (in standard_qp_callback) pathkey representations
* of the query's sort clause, distinct clause, etc.
*/
- current_rel = query_planner(root, standard_qp_callback, &qp_extra);
+ current_rel = query_planner(root,
+ standard_qp_callback, &qp_extra,
+ ¤t_rel_grouped);
/*
* Convert the query's result tlist into PathTarget format.
@@ -2201,6 +2207,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
{
current_rel = create_grouping_paths(root,
current_rel,
+ current_rel_grouped,
grouping_target,
grouping_target_parallel_safe,
&agg_costs,
@@ -3805,6 +3812,7 @@ get_number_of_groups(PlannerInfo *root,
static RelOptInfo *
create_grouping_paths(PlannerInfo *root,
RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
PathTarget *target,
bool target_parallel_safe,
const AggClauseCosts *agg_costs,
@@ -3895,7 +3903,8 @@ create_grouping_paths(PlannerInfo *root,
else
extra.patype = PARTITIONWISE_AGGREGATE_NONE;
- create_ordinary_grouping_paths(root, input_rel, grouped_rel,
+ create_ordinary_grouping_paths(root, input_rel, input_rel_grouped,
+ grouped_rel,
agg_costs, gd, &extra,
&partially_grouped_rel);
}
@@ -4053,6 +4062,7 @@ create_degenerate_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
*/
static void
create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
+ RelOptInfo *input_rel_grouped,
RelOptInfo *grouped_rel,
const AggClauseCosts *agg_costs,
grouping_sets_data *gd,
@@ -4100,13 +4110,23 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
if ((extra->flags & GROUPING_CAN_PARTIAL_AGG) != 0)
{
bool force_rel_creation;
+ bool have_agg_pushdown_paths;
/*
- * If we're doing partitionwise aggregation at this level, force
- * creation of a partially_grouped_rel so we can add partitionwise
- * paths to it.
+ * Check if the aggregate push-down feature succeeded to generate any
+ * paths. Dummy relation can appear here because grouped paths are not
+ * guaranteed to exist for a relation.
*/
- force_rel_creation = (patype == PARTITIONWISE_AGGREGATE_PARTIAL);
+ have_agg_pushdown_paths = input_rel_grouped != NULL &&
+ !IS_DUMMY_REL(input_rel_grouped);
+
+ /*
+ * If we're doing partitionwise aggregation at this level or if
+ * aggregate push-down succeeded to create some paths, force creation
+ * of a partially_grouped_rel so we can add the related paths to it.
+ */
+ force_rel_creation = patype == PARTITIONWISE_AGGREGATE_PARTIAL ||
+ have_agg_pushdown_paths;
partially_grouped_rel =
create_partial_grouping_paths(root,
@@ -4115,6 +4135,23 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
gd,
extra,
force_rel_creation);
+
+ /*
+ * No further processing is needed for paths provided by the aggregate
+ * push-down feature. Simply add them to the partially grouped
+ * relation.
+ */
+ if (have_agg_pushdown_paths)
+ {
+ ListCell *lc;
+
+ foreach(lc, input_rel_grouped->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ add_path(partially_grouped_rel, path);
+ }
+ }
}
/* Set out parameter. */
@@ -4139,10 +4176,14 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
/* Gather any partially grouped partial paths. */
if (partially_grouped_rel && partially_grouped_rel->partial_pathlist)
- {
gather_grouping_paths(root, partially_grouped_rel);
+
+ /*
+ * The non-partial paths can come either from the Gather above or from
+ * aggregate push-down.
+ */
+ if (partially_grouped_rel && partially_grouped_rel->pathlist)
set_cheapest(partially_grouped_rel);
- }
/*
* Estimate number of groups.
@@ -7302,6 +7343,7 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
/* Create grouping paths for this child relation. */
create_ordinary_grouping_paths(root, child_input_rel,
+ NULL,
child_grouped_rel,
agg_costs, gd, &child_extra,
&child_partially_grouped_rel);
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 3dcded506b..7656f5d66f 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -2456,6 +2456,39 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, Aggref))
+ {
+ Aggref *aggref = castNode(Aggref, node);
+
+ /*
+ * The upper plan targetlist can contain Aggref whose value has
+ * already been evaluated by the subplan. However this can only happen
+ * with specific value of aggsplit.
+ */
+ if (aggref->aggsplit == AGGSPLIT_INITIAL_SERIAL)
+ {
+ /* See if the Aggref has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ }
+
+ /* No referent found for Aggref */
+ elog(ERROR, "Aggref not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 14521728c6..d34065d61e 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -923,6 +923,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
memset(subroot->upper_rels, 0, sizeof(subroot->upper_rels));
memset(subroot->upper_targets, 0, sizeof(subroot->upper_targets));
subroot->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
subroot->grouping_map = NULL;
subroot->minmax_aggs = NIL;
subroot->qual_security_level = 0;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index e6d08aede5..f3264b35cb 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2527,8 +2527,7 @@ create_projection_path(PlannerInfo *root,
pathnode->path.pathtype = T_Result;
pathnode->path.parent = rel;
pathnode->path.pathtarget = target;
- /* For now, assume we are above any joins, so no parameterization */
- pathnode->path.param_info = NULL;
+ pathnode->path.param_info = subpath->param_info;
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe &&
@@ -2967,6 +2966,146 @@ create_agg_path(PlannerInfo *root,
return pathnode;
}
+/*
+ * Apply AGG_SORTED aggregation path to subpath if it's suitably sorted.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+AggPath *
+create_agg_sorted_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ RelAggInfo *agg_info)
+{
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ ListCell *lc1;
+ List *key_subset = NIL;
+ AggPath *result = NULL;
+
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, (Node *) agg_exprs, aggsplit, &agg_costs);
+
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until the
+ * final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath, target,
+ AGG_SORTED, aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+
+ /* The agg path should require no fewer parameters than the plain one. */
+ result->path.param_info = subpath->param_info;
+
+ return result;
+}
+
+/*
+ * Apply AGG_HASHED aggregation to subpath.
+ */
+AggPath *
+create_agg_hashed_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, RelAggInfo *agg_info)
+{
+ bool can_hash;
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ double hashaggtablesize;
+ Query *parse = root->parse;
+ AggPath *result = NULL;
+
+ /* Do not try to create hash table for each parameter value. */
+ Assert(subpath->param_info == NULL);
+
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, agg_exprs, aggsplit, &agg_costs);
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until
+ * the final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath,
+ target,
+ AGG_HASHED,
+ aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+ }
+ }
+
+ return result;
+}
+
/*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index ccd2a8bd33..3ee38bf1f6 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -17,18 +17,24 @@
#include <limits.h>
#include "miscadmin.h"
+#include "catalog/pg_class_d.h"
+#include "catalog/pg_constraint.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
+#include "optimizer/optimizer.h"
#include "optimizer/inherit.h"
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/placeholder.h"
#include "optimizer/plancat.h"
+#include "optimizer/planner.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+#include "parser/parse_oper.h"
#include "partitioning/partbounds.h"
#include "utils/hsearch.h"
+#include "utils/selfuncs.h"
/*
@@ -67,6 +73,9 @@ static void build_child_join_reltarget(PlannerInfo *root,
RelOptInfo *childrel,
int nappinfos,
AppendRelInfo **appinfos);
+static bool init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p);
/*
@@ -359,6 +368,102 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
return rel;
}
+/*
+ * build_simple_grouped_rel
+ * Construct a new RelOptInfo for a grouped base relation out of an
+ * existing non-grouped relation. On success, pointer to the corresponding
+ * RelAggInfo is stored in *agg_info_p in addition to returning the grouped
+ * relation.
+ */
+RelOptInfo *
+build_simple_grouped_rel(PlannerInfo *root, int relid,
+ RelAggInfo **agg_info_p)
+{
+ RangeTblEntry *rte;
+ RelOptInfo *rel_plain,
+ *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /* Isn't there any grouping expression to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return NULL;
+
+ rel_plain = root->simple_rel_array[relid];
+
+ /* Caller should only pass rti that represents base relation. */
+ Assert(rel_plain != NULL);
+
+ /*
+ * Not all RTE kinds are supported when grouping is considered.
+ *
+ * TODO Consider relaxing some of these restrictions.
+ */
+ rte = root->simple_rte_array[rel_plain->relid];
+ if (rte->rtekind != RTE_RELATION ||
+ rte->relkind == RELKIND_FOREIGN_TABLE ||
+ rte->tablesample != NULL)
+ return NULL;
+
+ /*
+ * Grouped append relation is not supported yet.
+ */
+ if (rte->inh)
+ return NULL;
+
+ /*
+ * Currently we do not support child relations ("other rels").
+ */
+ if (rel_plain->reloptkind != RELOPT_BASEREL)
+ return NULL;
+
+ /*
+ * Prepare the information we need for aggregation of the rel contents.
+ */
+ agg_info = create_rel_agg_info(root, rel_plain);
+ if (agg_info == NULL)
+ return NULL;
+
+ /*
+ * TODO Consider if 1) a flat copy is o.k., 2) it's safer in terms of
+ * adding new fields to RelOptInfo) to copy everything and then reset some
+ * fields, or to zero the structure and copy individual fields.
+ */
+ rel_grouped = makeNode(RelOptInfo);
+ memcpy(rel_grouped, rel_plain, sizeof(RelOptInfo));
+
+ /*
+ * Note on consider_startup: while the AGG_HASHED strategy needs the whole
+ * relation, AGG_SORTED does not. Therefore we do not force
+ * consider_startup to false.
+ */
+
+ /*
+ * Set the appropriate target for grouped paths.
+ *
+ * reltarget should match the target of partially aggregated paths.
+ */
+ rel_grouped->reltarget = agg_info->target;
+
+ /*
+ * Grouped paths must not be mixed with the plain ones.
+ */
+ rel_grouped->pathlist = NIL;
+ rel_grouped->partial_pathlist = NIL;
+ rel_grouped->cheapest_startup_path = NULL;
+ rel_grouped->cheapest_total_path = NULL;
+ rel_grouped->cheapest_unique_path = NULL;
+ rel_grouped->cheapest_parameterized_paths = NIL;
+
+ /*
+ * The number of aggregation input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ agg_info->input_rows = rel_plain->rows;
+
+ *agg_info_p = agg_info;
+ return rel_grouped;
+}
+
/*
* find_base_rel
* Find a base or other relation entry, which must already exist.
@@ -468,10 +573,14 @@ find_rel_info(RelInfoList *list, Relids relids)
foreach(l, list->items)
{
void *item = lfirst(l);
- Relids item_relids;
+ Relids item_relids = NULL;
- Assert(IsA(item, RelOptInfo));
- item_relids = ((RelOptInfo *) item)->relids;
+ Assert(IsA(item, RelOptInfo) ||IsA(item, RelAggInfo));
+
+ if (IsA(item, RelOptInfo))
+ item_relids = ((RelOptInfo *) item)->relids;
+ else if (IsA(item, RelAggInfo))
+ item_relids = ((RelAggInfo *) item)->relids;
if (bms_equal(item_relids, relids))
return item;
@@ -500,7 +609,7 @@ find_join_rel(PlannerInfo *root, Relids relids)
static void
add_rel_info(RelInfoList *list, void *data)
{
- Assert(IsA(data, RelOptInfo));
+ Assert(IsA(data, RelOptInfo) ||IsA(data, RelAggInfo));
/* GEQO requires us to append the new joinrel to the end of the list! */
list->items = lappend(list->items, data);
@@ -512,7 +621,11 @@ add_rel_info(RelInfoList *list, void *data)
RelInfoEntry *hentry;
bool found;
- relids = ((RelOptInfo *) data)->relids;
+ if (IsA(data, RelOptInfo))
+ relids = ((RelOptInfo *) data)->relids;
+ else if (IsA(data, RelAggInfo))
+ relids = ((RelAggInfo *) data)->relids;
+
hentry = (RelInfoEntry *) hash_search(list->hash,
&relids,
HASH_ENTER,
@@ -533,6 +646,57 @@ add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
add_rel_info(root->join_rel_list, joinrel);
}
+/*
+ * add_grouped_rel
+ * Add grouped base or join relation to the list of grouped relations in
+ * the given PlannerInfo. Also add the corresponding RelAggInfo to
+ * agg_info_list.
+ */
+void
+add_grouped_rel(PlannerInfo *root, RelOptInfo *rel, RelAggInfo *agg_info)
+{
+ add_rel_info(root->grouped_rel_list, rel);
+ add_rel_info(root->agg_info_list, agg_info);
+}
+
+/*
+ * find_grouped_rel
+ * Returns grouped relation entry (base or join relation) corresponding to
+ * 'relids' or NULL if none exists.
+ *
+ * If agg_info_p is a valid pointer, then pointer to RelAggInfo that
+ * corresponds to the relation returned is assigned to *agg_info_p.
+ */
+RelOptInfo *
+find_grouped_rel(PlannerInfo *root, Relids relids, RelAggInfo **agg_info_p)
+{
+ RelOptInfo *rel;
+
+ rel = (RelOptInfo *) find_rel_info(root->grouped_rel_list, relids);
+ if (rel == NULL)
+ {
+ if (agg_info_p)
+ *agg_info_p = NULL;
+
+ return NULL;
+ }
+
+ /* Is caller interested in RelAggInfo? */
+ if (agg_info_p)
+ {
+ RelAggInfo *agg_info;
+
+ agg_info = (RelAggInfo *) find_rel_info(root->agg_info_list, relids);
+
+ /* The relation exists, so the agg_info should be there too. */
+ Assert(agg_info != NULL);
+
+ *agg_info_p = agg_info;
+ }
+
+ return rel;
+}
+
/*
* set_foreign_rel_properties
* Set up foreign-join fields if outer and inner relation are foreign
@@ -595,6 +759,7 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* 'restrictlist_ptr': result variable. If not NULL, *restrictlist_ptr
* receives the list of RestrictInfo nodes that apply to this
* particular pair of joinable relations.
+ * 'agg_info' indicates that grouped join relation should be created.
*
* restrictlist_ptr makes the routine's API a little grotty, but it saves
* duplicated calculation of the restrictlist...
@@ -605,10 +770,12 @@ build_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr)
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info)
{
RelOptInfo *joinrel;
List *restrictlist;
+ bool grouped = agg_info != NULL;
/* This function should be used only for join between parents. */
Assert(!IS_OTHER_REL(outer_rel) && !IS_OTHER_REL(inner_rel));
@@ -616,7 +783,8 @@ build_join_rel(PlannerInfo *root,
/*
* See if we already have a joinrel for this set of base rels.
*/
- joinrel = find_join_rel(root, joinrelids);
+ joinrel = !grouped ? find_join_rel(root, joinrelids) :
+ find_grouped_rel(root, joinrelids, NULL);
if (joinrel)
{
@@ -709,9 +877,21 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ if (!grouped)
+ {
+ joinrel->reltarget = create_empty_pathtarget();
+ build_joinrel_tlist(root, joinrel, outer_rel);
+ build_joinrel_tlist(root, joinrel, inner_rel);
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ }
+ else
+ {
+ /*
+ * The target for grouped join should already have its cost and width
+ * computed, see create_rel_agg_info().
+ */
+ joinrel->reltarget = agg_info->target;
+ }
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -743,49 +923,73 @@ build_join_rel(PlannerInfo *root,
joinrel->has_eclass_joins = has_relevant_eclass_joinclause(root, joinrel);
/* Store the partition information. */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- sjinfo->jointype);
-
- /*
- * Set estimates of the joinrel's size.
- */
- set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
- sjinfo, restrictlist);
-
- /*
- * Set the consider_parallel flag if this joinrel could potentially be
- * scanned within a parallel worker. If this flag is false for either
- * inner_rel or outer_rel, then it must be false for the joinrel also.
- * Even if both are true, there might be parallel-restricted expressions
- * in the targetlist or quals.
- *
- * Note that if there are more than two rels in this relation, they could
- * be divided between inner_rel and outer_rel in any arbitrary way. We
- * assume this doesn't matter, because we should hit all the same baserels
- * and joinclauses while building up to this joinrel no matter which we
- * take; therefore, we should make the same decision here however we get
- * here.
- */
- if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
- is_parallel_safe(root, (Node *) restrictlist) &&
- is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
- joinrel->consider_parallel = true;
+ if (!grouped)
+ build_joinrel_partition_info(joinrel, outer_rel, inner_rel,
+ restrictlist, sjinfo->jointype);
/* Add the joinrel to the PlannerInfo. */
- add_join_rel(root, joinrel);
+ if (!grouped)
+ add_join_rel(root, joinrel);
+ else
+ add_grouped_rel(root, joinrel, agg_info);
/*
- * Also, if dynamic-programming join search is active, add the new joinrel
- * to the appropriate sublist. Note: you might think the Assert on number
- * of members should be for equality, but some of the level 1 rels might
- * have been joinrels already, so we can only assert <=.
+ * Also, if dynamic-programming join search is active, add the new
+ * joinrelset to the appropriate sublist. Note: you might think the
+ * Assert on number of members should be for equality, but some of the
+ * level 1 rels might have been joinrels already, so we can only assert
+ * <=.
+ *
+ * Do noting for grouped relation as it's stored aside from
+ * join_rel_level.
*/
- if (root->join_rel_level)
+ if (root->join_rel_level && !grouped)
{
Assert(root->join_cur_level > 0);
- Assert(root->join_cur_level <= bms_num_members(joinrel->relids));
+ Assert(root->join_cur_level <= bms_num_members(joinrelids));
root->join_rel_level[root->join_cur_level] =
- lappend(root->join_rel_level[root->join_cur_level], joinrel);
+ lappend(root->join_rel_level[root->join_cur_level],
+ joinrel);
+ }
+
+ /* Set estimates of the joinrel's size. */
+ if (!grouped)
+ {
+ set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
+ sjinfo, restrictlist);
+
+ /*
+ * Set the consider_parallel flag if this joinrel could potentially be
+ * scanned within a parallel worker. If this flag is false for either
+ * inner_rel or outer_rel, then it must be false for the joinrel also.
+ * Even if both are true, there might be parallel-restricted
+ * expressions in the targetlist or quals.
+ *
+ * Note that if there are more than two rels in this relation, they
+ * could be divided between inner_rel and outer_rel in any arbitrary
+ * way. We assume this doesn't matter, because we should hit all the
+ * same baserels and joinclauses while building up to this joinrel no
+ * matter which we take; therefore, we should make the same decision
+ * here however we get here.
+ */
+ if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
+ is_parallel_safe(root, (Node *) restrictlist) &&
+ is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
+ joinrel->consider_parallel = true;
+ }
+ else
+ {
+ /*
+ * Grouping essentially changes the number of rows.
+ *
+ * XXX We do not distinguish whether two plain rels are joined and the
+ * result is aggregated, or the aggregation has been already applied
+ * to one of the input rels. Is this worth extra effort, e.g.
+ * maintaining a separate RelOptInfo for each case (one difficulty
+ * that would introduce is construction of AppendPath)?
+ */
+ joinrel->rows = estimate_num_groups(root, agg_info->group_exprs,
+ agg_info->input_rows, NULL);
}
return joinrel;
@@ -1836,3 +2040,625 @@ build_child_join_reltarget(PlannerInfo *root,
childrel->reltarget->cost.per_tuple = parentrel->reltarget->cost.per_tuple;
childrel->reltarget->width = parentrel->reltarget->width;
}
+
+/*
+ * Check if the relation can produce grouped paths and return the information
+ * it'll need for it. The passed relation is the non-grouped one which has the
+ * reltarget already constructed.
+ */
+RelAggInfo *
+create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel)
+{
+ List *gvis;
+ List *aggregates = NIL;
+ bool found_other_rel_agg;
+ ListCell *lc;
+ RelAggInfo *result;
+ PathTarget *agg_input;
+ PathTarget *target = NULL;
+ List *grp_exprs_extra = NIL;
+ List *group_clauses_final;
+ int i;
+
+ /*
+ * The function shouldn't have been called if there's no opportunity for
+ * aggregation push-down.
+ */
+ Assert(root->grouped_var_list != NIL);
+
+ result = makeNode(RelAggInfo);
+
+ /*
+ * The current implementation of aggregation push-down cannot handle
+ * PlaceHolderVar (PHV).
+ *
+ * If we knew that the PHV should be evaluated in this target (and of
+ * course, if its expression matched some Aggref argument), we'd just let
+ * init_grouping_targets add that Aggref. On the other hand, if we knew
+ * that the PHV is evaluated below the current rel, we could ignore it
+ * because the referencing Aggref would take care of propagation of the
+ * value to upper joins.
+ *
+ * The problem is that the same PHV can be evaluated in the target of the
+ * current rel or in that of lower rel --- depending on the input paths.
+ * For example, consider rel->relids = {A, B, C} and if ph_eval_at = {B,
+ * C}. Path "A JOIN (B JOIN C)" implies that the PHV is evaluated by the
+ * "(B JOIN C)", while path "(A JOIN B) JOIN C" evaluates the PHV itself.
+ */
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *expr = lfirst(lc);
+
+ if (IsA(expr, PlaceHolderVar))
+ return NULL;
+ }
+
+ if (IS_SIMPLE_REL(rel))
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return NULL;
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the Agg plan
+ * would receive different input at the base rel level.
+ *
+ * XXX For RELOPT_JOINREL, do not return if all the joins that can set any
+ * entry of the grouped target (do we need to postpone this check until
+ * the grouped target is available, and init_grouping_targets take care?)
+ * of this rel to NULL are provably below rel. (It's ok if rel is one of
+ * these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return NULL;
+
+ /*
+ * Use equivalence classes to generate additional grouping expressions for
+ * the current rel. Without these we might not be able to apply
+ * aggregation to the relation result set.
+ *
+ * It's important that create_grouping_expr_grouped_var_infos has
+ * processed the explicit grouping columns by now. If the grouping clause
+ * contains multiple expressions belonging to the same EC, the original
+ * (i.e. not derived) one should be preferred when we build grouping
+ * target for a relation. Otherwise we have a problem when trying to match
+ * target entries to grouping clauses during plan creation, see
+ * get_grouping_expression().
+ */
+ gvis = list_copy(root->grouped_var_list);
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+ int relid = -1;
+
+ /* Only interested in grouping expressions. */
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ while ((relid = bms_next_member(rel->relids, relid)) >= 0)
+ {
+ GroupedVarInfo *gvi_trans;
+
+ gvi_trans = translate_expression_to_rels(root, gvi, relid);
+ if (gvi_trans != NULL)
+ gvis = lappend(gvis, gvi_trans);
+ }
+ }
+
+ /*
+ * Check if some aggregates or grouping expressions can be evaluated in
+ * this relation's target, and collect all vars referenced by these
+ * aggregates / grouping expressions;
+ */
+ found_other_rel_agg = false;
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which includes
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * init_grouping_targets will handle plain Var grouping
+ * expressions because it needs to look them up in
+ * grouped_var_list anyway.
+ */
+ if (IsA(gvi->gvexpr, Var))
+ continue;
+
+ /*
+ * Currently, GroupedVarInfo only handles Vars and Aggrefs.
+ */
+ Assert(IsA(gvi->gvexpr, Aggref));
+
+ /* We only derive grouped expressions using ECs, not aggregates */
+ Assert(!gvi->derived);
+
+ gvi->agg_partial = (Aggref *) copyObject(gvi->gvexpr);
+ mark_partial_aggref(gvi->agg_partial, AGGSPLIT_INITIAL_SERIAL);
+
+ /*
+ * Accept the aggregate.
+ */
+ aggregates = lappend(aggregates, gvi);
+ }
+ else if (IsA(gvi->gvexpr, Aggref))
+ {
+ /*
+ * Remember that there is at least one aggregate expression that
+ * needs something else than this rel.
+ */
+ found_other_rel_agg = true;
+
+ /*
+ * This condition effectively terminates creation of the
+ * RelAggInfo, so there's no reason to check the next
+ * GroupedVarInfo.
+ */
+ break;
+ }
+ }
+
+ /*
+ * Grouping makes little sense w/o aggregate function and w/o grouping
+ * expressions.
+ */
+ if (aggregates == NIL)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Give up if some other aggregate(s) need relations other than the
+ * current one.
+ *
+ * If the aggregate needs the current rel plus anything else, then the
+ * problem is that grouping of the current relation could make some input
+ * variables unavailable for the "higher aggregate", and it'd also
+ * decrease the number of input rows the "higher aggregate" receives.
+ *
+ * If the aggregate does not even need the current rel, then neither the
+ * current rel nor anything else should be grouped because we do not
+ * support join of two grouped relations.
+ */
+ if (found_other_rel_agg)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Create target for grouped paths as well as one for the input paths of
+ * the aggregation paths.
+ */
+ target = create_empty_pathtarget();
+ agg_input = create_empty_pathtarget();
+
+ /*
+ * Cannot suitable targets for the aggregation push-down be derived?
+ */
+ if (!init_grouping_targets(root, rel, target, agg_input, gvis,
+ &grp_exprs_extra))
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ list_free(gvis);
+
+ /*
+ * Aggregation push-down makes no sense w/o grouping expressions.
+ */
+ if ((list_length(target->exprs) + list_length(grp_exprs_extra)) == 0)
+ return NULL;
+
+ group_clauses_final = root->parse->groupClause;
+
+ /*
+ * If the aggregation target should have extra grouping expressions (in
+ * order to emit input vars for join conditions), add them now. This step
+ * includes assignment of tleSortGroupRef's which we can generate now.
+ */
+ if (list_length(grp_exprs_extra) > 0)
+ {
+ Index sortgroupref;
+
+ /*
+ * We'll have to add some clauses, but query group clause must be
+ * preserved.
+ */
+ group_clauses_final = list_copy(group_clauses_final);
+
+ /*
+ * Always start at root->max_sortgroupref. The extra grouping
+ * expressions aren't used during the final aggregation, so the
+ * sortgroupref values don't need to be unique across the query. Thus
+ * we don't have to increase root->max_sortgroupref, which makes
+ * recognition of the extra grouping expressions pretty easy.
+ */
+ sortgroupref = root->max_sortgroupref;
+
+ /*
+ * Generate the SortGroupClause's and add the expressions to the
+ * target.
+ */
+ foreach(lc, grp_exprs_extra)
+ {
+ Var *var = lfirst_node(Var, lc);
+ SortGroupClause *cl = makeNode(SortGroupClause);
+
+ /*
+ * Initialize the SortGroupClause.
+ *
+ * As the final aggregation will not use this grouping expression,
+ * we don't care whether sortop is < or >. The value of
+ * nulls_first should not matter for the same reason.
+ */
+ cl->tleSortGroupRef = ++sortgroupref;
+ get_sort_group_operators(var->vartype,
+ false, true, false,
+ &cl->sortop, &cl->eqop, NULL,
+ &cl->hashable);
+ group_clauses_final = lappend(group_clauses_final, cl);
+ add_column_to_pathtarget(target, (Expr *) var,
+ cl->tleSortGroupRef);
+
+ /*
+ * The aggregation input target must emit this var too.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) var,
+ cl->tleSortGroupRef);
+ }
+ }
+
+ /*
+ * Add aggregates to the grouping target.
+ */
+ add_aggregates_to_target(root, target, aggregates);
+
+ /*
+ * Build a list of grouping expressions and a list of the corresponding
+ * SortGroupClauses.
+ */
+ i = 0;
+ foreach(lc, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(lc);
+
+ if (IsA(texpr, Aggref))
+ {
+ /*
+ * Once we see Aggref, no grouping expressions should follow.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides being an aggregate, the target expression should have no
+ * other reason then being a column of a relation functionally
+ * dependent on the GROUP BY clause. So it's not actually a grouping
+ * column.
+ */
+ if (sortgroupref == 0)
+ continue;
+
+ /*
+ * group_clause_final contains the "local" clauses, so this search
+ * should succeed.
+ */
+ cl = get_sortgroupref_clause(sortgroupref, group_clauses_final);
+
+ result->group_clauses = list_append_unique(result->group_clauses,
+ cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ result->group_exprs = list_append_unique(result->group_exprs,
+ texpr);
+ }
+
+ /*
+ * Since neither target nor agg_input is supposed to be identical to the
+ * source reltarget, compute the width and cost again.
+ *
+ * target does not yet contain aggregates, but these will be accounted by
+ * AggPath.
+ */
+ set_pathtarget_cost_width(root, target);
+ set_pathtarget_cost_width(root, agg_input);
+
+ result->relids = bms_copy(rel->relids);
+ result->target = target;
+ result->agg_input = agg_input;
+
+ /* Finally collect the aggregates. */
+ while (lc != NULL)
+ {
+ Aggref *aggref = lfirst_node(Aggref, lc);
+
+ /*
+ * Partial aggregation is what the grouped paths should do.
+ */
+ result->agg_exprs = lappend(result->agg_exprs, aggref);
+ lc = lnext(target->exprs, lc);
+ }
+
+ /* The "input_rows" field should be set by caller. */
+ return result;
+}
+
+/*
+ * Initialize target for grouped paths (target) as well as a target for paths
+ * that generate input for aggregation (agg_input).
+ *
+ * group_exprs_extra_p receives a list of Var nodes for which we need to
+ * construct SortGroupClause. Those vars will then be used as additional
+ * grouping expressions, for the sake of join clauses.
+ *
+ * gvis a list of GroupedVarInfo's possibly useful for rel.
+ *
+ * Return true iff the targets could be initialized.
+ */
+static bool
+init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p)
+{
+ ListCell *lc1,
+ *lc2;
+ List *unresolved = NIL;
+ List *unresolved_sortgrouprefs = NIL;
+
+ foreach(lc1, rel->reltarget->exprs)
+ {
+ Var *tvar;
+ bool is_grouping;
+ Index sortgroupref = 0;
+ bool derived = false;
+ bool needed_by_aggregate;
+
+ /*
+ * Given that PlaceHolderVar currently prevents us from doing
+ * aggregation push-down, the source target cannot contain anything
+ * more complex than a Var.
+ */
+ tvar = lfirst_node(Var, lc1);
+
+ is_grouping = is_grouping_expression(gvis, (Expr *) tvar,
+ &sortgroupref, &derived);
+
+ /*
+ * Derived grouping expressions should not be referenced by the query
+ * targetlist, so let them fall into vars_unresolved. It'll be checked
+ * later if the current targetlist needs them. For example, we should
+ * not automatically use Var as a grouping expression if the only
+ * reason for it to be in the plain relation target is that it's
+ * referenced by aggregate argument, and it happens to be in the same
+ * EC as any grouping expression.
+ */
+ if (is_grouping && !derived)
+ {
+ Assert(sortgroupref > 0);
+
+ /*
+ * It's o.k. to use the target expression for grouping.
+ */
+ add_column_to_pathtarget(target, (Expr *) tvar, sortgroupref);
+
+ /*
+ * As for agg_input, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) tvar, sortgroupref);
+
+ /* Process the next expression. */
+ continue;
+ }
+
+ /*
+ * Is this Var needed in the query targetlist for anything else than
+ * aggregate input?
+ */
+ needed_by_aggregate = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc2);
+ ListCell *lc3;
+ List *vars;
+
+ if (!IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (!bms_is_member(tvar->varno, gvi->gv_eval_at))
+ continue;
+
+ /*
+ * XXX Consider some sort of caching.
+ */
+ vars = pull_var_clause((Node *) gvi->gvexpr, PVC_RECURSE_AGGREGATES);
+ foreach(lc3, vars)
+ {
+ Var *var = lfirst_node(Var, lc3);
+
+ if (equal(var, tvar))
+ {
+ needed_by_aggregate = true;
+ break;
+ }
+ }
+ list_free(vars);
+ if (needed_by_aggregate)
+ break;
+ }
+
+ if (needed_by_aggregate)
+ {
+ bool found = false;
+
+ foreach(lc2, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc2);
+
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ if (equal(te->expr, tvar))
+ {
+ found = true;
+ break;
+ }
+ }
+
+ /*
+ * If it's only Aggref input, add it to the aggregation input
+ * target and that's it.
+ */
+ if (!found)
+ {
+ add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
+ continue;
+ }
+ }
+
+ /*
+ * Further investigation involves dependency check, for which we need
+ * to have all the (plain-var) grouping expressions gathered.
+ */
+ unresolved = lappend(unresolved, tvar);
+ unresolved_sortgrouprefs = lappend_int(unresolved_sortgrouprefs,
+ sortgroupref);
+ }
+
+ /*
+ * Check for other possible reasons for the var to be in the plain target.
+ */
+ forboth(lc1, unresolved, lc2, unresolved_sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, lc1);
+ Index sortgroupref = lfirst_int(lc2);
+ RangeTblEntry *rte;
+ List *deps = NIL;
+ Relids relids_subtract;
+ int ndx;
+ RelOptInfo *baserel;
+
+ rte = root->simple_rte_array[var->varno];
+
+ /*
+ * Check if the Var can be in the grouping key even though it's not
+ * mentioned by the GROUP BY clause (and could not be derived using
+ * ECs).
+ */
+ if (sortgroupref == 0 &&
+ check_functional_grouping(rte->relid, var->varno,
+ var->varlevelsup,
+ target->exprs, &deps))
+ {
+ /*
+ * The var shouldn't be actually used as a grouping key (instead,
+ * the one this depends on will be), so sortgroupref should not be
+ * important.
+ */
+ add_new_column_to_pathtarget(target, (Expr *) var);
+ add_new_column_to_pathtarget(agg_input, (Expr *) var);
+
+ /*
+ * The var may or may not be present in generic grouping
+ * expression(s) in addition, but this is handled elsewhere.
+ */
+ continue;
+ }
+
+ /*
+ * Isn't the expression needed by joins above the current rel?
+ *
+ * The relids we're not interested in do include 0, which is the
+ * top-level targetlist. The only reason for relids to contain 0
+ * should be that arg_var is referenced either by aggregate or by
+ * grouping expression, but right now we're interested in the *other*
+ * reasons. (As soon aggregation is pushed down, the aggregates in the
+ * query targetlist no longer need direct reference to arg_var
+ * anyway.)
+ */
+ relids_subtract = bms_copy(rel->relids);
+ bms_add_member(relids_subtract, 0);
+
+ baserel = find_base_rel(root, var->varno);
+ ndx = var->varattno - baserel->min_attr;
+ if (bms_nonempty_difference(baserel->attr_needed[ndx],
+ relids_subtract))
+ {
+ /*
+ * The variable is needed by a join involving this relation. That
+ * case includes variable that is referenced by a generic grouping
+ * expression.
+ *
+ * The only way to bring this var to the aggregation output is to
+ * add it to the grouping expressions too.
+ */
+ if (sortgroupref > 0)
+ {
+ /*
+ * The var could be recognized as a potentially useful
+ * grouping expression at the top of the loop, so we can add
+ * it to the grouping target, as well as to the agg_input.
+ */
+ add_column_to_pathtarget(target, (Expr *) var, sortgroupref);
+ add_column_to_pathtarget(agg_input, (Expr *) var, sortgroupref);
+ }
+ else
+ {
+ /*
+ * Since root->parse->groupClause is not supposed to contain
+ * this expression, we need to construct special
+ * SortGroupClause. Its tleSortGroupRef needs to be unique
+ * within target_agg, so postpone creation of the
+ * SortGroupRefs until we're done with the iteration of
+ * rel->reltarget->exprs.
+ */
+ *group_exprs_extra_p = lappend(*group_exprs_extra_p, var);
+ }
+ }
+ else
+ {
+ /*
+ * As long as the query is semantically correct, arriving here
+ * means that the var is referenced by a generic grouping
+ * expression but not referenced by any join.
+ *
+ * create_rel_agg_info() should add this variable to "agg_input"
+ * target and also add the whole generic expression to "target",
+ * but that's subject to future enhancement of the aggregate
+ * push-down feature.
+ */
+ return false;
+ }
+ }
+
+ return true;
+}
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
index 02a3c6b165..cfb9403859 100644
--- a/src/backend/optimizer/util/tlist.c
+++ b/src/backend/optimizer/util/tlist.c
@@ -825,6 +825,62 @@ apply_pathtarget_labeling_to_tlist(List *tlist, PathTarget *target)
}
}
+/*
+ * For each aggregate grouping expression or aggregate to the grouped target.
+ *
+ * Caller passes the expressions in the form of GroupedVarInfos so that we
+ * don't have to look for gvid.
+ */
+void
+add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions)
+{
+ ListCell *lc;
+
+ /* Create the vars and add them to the target. */
+ foreach(lc, expressions)
+ {
+ GroupedVarInfo *gvi;
+
+ gvi = lfirst_node(GroupedVarInfo, lc);
+ add_column_to_pathtarget(target, (Expr *) gvi->agg_partial,
+ gvi->sortgroupref);
+ }
+}
+
+/*
+ * Find out if expr can be used as grouping expression in reltarget.
+ *
+ * sortgroupref and is_derived reflect the ->sortgroupref and ->derived fields
+ * of the corresponding GroupedVarInfo.
+ */
+bool
+is_grouping_expression(List *gvis, Expr *expr, Index *sortgroupref,
+ bool *is_derived)
+{
+ ListCell *lc;
+
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (equal(gvi->gvexpr, expr))
+ {
+ Assert(gvi->sortgroupref > 0);
+
+ *sortgroupref = gvi->sortgroupref;
+ *is_derived = gvi->derived;
+ return true;
+ }
+ }
+
+ /* The expression cannot be used as grouping key. */
+ return false;
+}
+
/*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index e5f8a1301f..05d1ca0967 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -1028,6 +1028,15 @@ static struct config_bool ConfigureNamesBool[] =
false,
NULL, NULL, NULL
},
+ {
+ {"enable_agg_pushdown", PGC_USERSET, QUERY_TUNING_METHOD,
+ gettext_noop("Enables aggregation push-down."),
+ NULL
+ },
+ &enable_agg_pushdown,
+ false,
+ NULL, NULL, NULL
+ },
{
{"enable_parallel_append", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables the planner's use of parallel append plans."),
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 316c1ecbb9..cb8bba4607 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -224,6 +224,7 @@ typedef enum NodeTag
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_RelAggInfo,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
@@ -268,6 +269,7 @@ typedef enum NodeTag
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index aca7993af3..3a1a158ba1 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -240,6 +240,23 @@ struct PlannerInfo
*/
struct RelInfoList *join_rel_list; /* list of join-relation RelOptInfos */
+ /*
+ * grouped_rel_list is a list of RelOptInfos that represent grouped
+ * relations, both base relations and joins. Unlike join_rel_list, base
+ * relations are accepted because grouped base relation is of rather
+ * limited scope, i.e. it's only needed during join search. Thus it does
+ * not deserve separate storage like simple_rel_array.
+ */
+ struct RelInfoList *grouped_rel_list; /* list of grouped relation
+ * RelOptInfos */
+
+ /*
+ * agg_info_list contains one instance of RelAggInfo per an item of
+ * grouped_rel_list.
+ */
+ struct RelInfoList *agg_info_list; /* list of grouped relation
+ * RelAggInfos */
+
/*
* When doing a dynamic-programming-style join search, join_rel_level[k]
* is a list of all join-relation RelOptInfos of level k, and
@@ -287,6 +304,8 @@ struct PlannerInfo
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
@@ -318,6 +337,12 @@ struct PlannerInfo
*/
List *processed_tlist;
+ /*
+ * The maximum ressortgroupref among target entries in processed_list.
+ * Useful when adding extra grouping expressions for partial aggregation.
+ */
+ int max_sortgroupref;
+
/* Fields filled during create_plan() for use in setrefs.c */
AttrNumber *grouping_map; /* for GroupingFunc fixup */
List *minmax_aggs; /* List of MinMaxAggInfos */
@@ -760,6 +785,60 @@ typedef struct RelInfoList
struct HTAB *hash;
} RelInfoList;
+/*
+ * RelAggInfo
+ * Information needed to create grouped paths for base rels and joins.
+ *
+ * "relids" is the set of base-relation identifiers, just like with
+ * RelOptInfo.
+ *
+ * "target" will be used as pathtarget if partial aggregation is applied to
+ * base relation or join. The same target will also --- if the relation is a
+ * join --- be used to joinin grouped path to a non-grouped one. This target
+ * can contain plain-Var grouping expressions and Aggref nodes.
+ *
+ * Note: There's a convention that Aggref expressions are supposed to follow
+ * the other expressions of the target. Iterations of ->exprs may rely on this
+ * arrangement.
+ *
+ * "agg_input" contains Vars used either as grouping expressions or aggregate
+ * arguments. Paths providing the aggregation plan with input data should use
+ * this target. The only difference from reltarget of the non-grouped relation
+ * is that some items can have sortgroupref initialized.
+ *
+ * "input_rows" is the estimated number of input rows for AggPath. It's
+ * actually just a workspace for users of the structure, i.e. not initialized
+ * when instance of the structure is created.
+ *
+ * "group_clauses" and "group_exprs" are lists of SortGroupClause and the
+ * corresponding grouping expressions respectively.
+ *
+ * "agg_exprs" is a list of Aggref nodes for the aggregation of the relation's
+ * paths.
+ *
+ * "rel_grouped" is the relation containing the partially aggregated paths.
+ */
+typedef struct RelAggInfo
+{
+ NodeTag type;
+
+ Relids relids; /* Base rels contained in this grouped rel. */
+
+ struct PathTarget *target; /* Target for grouped paths. */
+
+ struct PathTarget *agg_input; /* pathtarget of paths that generate input
+ * for aggregation paths. */
+
+ double input_rows;
+
+ List *group_clauses;
+ List *group_exprs;
+
+ List *agg_exprs; /* Aggref expressions. */
+
+ RelOptInfo *rel_grouped; /* Grouped relation. */
+} RelAggInfo;
+
/*
* IndexOptInfo
* Per-index information for planning/optimization
@@ -2283,6 +2362,26 @@ typedef struct PlaceHolderInfo
int32 ph_width; /* estimated attribute width */
} PlaceHolderInfo;
+/*
+ * GroupedVarInfo exists for each expression that can be used as an aggregate
+ * or grouping expression evaluated below a join.
+ */
+typedef struct GroupedVarInfo
+{
+ NodeTag type;
+
+ Expr *gvexpr; /* the represented expression. */
+ Aggref *agg_partial; /* if gvexpr is aggregate, agg_partial is the
+ * corresponding partial aggregate */
+ Index sortgroupref; /* If gvexpr is a grouping expression, this is
+ * the tleSortGroupRef of the corresponding
+ * SortGroupClause. */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+ bool derived; /* derived from another GroupedVarInfo using
+ * equeivalence classes? */
+} GroupedVarInfo;
+
/*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
index b7456e3e59..8544bb6745 100644
--- a/src/include/optimizer/clauses.h
+++ b/src/include/optimizer/clauses.h
@@ -55,4 +55,6 @@ extern void CommuteOpExpr(OpExpr *clause);
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
+extern GroupedVarInfo *translate_expression_to_rels(PlannerInfo *root,
+ GroupedVarInfo *gvi, Index relid);
#endif /* CLAUSES_H */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index e450fe112a..11b823cc52 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -210,6 +210,14 @@ extern AggPath *create_agg_path(PlannerInfo *root,
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+extern AggPath *create_agg_sorted_path(PlannerInfo *root,
+ RelOptInfo *rel,
+ Path *subpath,
+ RelAggInfo *agg_info);
+extern AggPath *create_agg_hashed_path(PlannerInfo *root,
+ RelOptInfo *rel,
+ Path *subpath,
+ RelAggInfo *agg_info);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
@@ -280,14 +288,21 @@ extern void setup_simple_rel_arrays(PlannerInfo *root);
extern void expand_planner_arrays(PlannerInfo *root, int add_size);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
+extern RelOptInfo *build_simple_grouped_rel(PlannerInfo *root, int relid,
+ RelAggInfo **agg_info_p);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
+extern void add_grouped_rel(PlannerInfo *root, RelOptInfo *rel,
+ RelAggInfo *agg_info);
+extern RelOptInfo *find_grouped_rel(PlannerInfo *root, Relids relids,
+ RelAggInfo **agg_info_p);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr);
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info);
extern Relids min_join_parameterization(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
@@ -313,5 +328,5 @@ extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo, JoinType jointype);
-
+extern RelAggInfo *create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index 9ab73bd20c..e23ccb72aa 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -21,6 +21,7 @@
* allpaths.c
*/
extern PGDLLIMPORT bool enable_geqo;
+extern PGDLLIMPORT bool enable_agg_pushdown;
extern PGDLLIMPORT int geqo_threshold;
extern PGDLLIMPORT int min_parallel_table_scan_size;
extern PGDLLIMPORT int min_parallel_index_scan_size;
@@ -52,8 +53,14 @@ extern RelOptInfo *make_one_rel(PlannerInfo *root, List *joinlist);
extern RelOptInfo *standard_join_search(PlannerInfo *root, int levels_needed,
List *initial_rels);
+
extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
+extern void generate_grouping_paths(PlannerInfo *root,
+ RelOptInfo *rel_grouped,
+ RelOptInfo *rel_plain,
+ RelAggInfo *agg_info);
+
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages, int max_workers);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index eab486a621..553bb437ee 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -28,7 +28,8 @@ typedef void (*query_pathkeys_callback) (PlannerInfo *root, void *extra);
* prototypes for plan/planmain.c
*/
extern RelOptInfo *query_planner(PlannerInfo *root,
- query_pathkeys_callback qp_callback, void *qp_extra);
+ query_pathkeys_callback qp_callback, void *qp_extra,
+ RelOptInfo **final_rel_grouped_p);
/*
* prototypes for plan/planagg.c
@@ -69,6 +70,7 @@ extern void add_other_rels_to_query(PlannerInfo *root);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
Relids where_needed, bool create_new_ph);
+extern void setup_aggregate_pushdown(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
index 1d4c7da545..6218517672 100644
--- a/src/include/optimizer/tlist.h
+++ b/src/include/optimizer/tlist.h
@@ -50,8 +50,14 @@ extern void split_pathtarget_at_srfs(PlannerInfo *root,
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+/* TODO Find the best location (position and in some cases even file) for the
+ * following ones. */
+extern void add_aggregates_to_target(PlannerInfo *root, PathTarget *target,
+ List *expressions);
+extern bool is_grouping_expression(List *gvis, Expr *expr,
+ Index *sortgroupref, bool *is_derived);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
-
#endif /* TLIST_H */
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
new file mode 100644
index 0000000000..b3a97f86d6
--- /dev/null
+++ b/src/test/regress/expected/agg_pushdown.out
@@ -0,0 +1,217 @@
+BEGIN;
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+CREATE INDEX ON agg_pushdown_child1(parent);
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+COMMIT;
+ANALYZE;
+SET enable_agg_pushdown TO on;
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+--------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(10 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(11 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (p.i = c1.parent)
+ -> Sort
+ Sort Key: p.i
+ -> Seq Scan on agg_pushdown_parent p
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(12 rows)
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(8 rows)
+
+SET enable_seqscan TO on;
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.x = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(11 rows)
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: ((k = c1.j) AND (parent = c1.parent))
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(13 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+----------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Hash
+ -> Seq Scan on agg_pushdown_child2 c2
+(15 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: ((c1.j = c2.k) AND (c1.parent = c2.parent))
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(13 rows)
+
diff --git a/src/test/regress/expected/sysviews.out b/src/test/regress/expected/sysviews.out
index a1c90eb905..55cfd3fff7 100644
--- a/src/test/regress/expected/sysviews.out
+++ b/src/test/regress/expected/sysviews.out
@@ -72,6 +72,7 @@ select count(*) >= 0 as ok from pg_prepared_xacts;
select name, setting from pg_settings where name like 'enable%';
name | setting
--------------------------------+---------
+ enable_agg_pushdown | off
enable_bitmapscan | on
enable_gathermerge | on
enable_hashagg | on
@@ -89,7 +90,7 @@ select name, setting from pg_settings where name like 'enable%';
enable_seqscan | on
enable_sort | on
enable_tidscan | on
-(17 rows)
+(18 rows)
-- Test that the pg_timezone_names and pg_timezone_abbrevs views are
-- more-or-less working. We can't test their contents in any great detail
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index f86f5c5682..ab62fed156 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -142,6 +142,7 @@ test: stats_ext
test: collate.linux.utf8
test: select_parallel
test: write_parallel
+test: agg_pushdown
test: publication
test: subscription
test: select_views
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
new file mode 100644
index 0000000000..6cf2ed9c20
--- /dev/null
+++ b/src/test/regress/sql/agg_pushdown.sql
@@ -0,0 +1,117 @@
+BEGIN;
+
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+
+CREATE INDEX ON agg_pushdown_child1(parent);
+
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+COMMIT;
+ANALYZE;
+
+SET enable_agg_pushdown TO on;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_seqscan TO on;
+
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
--
2.20.1
Re: WIP: Aggregation push-down
Hello,
Thank you for this great feature !
I hope this will be reviewed/validated soon ;o)
Just a comment:
set enable_agg_pushdown to true;
isn't displayed in EXPLAIN (SETTINGS) syntax.
The following modification seems to fix that:
src/backend/utils/misc/guc.c
{"enable_agg_pushdown", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables aggregation push-down."),
NULL,
GUC_EXPLAIN <<<<--- line Added --->>>>
},
&enable_agg_pushdown,
false,
NULL, NULL, NULL
},
then
postgres=# set enable_agg_pushdown = true;
SET
postgres=# explain (settings) select 1;
QUERY PLAN
------------------------------------------
Result (cost=0.00..0.01 rows=1 width=4)
Settings: enable_agg_pushdown = 'on'
(2 rows)
Regards
PAscal
--
Sent from: https://www.postgresql-archive.org/PostgreSQL-hackers-f1928748.html
Re: WIP: Aggregation push-down
legrand legrand <legrand_legrand@hotmail.com> wrote:
set enable_agg_pushdown to true;
isn't displayed in EXPLAIN (SETTINGS) syntax.The following modification seems to fix that:
src/backend/utils/misc/guc.c
{"enable_agg_pushdown", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables aggregation push-down."),
NULL,
GUC_EXPLAIN <<<<--- line Added --->>>>
},
&enable_agg_pushdown,
false,
NULL, NULL, NULL
},
Thanks. I'll include this change in the next version.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
Re: [HACKERS] WIP: Aggregation push-down
Hi,
I've been looking at the 'make_join_rel()' part of the patch, and I'm
aware that, if we are joining A and B, a 'grouped join rel (AB)' would
be created besides the 'plain join rel (AB)', and paths are added by 1)
applying partial aggregate to the paths of the 'plain join rel (AB)', or
2) joining grouped A to plain B or joining plain A to grouped B.
This is a smart idea. One issue I can see is that some logic would have
to be repeated several times. For example, the validity check for the
same proposed join would be performed at most three times by
join_is_legal().
I'm thinking of another idea that, instead of using a separate
RelOptInfo for the grouped rel, we add in RelOptInfo a
'grouped_pathlist' for the grouped paths, just like how we implement
parallel query via adding 'partial_pathlist'.
For base rel, if we decide it can produce grouped paths, we create the
grouped paths by applying partial aggregation to the paths in 'pathlist'
and add them into 'grouped_pathlist'.
For join rel (AB), we can create the grouped paths for it by:
1) joining a grouped path from 'A->grouped_pathlist' to a plain path
from 'B->pathlist', or vice versa;
2) applying partial aggregation to the paths in '(AB)->pathlist'.
This is basically the same idea, just moves the grouped paths from the
grouped rel to a 'grouped_pathlist'. With it we should not need to make
any code changes to 'make_join_rel()'. Most code changes would happen in
'add_paths_to_joinrel()'.
Will this idea work? Is it better or worse?
Thanks
Richard
Re: [HACKERS] WIP: Aggregation push-down
Richard Guo <guofenglinux@gmail.com> wrote:
Hi,
I've been looking at the 'make_join_rel()' part of the patch, and I'm
aware that, if we are joining A and B, a 'grouped join rel (AB)' would
be created besides the 'plain join rel (AB)', and paths are added by 1)
applying partial aggregate to the paths of the 'plain join rel (AB)', or
2) joining grouped A to plain B or joining plain A to grouped B.This is a smart idea. One issue I can see is that some logic would have
to be repeated several times. For example, the validity check for the
same proposed join would be performed at most three times by
join_is_legal().I'm thinking of another idea that, instead of using a separate
RelOptInfo for the grouped rel, we add in RelOptInfo a
'grouped_pathlist' for the grouped paths, just like how we implement
parallel query via adding 'partial_pathlist'.For base rel, if we decide it can produce grouped paths, we create the
grouped paths by applying partial aggregation to the paths in 'pathlist'
and add them into 'grouped_pathlist'.For join rel (AB), we can create the grouped paths for it by:
1) joining a grouped path from 'A->grouped_pathlist' to a plain path
from 'B->pathlist', or vice versa;
2) applying partial aggregation to the paths in '(AB)->pathlist'.This is basically the same idea, just moves the grouped paths from the
grouped rel to a 'grouped_pathlist'. With it we should not need to make
any code changes to 'make_join_rel()'. Most code changes would happen in
'add_paths_to_joinrel()'.Will this idea work? Is it better or worse?
I tried such approach in an earlier version of the patch [1]/messages/by-id/18007.1513957437@localhost, and I think the
reason also was to avoid repeated calls of functions like
join_is_legal(). However there were objections against such approach,
e.g. [2]/messages/by-id/CA+Tgmob8og+9HzMg1vM+3LwDm2f_bHUi9+g1bqLDTjqpw5s+nQ@mail.gmail.com, and I admit that it was more invasive than what the current patch
version does.
Perhaps we can cache the result of join_is_legal() that we get for the plain
relation, and use it for the group relation. I'll consider that. Thanks.
[1]: /messages/by-id/18007.1513957437@localhost
[2]: /messages/by-id/CA+Tgmob8og+9HzMg1vM+3LwDm2f_bHUi9+g1bqLDTjqpw5s+nQ@mail.gmail.com
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
Re: WIP: Aggregation push-down
Antonin Houska-2 wrote
Alvaro Herrera <
alvherre@
> wrote:
This stuff seems very useful. How come it sits unreviewed for so long?
I think the review is hard for people who are not interested in the
planner
very much. And as for further development, there are a few design
decisions
that can hardly be resolved without Tom Lane's comments. Right now I
recall
two problems: 1) is the way I currently store RelOptInfo for the grouped
relations correct?, 2) how should we handle types for which logical
equality
does not imply physical (byte-wise) equality?Fortunately it seems now that I'm not the only one who cares about 2), so
this
problem might be resolved soon:/messages/by-id/CAH2-Wzn3Ee49Gmxb7V1VJ3-AC8fWn-Fr8pfWQebHe8rYRxt5OQ@mail.gmail.com
But 1) still remains.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
Hello
would "pgsql: Add equalimage B-Tree support functions."
/messages/by-id/E1j72NY-0002gi-2B@gemulon.postgresql.org
help for 2) ?
Regards
PAscal
--
Sent from: https://www.postgresql-archive.org/PostgreSQL-hackers-f1928748.html
Re: WIP: Aggregation push-down
legrand legrand <legrand_legrand@hotmail.com> wrote:
Antonin Houska-2 wrote
Right now I recall two problems: 1) is the way I currently store
RelOptInfo for the grouped relations correct?, 2) how should we handle
types for which logical equality does not imply physical (byte-wise)
equality?Fortunately it seems now that I'm not the only one who cares about 2), so
this
problem might be resolved soon:/messages/by-id/CAH2-Wzn3Ee49Gmxb7V1VJ3-AC8fWn-Fr8pfWQebHe8rYRxt5OQ@mail.gmail.com
But 1) still remains.
Hello
would "pgsql: Add equalimage B-Tree support functions."
/messages/by-id/E1j72NY-0002gi-2B@gemulon.postgresql.org
Yes, it seems so. I'll adapt the patch soon, hopefully next week. Thanks for
reminder.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
Re: WIP: Aggregation push-down
On Thu, Feb 27, 2020 at 4:50 PM Antonin Houska <ah@cybertec.at> wrote:
legrand legrand <legrand_legrand@hotmail.com> wrote:
Antonin Houska-2 wrote
Right now I recall two problems: 1) is the way I currently store
RelOptInfo for the grouped relations correct?, 2) how should we handle
types for which logical equality does not imply physical (byte-wise)
equality?Fortunately it seems now that I'm not the only one who cares about 2),
so
this
problem might be resolved soon:/messages/by-id/CAH2-Wzn3Ee49Gmxb7V1VJ3-AC8fWn-Fr8pfWQebHe8rYRxt5OQ@mail.gmail.com
But 1) still remains.
Hello
would "pgsql: Add equalimage B-Tree support functions."/messages/by-id/E1j72NY-0002gi-2B@gemulon.postgresql.org
Yes, it seems so. I'll adapt the patch soon, hopefully next week. Thanks
for
reminder.--
Antonin Houska
Web: https://www.cybertec-postgresql.com
Hi Antonin:
The more tests on your patch, the more powerful I feel it is! At the same
time,
I think the most difficult part to understand your design is you can accept
any number of generic join clauses, so I guess more explanation on this
part
may be helpful.
At the code level, I did some slight changes on init_grouping_targets which
may
make the code easier to read. You are free to to use/not use it.
Hope your patch get more attention soon!
Best Regards
Andy Fan
Attachments:
v2-0001-tiny-changes-for-init_grouping_targets.patchapplication/octet-stream; name=v2-0001-tiny-changes-for-init_grouping_targets.patchDownload
From 7a2a460f80c61909b3f6ab7b39f3ecf2c2fba2b0 Mon Sep 17 00:00:00 2001
From: Andy Fan <zhihui.fan1213@gmail.com>
Date: Tue, 14 Apr 2020 11:27:41 +0800
Subject: [PATCH v2] tiny changes for init_grouping_targets
---
src/backend/optimizer/util/relnode.c | 274 +++++++++++----------------
1 file changed, 107 insertions(+), 167 deletions(-)
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 3ee38bf1f6..68a347e2d9 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -73,6 +73,7 @@ static void build_child_join_reltarget(PlannerInfo *root,
RelOptInfo *childrel,
int nappinfos,
AppendRelInfo **appinfos);
+static bool aggref_used_var(PlannerInfo *root, Var *var);
static bool init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
PathTarget *target, PathTarget *agg_input,
List *gvis, List **group_exprs_extra_p);
@@ -2066,8 +2067,6 @@ create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel)
*/
Assert(root->grouped_var_list != NIL);
- result = makeNode(RelAggInfo);
-
/*
* The current implementation of aggregation push-down cannot handle
* PlaceHolderVar (PHV).
@@ -2334,6 +2333,8 @@ create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel)
* SortGroupClauses.
*/
i = 0;
+
+ result = makeNode(RelAggInfo);
foreach(lc, target->exprs)
{
Index sortgroupref = 0;
@@ -2429,10 +2430,15 @@ init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
PathTarget *target, PathTarget *agg_input,
List *gvis, List **group_exprs_extra_p)
{
- ListCell *lc1,
- *lc2;
- List *unresolved = NIL;
- List *unresolved_sortgrouprefs = NIL;
+ ListCell *lc1;
+ List *grouping_columns = NIL;
+
+ foreach(lc1, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvar = lfirst_node(GroupedVarInfo, lc1);
+ if (IsA(gvar->gvexpr, Var))
+ grouping_columns = lappend(grouping_columns, gvar->gvexpr);
+ }
foreach(lc1, rel->reltarget->exprs)
{
@@ -2440,7 +2446,8 @@ init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
bool is_grouping;
Index sortgroupref = 0;
bool derived = false;
- bool needed_by_aggregate;
+ RangeTblEntry *rte;
+ List *deps = NIL;
/*
* Given that PlaceHolderVar currently prevents us from doing
@@ -2452,6 +2459,8 @@ init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
is_grouping = is_grouping_expression(gvis, (Expr *) tvar,
&sortgroupref, &derived);
+ rte = root->simple_rte_array[tvar->varno];
+
/*
* Derived grouping expressions should not be referenced by the query
* targetlist, so let them fall into vars_unresolved. It'll be checked
@@ -2475,190 +2484,121 @@ init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
* sortgroupref in addition.
*/
add_column_to_pathtarget(agg_input, (Expr *) tvar, sortgroupref);
-
- /* Process the next expression. */
- continue;
}
-
- /*
- * Is this Var needed in the query targetlist for anything else than
- * aggregate input?
- */
- needed_by_aggregate = false;
- foreach(lc2, root->grouped_var_list)
+ else if (aggref_used_var(root, tvar) &&
+ tlist_member((Expr*) tvar, root->processed_tlist) == NULL)
{
- GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc2);
- ListCell *lc3;
- List *vars;
-
- if (!IsA(gvi->gvexpr, Aggref))
- continue;
-
- if (!bms_is_member(tvar->varno, gvi->gv_eval_at))
- continue;
-
- /*
- * XXX Consider some sort of caching.
- */
- vars = pull_var_clause((Node *) gvi->gvexpr, PVC_RECURSE_AGGREGATES);
- foreach(lc3, vars)
- {
- Var *var = lfirst_node(Var, lc3);
-
- if (equal(var, tvar))
- {
- needed_by_aggregate = true;
- break;
- }
- }
- list_free(vars);
- if (needed_by_aggregate)
- break;
+ add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
}
-
- if (needed_by_aggregate)
- {
- bool found = false;
-
- foreach(lc2, root->processed_tlist)
- {
- TargetEntry *te = lfirst_node(TargetEntry, lc2);
-
- if (IsA(te->expr, Aggref))
- continue;
-
- if (equal(te->expr, tvar))
- {
- found = true;
- break;
- }
- }
-
- /*
- * If it's only Aggref input, add it to the aggregation input
- * target and that's it.
- */
- if (!found)
- {
- add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
- continue;
- }
- }
-
- /*
- * Further investigation involves dependency check, for which we need
- * to have all the (plain-var) grouping expressions gathered.
- */
- unresolved = lappend(unresolved, tvar);
- unresolved_sortgrouprefs = lappend_int(unresolved_sortgrouprefs,
- sortgroupref);
- }
-
- /*
- * Check for other possible reasons for the var to be in the plain target.
- */
- forboth(lc1, unresolved, lc2, unresolved_sortgrouprefs)
- {
- Var *var = lfirst_node(Var, lc1);
- Index sortgroupref = lfirst_int(lc2);
- RangeTblEntry *rte;
- List *deps = NIL;
- Relids relids_subtract;
- int ndx;
- RelOptInfo *baserel;
-
- rte = root->simple_rte_array[var->varno];
-
- /*
- * Check if the Var can be in the grouping key even though it's not
- * mentioned by the GROUP BY clause (and could not be derived using
- * ECs).
- */
- if (sortgroupref == 0 &&
- check_functional_grouping(rte->relid, var->varno,
- var->varlevelsup,
- target->exprs, &deps))
+ else if (sortgroupref == 0 &&
+ check_functional_grouping(rte->relid, tvar->varno,
+ tvar->varlevelsup, grouping_columns, &deps))
{
/*
* The var shouldn't be actually used as a grouping key (instead,
* the one this depends on will be), so sortgroupref should not be
* important.
*/
- add_new_column_to_pathtarget(target, (Expr *) var);
- add_new_column_to_pathtarget(agg_input, (Expr *) var);
-
- /*
- * The var may or may not be present in generic grouping
- * expression(s) in addition, but this is handled elsewhere.
- */
- continue;
+ add_new_column_to_pathtarget(target, (Expr *) tvar);
+ add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
}
-
- /*
- * Isn't the expression needed by joins above the current rel?
- *
- * The relids we're not interested in do include 0, which is the
- * top-level targetlist. The only reason for relids to contain 0
- * should be that arg_var is referenced either by aggregate or by
- * grouping expression, but right now we're interested in the *other*
- * reasons. (As soon aggregation is pushed down, the aggregates in the
- * query targetlist no longer need direct reference to arg_var
- * anyway.)
- */
- relids_subtract = bms_copy(rel->relids);
- bms_add_member(relids_subtract, 0);
-
- baserel = find_base_rel(root, var->varno);
- ndx = var->varattno - baserel->min_attr;
- if (bms_nonempty_difference(baserel->attr_needed[ndx],
- relids_subtract))
+ else
{
+ Relids relids_subtract;
+ int ndx;
+ RelOptInfo *baserel;
/*
- * The variable is needed by a join involving this relation. That
- * case includes variable that is referenced by a generic grouping
- * expression.
+ * Isn't the expression needed by joins above the current rel?
*
- * The only way to bring this var to the aggregation output is to
- * add it to the grouping expressions too.
+ * The relids we're not interested in do include 0, which is the
+ * top-level targetlist. The only reason for relids to contain 0
+ * should be that arg_var is referenced either by aggregate or by
+ * grouping expression, but right now we're interested in the *other*
+ * reasons. (As soon aggregation is pushed down, the aggregates in the
+ * query targetlist no longer need direct reference to arg_var
+ * anyway.)
*/
- if (sortgroupref > 0)
+ relids_subtract = bms_copy(rel->relids);
+ bms_add_member(relids_subtract, 0);
+
+ baserel = find_base_rel(root, tvar->varno);
+ ndx = tvar->varattno - baserel->min_attr;
+ if (bms_nonempty_difference(baserel->attr_needed[ndx],
+ relids_subtract))
{
/*
- * The var could be recognized as a potentially useful
- * grouping expression at the top of the loop, so we can add
- * it to the grouping target, as well as to the agg_input.
+ * The variable is needed by a join involving this relation. That
+ * case includes variable that is referenced by a generic grouping
+ * expression.
+ *
+ * The only way to bring this var to the aggregation output is to
+ * add it to the grouping expressions too.
*/
- add_column_to_pathtarget(target, (Expr *) var, sortgroupref);
- add_column_to_pathtarget(agg_input, (Expr *) var, sortgroupref);
+ if (sortgroupref > 0)
+ {
+ /*
+ * The var could be recognized as a potentially useful
+ * grouping expression at the top of the loop, so we can add
+ * it to the grouping target, as well as to the agg_input.
+ */
+ add_column_to_pathtarget(target, (Expr *) tvar, sortgroupref);
+ add_column_to_pathtarget(agg_input, (Expr *) tvar, sortgroupref);
+ }
+ else
+ {
+ /*
+ * Since root->parse->groupClause is not supposed to contain
+ * this expression, we need to construct special
+ * SortGroupClause. Its tleSortGroupRef needs to be unique
+ * within target_agg, so postpone creation of the
+ * SortGroupRefs until we're done with the iteration of
+ * rel->reltarget->exprs.
+ */
+ *group_exprs_extra_p = lappend(*group_exprs_extra_p, tvar);
+ }
}
else
{
/*
- * Since root->parse->groupClause is not supposed to contain
- * this expression, we need to construct special
- * SortGroupClause. Its tleSortGroupRef needs to be unique
- * within target_agg, so postpone creation of the
- * SortGroupRefs until we're done with the iteration of
- * rel->reltarget->exprs.
+ * As long as the query is semantically correct, arriving here
+ * means that the var is referenced by a generic grouping
+ * expression but not referenced by any join.
+ *
+ * create_rel_agg_info() should add this variable to "agg_input"
+ * target and also add the whole generic expression to "target",
+ * but that's subject to future enhancement of the aggregate
+ * push-down feature.
*/
- *group_exprs_extra_p = lappend(*group_exprs_extra_p, var);
+ return false;
}
}
- else
- {
- /*
- * As long as the query is semantically correct, arriving here
- * means that the var is referenced by a generic grouping
- * expression but not referenced by any join.
- *
- * create_rel_agg_info() should add this variable to "agg_input"
- * target and also add the whole generic expression to "target",
- * but that's subject to future enhancement of the aggregate
- * push-down feature.
- */
- return false;
- }
}
-
return true;
}
+
+static bool
+aggref_used_var(PlannerInfo *root, Var *var)
+{
+
+ ListCell *lc;
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+ List *vars;
+
+ if (!IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (!bms_is_member(var->varno, gvi->gv_eval_at))
+ continue;
+
+ /*
+ * XXX Consider some sort of caching.
+ */
+ vars = pull_var_clause((Node *) gvi->gvexpr, PVC_RECURSE_AGGREGATES);
+ if (list_member(vars, var))
+ return true;
+ }
+
+ return false;
+}
--
2.21.0
Re: WIP: Aggregation push-down
1) v14-0001-Introduce-RelInfoList-structure.patch
-------------------------------------------------- I'm not entirely sure why we need this change. We had the list+hash
before, so I assume we do this because we need the output functions?I believe that this is what Tom proposed in [1], see "Maybe an appropriate
preliminary patch is to refactor the support code ..." near the end of that
message. The point is that now we need two lists: one for "plain" relations
and one for grouped ones.
I guess what Toms suggested[1]/messages/by-id/9726.1542577439@sss.pgh.pa.us is to store the the grouped ones into
root->upper_rels rather than a separated member, see fetch_upper_rel
or UpperRelationKind. If we do need the list+hash method for long list
lookup,
we can merge it into upper_rels. If we want this benefits at other place
rather
than root->upper_rels, we can store a generic type in RelInfoList, looks
currently
it is bounded to RelAggInfo besides RelOptInfo. But overall, personally I
think we can
ignore such optimization at the first stage to save the attention of the
core reviewers
since they are too precious :) Just FYI
[1]: /messages/by-id/9726.1542577439@sss.pgh.pa.us
Best Regards
Andy Fan
Re: WIP: Aggregation push-down
Andy Fan <zhihui.fan1213@gmail.com> wrote:
The more tests on your patch, the more powerful I feel it is!
Thanks for the appreciation. Given the poor progress it's increasingly hard
for me to find motivation to work on it. I'll try to be more professional :-)
At the same time, I think the most difficult part to understand your design
is you can accept any number of generic join clauses, so I guess more
explanation on this part may be helpful.
ok, I'll consider adding some comments, although the concept is mentioned in
optimizer/README
+Furthermore, extra grouping columns can be added to the partial Agg node if a
+join clause above that node references a column which is not in the query
+GROUP BY clause and which could not be derived using equivalence class.
+
...At the code level, I did some slight changes on init_grouping_targets which may
make the code easier to read. You are free to to use/not use it.
I'm going to accept your change of create_rel_agg_info(), but I hesitate about
the changes to init_grouping_targets().
First, is it worth to spend CPU cycles on construction of an extra list
grouping_columns? Is there a corner case in which we cannot simply pass
grouping_columns=target->exprs to check_functional_grouping()?
Second, it's obvious that you prefer the style
foreach ()
{
if ()
...
else if ()
...
else
...
}
over this
foreach ()
{
if ()
{
...
continue;
}
if ()
{
...
continue;
}
...
}
I often prefer the latter and I see that the existing planner code uses this
style quite often too. I think the reason is that it allows for more complex
tests, while the "else-if style" requires all tests to take place inside the
"if ()" expression. However, if several (not necessarily tightly related)
tests become "compressed" this way, it's less obvious how where to put
comments. Indeed it seems that some comments got lost due to your changes.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
Re: WIP: Aggregation push-down
Andy Fan <zhihui.fan1213@gmail.com> wrote:
1) v14-0001-Introduce-RelInfoList-structure.patch
-------------------------------------------------- I'm not entirely sure why we need this change. We had the list+hash
before, so I assume we do this because we need the output functions?I believe that this is what Tom proposed in [1], see "Maybe an appropriate
preliminary patch is to refactor the support code ..." near the end of that
message. The point is that now we need two lists: one for "plain" relations
and one for grouped ones.I guess what Toms suggested[1] is to store the the grouped ones into
root->upper_rels rather than a separated member, see fetch_upper_rel
or UpperRelationKind. If we do need the list+hash method for long list lookup,
we can merge it into upper_rels. If we want this benefits at other place rather
than root->upper_rels, we can store a generic type in RelInfoList, looks currently
it is bounded to RelAggInfo besides RelOptInfo. But overall, personally I think we can
ignore such optimization at the first stage to save the attention of the core reviewers
since they are too precious :) Just FYI
Hm, you seem to be right, not sure why I missed the point. I thought that the
reason Tom doesn't like storing the grouped relations in simple_rel_array is
that we only need the grouped base relations inside query_planner(), but
simple_rel_array is used higher in the stack. So I introduced a new field and
used it only in query_planner() and subroutines.
Yes, it's better to use root->upper_rels than to introduce the new field. I'll
adjust the patch. Thanks.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
Re: WIP: Aggregation push-down
On Fri, Apr 24, 2020 at 8:10 PM Antonin Houska <ah@cybertec.at> wrote:
Andy Fan <zhihui.fan1213@gmail.com> wrote:
The more tests on your patch, the more powerful I feel it is!
Thanks for the appreciation. Given the poor progress it's increasingly hard
for me to find motivation to work on it. I'll try to be more professional
:-)
Let's not give up:) I see your patch can push down the aggregation in 3
cases
at least:
1). The group by clause exists in the join eq clause.
2). The group by clause doesn't exist in join eq clause, but we have
pk on on side of join eq clause.
3). The group by clause doesn't exist in join eq clause, and we don't have
the pk as well.
Tom well explained the correctness of the first 2 cases [1]/messages/by-id/9726.1542577439@sss.pgh.pa.us and probably
the case
3) is correct as well, but it is a bit of hard to understand. If this is a
case for others
as well, that probably make the review harder.
So my little suggestion is can we split the patch into some smaller commit
to handle each case? like: commit 1 & 2 handles case 1 & 2，commit 3 handles
join relation as well. commit 4 handles the case 3. Commit 5 can avoid
the
two-phase aggregation for case 2. Commit 6 introduces the aggmultifn. and
commit 7 to handle some outer join case Ashutosh raised at [2]/messages/by-id/CAFjFpRdpeMTd8kYbM_x0769V-aEKst5Nkg3+coG=8ki7s8Zqjw@mail.gmail.com. However not
all the cases need to be handled at one time.
Actually when I read the code, I want such separation by my own to verify my
understanding is correct, but I don't have such time at least now. It
maybe much
easier for you since you are the author. I will be pretty pleasure to help
in any case
after I close my current working item (Hopefully in 2 months).
At the code level, I did some slight changes on init_grouping_targets
which maymake the code easier to read. You are free to to use/not use it.
I'm going to accept your change of create_rel_agg_info(), but I hesitate
about
the changes to init_grouping_targets().First, is it worth to spend CPU cycles on construction of an extra list
grouping_columns? Is there a corner case in which we cannot simply pass
grouping_columns=target->exprs to check_functional_grouping()?Second, it's obvious that you prefer the style
foreach ()
{
if ()
...
else if ()
...
else
...
}over this
foreach ()
{
if ()
{
...
continue;
}if ()
{
...
continue;
}...
}I often prefer the latter and I see that the existing planner code uses
this
style quite often too. I think the reason is that it allows for more
complex
tests, while the "else-if style" requires all tests to take place inside
the
"if ()" expression. However, if several (not necessarily tightly related)
tests become "compressed" this way, it's less obvious how where to put
comments. Indeed it seems that some comments got lost due to your changes.
Your explanation looks reasonable, thanks for that. the changes also used
some builtin function to avoid the one more level for-loop.
like tlist_member.
As for the high level design, based on my current knowledge, probably no
need
to change.
[1]: /messages/by-id/9726.1542577439@sss.pgh.pa.us
[2]: /messages/by-id/CAFjFpRdpeMTd8kYbM_x0769V-aEKst5Nkg3+coG=8ki7s8Zqjw@mail.gmail.com
/messages/by-id/CAFjFpRdpeMTd8kYbM_x0769V-aEKst5Nkg3+coG=8ki7s8Zqjw@mail.gmail.com
Re: WIP: Aggregation push-down
Andy Fan <zhihui.fan1213@gmail.com> wrote:
On Fri, Apr 24, 2020 at 8:10 PM Antonin Houska <ah@cybertec.at> wrote:
Andy Fan <zhihui.fan1213@gmail.com> wrote:
Let's not give up:) I see your patch can push down the aggregation in 3 cases
at least:1). The group by clause exists in the join eq clause.
2). The group by clause doesn't exist in join eq clause, but we have
pk on on side of join eq clause.
3). The group by clause doesn't exist in join eq clause, and we don't have
the pk as well.Tom well explained the correctness of the first 2 cases [1]
I admit that I don't quite understand his statement
We can't simply aggregate during the scan of "a", because some values of
"x" might not appear at all in the input of the naively-computed aggregate
(if their rows have no join partner in "b"), or might appear multiple
times (if their rows have multiple join partners).
Consider the following tables
======= ======
A B
======= ======
x | y z
------- ------
3 | 4 1
1 | 4 2
4 | 0 3
1 | 0 4
and this query
SELECT sum(a.x), a.y FROM a, b WHERE a.y > b.z GROUP BY a.y;
Without aggregate push-down, the aggregation input is
a.x | a.y | b.z
----------------
3 | 4 | 1
3 | 4 | 2
3 | 4 | 3
1 | 4 | 1
1 | 4 | 2
1 | 4 | 3
and the query result is
sum(a.x) | a.y
-----------------
12 | 4
With the push-down, "A" aggregated looks like
sum(a.x) | a.y
-----------------
4 | 4
5 | 0
and the join with "B" produces this:
sum(a.x) | a.y | b.z
---------------------
4 | 4 | 1
4 | 4 | 2
4 | 4 | 3
After that, the final aggregation yields the same result that we got w/o the
aggregate push-down:
sum(a.x) | a.y
-----------------
12 | 4
So if a row of "A" has no / multiple join partners in "B", the same is true
for the group produced by the pushed-down aggregation. Of course it's assumed
that the input rows belonging to particular group can be distributed among the
PartialAgg nodes in arbitrary way, for example:
Agg
<value 1>
<value 1>
<value 2>
<value 2>
needs to be equivalent to
FinalAgg
PartialAgg
<value 1>
<value 2>
PartialAgg
<value 1>
<value 2>
This should be o.k. because the partial aggregation performed currently by
parallel workers relies o the same "distributive property" of aggregation.
So my little suggestion is can we split the patch into some smaller commit
to handle each case? like: commit 1 & 2 handles case 1 & 2，commit 3 handles
join relation as well. commit 4 handles the case 3.
To separate the patch this way effectively means to rework the feature almost
from scratch. I'm still not convinced that such a rework is necesary.
Commit 5 can avoid the two-phase aggregation for case 2.
The two-phase aggregation can be avoided in more generic way as soon as the
"unique key" feature (I think you do participated in the related discussion)
is committed.
Commit 6 introduces the aggmultifnf.
I used this function in the initial version of the patch [3]/messages/by-id/29111.1483984605@localhost, to join two
grouped relations. The current version does not try to join two grouped
relations and it's hard to imagine how to determine the "multiplication
factor". I suppose this is not really trivial. Since the current version of
the feature hasn't find it's way into PG core for several years, I'm not
willing to make it even trickier soon, if at all.
and commit 7 to handle some outer join case Ashutosh raised at [2].
That was too high-level design, as he admits in [4]/messages/by-id/CAFjFpRfUKq3Asgtki1XctPkCN6YC4oA2vNWh66OgBo-H26ePWA@mail.gmail.com.
However not all the cases need to be handled at one time.
Sure.
At the code level, I did some slight changes on init_grouping_targets which may
make the code easier to read. You are free to to use/not use it.I'm going to accept your change of create_rel_agg_info(), but I hesitate about
the changes to init_grouping_targets().First, is it worth to spend CPU cycles on construction of an extra list
grouping_columns? Is there a corner case in which we cannot simply pass
grouping_columns=target->exprs to check_functional_grouping()?Second, it's obvious that you prefer the style
foreach ()
{
if ()
...
else if ()
...
else
...
}over this
foreach ()
{
if ()
{
...
continue;
}if ()
{
...
continue;
}...
}I often prefer the latter and I see that the existing planner code uses this
style quite often too. I think the reason is that it allows for more complex
tests, while the "else-if style" requires all tests to take place inside the
"if ()" expression. However, if several (not necessarily tightly related)
tests become "compressed" this way, it's less obvious how where to put
comments. Indeed it seems that some comments got lost due to your changes.Your explanation looks reasonable, thanks for that. the changes also used
some builtin function to avoid the one more level for-loop. like tlist_member.
Eventually I've reworked init_grouping_targets(), although not exactly as you
proposed. I hope it's easier to understand now.
[1] /messages/by-id/9726.1542577439@sss.pgh.pa.us
[2] /messages/by-id/CAFjFpRdpeMTd8kYbM_x0769V-aEKst5Nkg3+coG=8ki7s8Zqjw@mail.gmail.com
[3]: /messages/by-id/29111.1483984605@localhost
[4]: /messages/by-id/CAFjFpRfUKq3Asgtki1XctPkCN6YC4oA2vNWh66OgBo-H26ePWA@mail.gmail.com
The next version is attached.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
Attachments:
v16-0001-Introduce-RelInfoList-structure.patchtext/x-diffDownload
From 8bdd663c01444481b434d82479665904aebbe62f Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Mon, 18 May 2020 14:28:31 +0200
Subject: [PATCH 1/3] Introduce RelInfoList structure.
This patch puts join_rel_list and join_rel_hash fields of PlannerInfo
structure into a new structure RelInfoList. It also adjusts add_join_rel() and
find_join_rel() functions so they only call add_rel_info() and find_rel_info()
respectively.
fetch_upper_rel() now uses the new API and the hash table as well because the
list stored in root->upper_rels[UPPERREL_PARTIAL_GROUP_AGG] will contain many
relations as soon as the aggregate push-down feature is added.
---
contrib/postgres_fdw/postgres_fdw.c | 3 +-
src/backend/nodes/outfuncs.c | 11 ++
src/backend/optimizer/geqo/geqo_eval.c | 12 +-
src/backend/optimizer/plan/planmain.c | 3 +-
src/backend/optimizer/util/relnode.c | 182 +++++++++++++++----------
src/include/nodes/nodes.h | 1 +
src/include/nodes/pathnodes.h | 30 ++--
7 files changed, 149 insertions(+), 93 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 9fc53cad68..a46834a377 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -5251,7 +5251,8 @@ foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
*/
Assert(fpinfo->relation_index == 0); /* shouldn't be set yet */
fpinfo->relation_index =
- list_length(root->parse->rtable) + list_length(root->join_rel_list);
+ list_length(root->parse->rtable) +
+ list_length(root->join_rel_list->items);
return true;
}
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index ebf3ce37aa..7ba92c6f28 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2314,6 +2314,14 @@ _outRelOptInfo(StringInfo str, const RelOptInfo *node)
WRITE_NODE_FIELD(partitioned_child_rels);
}
+static void
+_outRelInfoList(StringInfo str, const RelInfoList *node)
+{
+ WRITE_NODE_TYPE("RELOPTINFOLIST");
+
+ WRITE_NODE_FIELD(items);
+}
+
static void
_outIndexOptInfo(StringInfo str, const IndexOptInfo *node)
{
@@ -4111,6 +4119,9 @@ outNode(StringInfo str, const void *obj)
case T_RelOptInfo:
_outRelOptInfo(str, obj);
break;
+ case T_RelInfoList:
+ _outRelInfoList(str, obj);
+ break;
case T_IndexOptInfo:
_outIndexOptInfo(str, obj);
break;
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index ff33acc7b6..6b04ab3c2a 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -92,11 +92,11 @@ geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
*
* join_rel_level[] shouldn't be in use, so just Assert it isn't.
*/
- savelength = list_length(root->join_rel_list);
- savehash = root->join_rel_hash;
+ savelength = list_length(root->join_rel_list->items);
+ savehash = root->join_rel_list->hash;
Assert(root->join_rel_level == NULL);
- root->join_rel_hash = NULL;
+ root->join_rel_list->hash = NULL;
/* construct the best path for the given combination of relations */
joinrel = gimme_tree(root, tour, num_gene);
@@ -121,9 +121,9 @@ geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
* Restore join_rel_list to its former state, and put back original
* hashtable if any.
*/
- root->join_rel_list = list_truncate(root->join_rel_list,
- savelength);
- root->join_rel_hash = savehash;
+ root->join_rel_list->items = list_truncate(root->join_rel_list->items,
+ savelength);
+ root->join_rel_list->hash = savehash;
/* release all the memory acquired within gimme_tree */
MemoryContextSwitchTo(oldcxt);
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 62dfc6d44a..5fa33ec200 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -65,8 +65,7 @@ query_planner(PlannerInfo *root,
* NOTE: append_rel_list was set up by subquery_planner, so do not touch
* here.
*/
- root->join_rel_list = NIL;
- root->join_rel_hash = NULL;
+ root->join_rel_list = makeNode(RelInfoList);
root->join_rel_level = NULL;
root->join_cur_level = 0;
root->canon_pathkeys = NIL;
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index a203e6f1ff..a95e6364ae 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -32,11 +32,15 @@
#include "utils/lsyscache.h"
-typedef struct JoinHashEntry
+/*
+ * An entry of a hash table that we use to make lookup for RelOptInfo
+ * structures more efficient.
+ */
+typedef struct RelInfoEntry
{
- Relids join_relids; /* hash key --- MUST BE FIRST */
- RelOptInfo *join_rel;
-} JoinHashEntry;
+ Relids relids; /* hash key --- MUST BE FIRST */
+ void *data;
+} RelInfoEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *input_rel);
@@ -390,11 +394,11 @@ find_base_rel(PlannerInfo *root, int relid)
}
/*
- * build_join_rel_hash
- * Construct the auxiliary hash table for join relations.
+ * build_rel_hash
+ * Construct the auxiliary hash table for relation specific data.
*/
static void
-build_join_rel_hash(PlannerInfo *root)
+build_rel_hash(RelInfoList *list)
{
HTAB *hashtab;
HASHCTL hash_ctl;
@@ -403,47 +407,53 @@ build_join_rel_hash(PlannerInfo *root)
/* Create the hash table */
MemSet(&hash_ctl, 0, sizeof(hash_ctl));
hash_ctl.keysize = sizeof(Relids);
- hash_ctl.entrysize = sizeof(JoinHashEntry);
+ hash_ctl.entrysize = sizeof(RelInfoEntry);
hash_ctl.hash = bitmap_hash;
hash_ctl.match = bitmap_match;
hash_ctl.hcxt = CurrentMemoryContext;
- hashtab = hash_create("JoinRelHashTable",
+ hashtab = hash_create("RelHashTable",
256L,
&hash_ctl,
HASH_ELEM | HASH_FUNCTION | HASH_COMPARE | HASH_CONTEXT);
/* Insert all the already-existing joinrels */
- foreach(l, root->join_rel_list)
+ foreach(l, list->items)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
- JoinHashEntry *hentry;
+ void *item = lfirst(l);
+ RelInfoEntry *hentry;
bool found;
+ Relids relids;
+
+ Assert(IsA(item, RelOptInfo));
+ relids = ((RelOptInfo *) item)->relids;
- hentry = (JoinHashEntry *) hash_search(hashtab,
- &(rel->relids),
- HASH_ENTER,
- &found);
+ hentry = (RelInfoEntry *) hash_search(hashtab,
+ &relids,
+ HASH_ENTER,
+ &found);
Assert(!found);
- hentry->join_rel = rel;
+ hentry->data = item;
}
- root->join_rel_hash = hashtab;
+ list->hash = hashtab;
}
/*
- * find_join_rel
- * Returns relation entry corresponding to 'relids' (a set of RT indexes),
- * or NULL if none exists. This is for join relations.
+ * find_rel_info
+ * Find a base or join relation entry.
*/
-RelOptInfo *
-find_join_rel(PlannerInfo *root, Relids relids)
+static void *
+find_rel_info(RelInfoList *list, Relids relids)
{
+ if (list == NULL)
+ return NULL;
+
/*
* Switch to using hash lookup when list grows "too long". The threshold
* is arbitrary and is known only here.
*/
- if (!root->join_rel_hash && list_length(root->join_rel_list) > 32)
- build_join_rel_hash(root);
+ if (!list->hash && list_length(list->items) > 32)
+ build_rel_hash(list);
/*
* Use either hashtable lookup or linear search, as appropriate.
@@ -453,34 +463,90 @@ find_join_rel(PlannerInfo *root, Relids relids)
* so would force relids out of a register and thus probably slow down the
* list-search case.
*/
- if (root->join_rel_hash)
+ if (list->hash)
{
Relids hashkey = relids;
- JoinHashEntry *hentry;
+ RelInfoEntry *hentry;
- hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
- &hashkey,
- HASH_FIND,
- NULL);
+ hentry = (RelInfoEntry *) hash_search(list->hash,
+ &hashkey,
+ HASH_FIND,
+ NULL);
if (hentry)
- return hentry->join_rel;
+ return hentry->data;
}
else
{
ListCell *l;
- foreach(l, root->join_rel_list)
+ foreach(l, list->items)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
+ void *item = lfirst(l);
+ Relids item_relids = NULL;
+
+ Assert(IsA(item, RelOptInfo));
- if (bms_equal(rel->relids, relids))
- return rel;
+ item_relids = ((RelOptInfo *) item)->relids;
+ if (bms_equal(item_relids, relids))
+ return item;
}
}
return NULL;
}
+/*
+ * find_join_rel
+ * Returns relation entry corresponding to 'relids' (a set of RT indexes),
+ * or NULL if none exists. This is for join relations.
+ */
+RelOptInfo *
+find_join_rel(PlannerInfo *root, Relids relids)
+{
+ return (RelOptInfo *) find_rel_info(root->join_rel_list, relids);
+}
+
+/*
+ * add_rel_info
+ * Add relation specific info to a list, and also add it to the auxiliary
+ * hashtable if there is one.
+ */
+static void
+add_rel_info(RelInfoList *list, void *data)
+{
+ Assert(IsA(data, RelOptInfo));
+
+ /* GEQO requires us to append the new joinrel to the end of the list! */
+ list->items = lappend(list->items, data);
+
+ /* store it into the auxiliary hashtable if there is one. */
+ if (list->hash)
+ {
+ Relids relids;
+ RelInfoEntry *hentry;
+ bool found;
+
+ relids = ((RelOptInfo *) data)->relids;
+ hentry = (RelInfoEntry *) hash_search(list->hash,
+ &relids,
+ HASH_ENTER,
+ &found);
+ Assert(!found);
+ hentry->data = data;
+ }
+}
+
+/*
+ * add_join_rel
+ * Add given join relation to the list of join relations in the given
+ * PlannerInfo.
+ */
+static void
+add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
+{
+ add_rel_info(root->join_rel_list, joinrel);
+}
+
/*
* set_foreign_rel_properties
* Set up foreign-join fields if outer and inner relation are foreign
@@ -531,32 +597,6 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
}
}
-/*
- * add_join_rel
- * Add given join relation to the list of join relations in the given
- * PlannerInfo. Also add it to the auxiliary hashtable if there is one.
- */
-static void
-add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
-{
- /* GEQO requires us to append the new joinrel to the end of the list! */
- root->join_rel_list = lappend(root->join_rel_list, joinrel);
-
- /* store it into the auxiliary hashtable if there is one. */
- if (root->join_rel_hash)
- {
- JoinHashEntry *hentry;
- bool found;
-
- hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
- &(joinrel->relids),
- HASH_ENTER,
- &found);
- Assert(!found);
- hentry->join_rel = joinrel;
- }
-}
-
/*
* build_join_rel
* Returns relation entry corresponding to the union of two given rels,
@@ -1191,22 +1231,14 @@ subbuild_joinrel_joinlist(RelOptInfo *joinrel,
RelOptInfo *
fetch_upper_rel(PlannerInfo *root, UpperRelationKind kind, Relids relids)
{
+ RelInfoList *list = &root->upper_rels[kind];
RelOptInfo *upperrel;
- ListCell *lc;
-
- /*
- * For the moment, our indexing data structure is just a List for each
- * relation kind. If we ever get so many of one kind that this stops
- * working well, we can improve it. No code outside this function should
- * assume anything about how to find a particular upperrel.
- */
/* If we already made this upperrel for the query, return it */
- foreach(lc, root->upper_rels[kind])
+ if (list)
{
- upperrel = (RelOptInfo *) lfirst(lc);
-
- if (bms_equal(upperrel->relids, relids))
+ upperrel = find_rel_info(list, relids);
+ if (upperrel)
return upperrel;
}
@@ -1225,7 +1257,7 @@ fetch_upper_rel(PlannerInfo *root, UpperRelationKind kind, Relids relids)
upperrel->cheapest_unique_path = NULL;
upperrel->cheapest_parameterized_paths = NIL;
- root->upper_rels[kind] = lappend(root->upper_rels[kind], upperrel);
+ add_rel_info(&root->upper_rels[kind], upperrel);
return upperrel;
}
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 381d84b4e4..be5ab273f0 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -222,6 +222,7 @@ typedef enum NodeTag
T_PlannerInfo,
T_PlannerGlobal,
T_RelOptInfo,
+ T_RelInfoList,
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 485d1b06c9..44374de796 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -93,6 +93,23 @@ typedef enum InheritanceKind
INHKIND_PARTITIONED
} InheritanceKind;
+/*
+ * Hashed list to store relation specific info and to retrieve it by relids.
+ *
+ * For small problems we just scan the list to do lookups, but when there are
+ * many relations we build a hash table for faster lookups. The hash table is
+ * present and valid when rel_hash is not NULL. Note that we still maintain
+ * the list even when using the hash table for lookups; this simplifies life
+ * for GEQO.
+ */
+typedef struct RelInfoList
+{
+ NodeTag type;
+
+ List *items;
+ struct HTAB *hash;
+} RelInfoList;
+
/*----------
* PlannerGlobal
* Global information for planning/optimization
@@ -236,15 +253,9 @@ struct PlannerInfo
/*
* join_rel_list is a list of all join-relation RelOptInfos we have
- * considered in this planning run. For small problems we just scan the
- * list to do lookups, but when there are many join relations we build a
- * hash table for faster lookups. The hash table is present and valid
- * when join_rel_hash is not NULL. Note that we still maintain the list
- * even when using the hash table for lookups; this simplifies life for
- * GEQO.
+ * considered in this planning run.
*/
- List *join_rel_list; /* list of join-relation RelOptInfos */
- struct HTAB *join_rel_hash; /* optional hashtable for join relations */
+ struct RelInfoList *join_rel_list; /* list of join-relation RelOptInfos */
/*
* When doing a dynamic-programming-style join search, join_rel_level[k]
@@ -308,7 +319,8 @@ struct PlannerInfo
List *initial_rels; /* RelOptInfos we are now trying to join */
/* Use fetch_upper_rel() to get any particular upper rel */
- List *upper_rels[UPPERREL_FINAL + 1]; /* upper-rel RelOptInfos */
+ RelInfoList upper_rels[UPPERREL_FINAL + 1]; /* upper-rel RelOptInfos */
+
/* Result tlists chosen by grouping_planner for upper-stage processing */
struct PathTarget *upper_targets[UPPERREL_FINAL + 1];
--
2.20.1
v16-0002-Aggregate-push-down-basic-functionality.patchtext/x-diffDownload
From 80494b93eca35c22a08b31619642d5b8dd7a6e2a Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Mon, 18 May 2020 14:28:31 +0200
Subject: [PATCH 2/3] Aggregate push-down - basic functionality.
With this patch, partial aggregation can be applied to a base relation or to a
join, and the resulting "grouped" relations can be joined to other "plain"
relations. Once all tables are joined, the aggregation is finalized. See
README for more information.
The next patches will enable the aggregate push-down feature for parallel
query processing, for partitioned tables and for foreign tables.
---
src/backend/nodes/copyfuncs.c | 20 +-
src/backend/nodes/outfuncs.c | 34 +
src/backend/optimizer/README | 69 ++
src/backend/optimizer/path/allpaths.c | 147 ++++
src/backend/optimizer/path/costsize.c | 17 +-
src/backend/optimizer/path/equivclass.c | 130 +++
src/backend/optimizer/path/joinrels.c | 195 ++++-
src/backend/optimizer/plan/initsplan.c | 290 +++++++
src/backend/optimizer/plan/planmain.c | 12 +
src/backend/optimizer/plan/planner.c | 47 +-
src/backend/optimizer/plan/setrefs.c | 33 +
src/backend/optimizer/prep/prepjointree.c | 1 +
src/backend/optimizer/util/pathnode.c | 143 +++-
src/backend/optimizer/util/relnode.c | 923 ++++++++++++++++++++-
src/backend/optimizer/util/tlist.c | 31 +
src/backend/utils/misc/guc.c | 10 +
src/include/nodes/nodes.h | 2 +
src/include/nodes/pathnodes.h | 89 ++
src/include/optimizer/clauses.h | 2 +
src/include/optimizer/pathnode.h | 19 +-
src/include/optimizer/paths.h | 6 +
src/include/optimizer/planmain.h | 1 +
src/include/optimizer/tlist.h | 4 +-
src/test/regress/expected/agg_pushdown.out | 215 +++++
src/test/regress/expected/sysviews.out | 3 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/agg_pushdown.sql | 114 +++
27 files changed, 2477 insertions(+), 81 deletions(-)
create mode 100644 src/test/regress/expected/agg_pushdown.out
create mode 100644 src/test/regress/sql/agg_pushdown.sql
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 491452ae2d..4557b2d25a 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2251,8 +2251,8 @@ _copyOnConflictExpr(const OnConflictExpr *from)
/* ****************************************************************
* pathnodes.h copy functions
*
- * We don't support copying RelOptInfo, IndexOptInfo, or Path nodes.
- * There are some subsidiary structs that are useful to copy, though.
+ * We don't support copying RelOptInfo, IndexOptInfo, RelAggInfo or Path
+ * nodes. There are some subsidiary structs that are useful to copy, though.
* ****************************************************************
*/
@@ -2395,6 +2395,19 @@ _copyPlaceHolderInfo(const PlaceHolderInfo *from)
return newnode;
}
+static GroupedVarInfo *
+_copyGroupedVarInfo(const GroupedVarInfo *from)
+{
+ GroupedVarInfo *newnode = makeNode(GroupedVarInfo);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gv_eval_at);
+
+ return newnode;
+}
+
/* ****************************************************************
* parsenodes.h copy functions
* ****************************************************************
@@ -5167,6 +5180,9 @@ copyObjectImpl(const void *from)
case T_PlaceHolderInfo:
retval = _copyPlaceHolderInfo(from);
break;
+ case T_GroupedVarInfo:
+ retval = _copyGroupedVarInfo(from);
+ break;
/*
* VALUE NODES
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 7ba92c6f28..b1efe73d86 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2222,6 +2222,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_BITMAPSET_FIELD(all_baserels);
WRITE_BITMAPSET_FIELD(nullable_baserels);
WRITE_NODE_FIELD(join_rel_list);
+ WRITE_NODE_FIELD(agg_info_list);
WRITE_INT_FIELD(join_cur_level);
WRITE_NODE_FIELD(init_plans);
WRITE_NODE_FIELD(cte_plan_ids);
@@ -2236,6 +2237,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(append_rel_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
@@ -2243,6 +2245,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(distinct_pathkeys);
WRITE_NODE_FIELD(sort_pathkeys);
WRITE_NODE_FIELD(processed_tlist);
+ WRITE_INT_FIELD(max_sortgroupref);
WRITE_NODE_FIELD(minmax_aggs);
WRITE_FLOAT_FIELD(total_table_pages, "%.0f");
WRITE_FLOAT_FIELD(tuple_fraction, "%.4f");
@@ -2464,6 +2467,20 @@ _outParamPathInfo(StringInfo str, const ParamPathInfo *node)
WRITE_NODE_FIELD(ppi_clauses);
}
+static void
+_outRelAggInfo(StringInfo str, const RelAggInfo *node)
+{
+ WRITE_NODE_TYPE("RELAGGINFO");
+
+ WRITE_BITMAPSET_FIELD(relids);
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(agg_input);
+ WRITE_FLOAT_FIELD(input_rows, "%.0f");
+ WRITE_NODE_FIELD(group_clauses);
+ WRITE_NODE_FIELD(group_exprs);
+ WRITE_NODE_FIELD(agg_exprs);
+}
+
static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
@@ -2565,6 +2582,17 @@ _outPlaceHolderInfo(StringInfo str, const PlaceHolderInfo *node)
WRITE_INT_FIELD(ph_width);
}
+static void
+_outGroupedVarInfo(StringInfo str, const GroupedVarInfo *node)
+{
+ WRITE_NODE_TYPE("GROUPEDVARINFO");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_BITMAPSET_FIELD(gv_eval_at);
+}
+
static void
_outMinMaxAggInfo(StringInfo str, const MinMaxAggInfo *node)
{
@@ -4143,6 +4171,9 @@ outNode(StringInfo str, const void *obj)
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_RelAggInfo:
+ _outRelAggInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
@@ -4161,6 +4192,9 @@ outNode(StringInfo str, const void *obj)
case T_PlaceHolderInfo:
_outPlaceHolderInfo(str, obj);
break;
+ case T_GroupedVarInfo:
+ _outGroupedVarInfo(str, obj);
+ break;
case T_MinMaxAggInfo:
_outMinMaxAggInfo(str, obj);
break;
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
index 40558cdf6b..8d1de16cda 100644
--- a/src/backend/optimizer/README
+++ b/src/backend/optimizer/README
@@ -1154,3 +1154,72 @@ breaking down aggregation or grouping over a partitioned relation into
aggregation or grouping over its partitions is called partitionwise
aggregation. Especially when the partition keys match the GROUP BY clause,
this can be significantly faster than the regular method.
+
+Aggregate push-down
+-------------------
+
+The obvious way to evaluate aggregates is to evaluate the FROM clause of the
+SQL query (this is what query_planner does) and use the resulting paths as the
+input of Agg node. However, if the groups are large enough, it may be more
+efficient to apply the partial aggregation to the output of base relation
+scan, and finalize it when we have all relations of the query joined:
+
+ EXPLAIN
+ SELECT a.i, avg(b.y)
+ FROM a JOIN b ON b.j = a.i
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+ -> Index Only Scan using a_pkey on a
+ Index Cond: (i = b.j)
+
+Thus the join above the partial aggregate node receives fewer input rows, and
+so the number of outer-to-inner pairs of tuples to be checked can be
+significantly lower, which can in turn lead to considerably lower join cost.
+
+Note that there's often no GROUP BY expression to be used for the partial
+aggregation, so we use equivalence classes to derive grouping expression: in
+the example above, the grouping key "b.j" was derived from "a.i".
+
+Also note that in this case the partial aggregate uses the "b.j" as grouping
+column although the column does not appear in the query target list. The point
+is that "b.j" is needed to evaluate the join condition, and there's no other
+way for the partial aggregate to emit its values.
+
+Besides base relation, the aggregation can also be pushed down to join:
+
+ EXPLAIN
+ SELECT a.i, avg(b.y + c.v)
+ FROM a JOIN b ON b.j = a.i
+ JOIN c ON c.k = a.i
+ WHERE b.j = c.k GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (b.j = a.i)
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Hash Join
+ Hash Cond: (b.j = c.k)
+ -> Seq Scan on b
+ -> Hash
+ -> Seq Scan on c
+ -> Hash
+ -> Seq Scan on a
+
+Whether the Agg node is created out of base relation or out of join, it's
+added to a separate RelOptInfo that we call "grouped relation". Grouped
+relation can be joined to a non-grouped relation, which results in a grouped
+relation too. Join of two grouped relations does not seem to be very useful
+and is currently not supported.
+
+If query_planner produces a grouped relation that contains valid paths, these
+are simply added to the UPPERREL_PARTIAL_GROUP_AGG relation. Further
+processing of these paths then does not differ from processing of other
+partially grouped paths.
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index d984da25d7..84a918dc58 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -60,6 +60,7 @@ typedef struct pushdown_safety_info
/* These parameters are set by GUC */
bool enable_geqo = false; /* just in case GUC doesn't set it */
+bool enable_agg_pushdown;
int geqo_threshold;
int min_parallel_table_scan_size;
int min_parallel_index_scan_size;
@@ -73,6 +74,7 @@ join_search_hook_type join_search_hook = NULL;
static void set_base_rel_consider_startup(PlannerInfo *root);
static void set_base_rel_sizes(PlannerInfo *root);
+static void setup_base_grouped_rels(PlannerInfo *root);
static void set_base_rel_pathlists(PlannerInfo *root);
static void set_rel_size(PlannerInfo *root, RelOptInfo *rel,
Index rti, RangeTblEntry *rte);
@@ -124,6 +126,9 @@ static void set_result_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
+static void add_grouped_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, AggStrategy aggstrategy,
+ RelAggInfo *agg_info);
static RelOptInfo *make_rel_from_joinlist(PlannerInfo *root, List *joinlist);
static bool subquery_is_pushdown_safe(Query *subquery, Query *topquery,
pushdown_safety_info *safetyInfo);
@@ -184,6 +189,13 @@ make_one_rel(PlannerInfo *root, List *joinlist)
*/
set_base_rel_sizes(root);
+ /*
+ * Now that the sizes are known, we can estimate the sizes of the grouped
+ * relations.
+ */
+ if (root->grouped_var_list)
+ setup_base_grouped_rels(root);
+
/*
* We should now have size estimates for every actual table involved in
* the query, and we also know which if any have been deleted from the
@@ -324,6 +336,48 @@ set_base_rel_sizes(PlannerInfo *root)
}
}
+/*
+ * setup_based_grouped_rels
+ * For each "plain" relation build a grouped relation if aggregate pushdown
+ * is possible and if this relation is suitable for partial aggregation.
+ */
+static void
+setup_base_grouped_rels(PlannerInfo *root)
+{
+ Index rti;
+
+ for (rti = 1; rti < root->simple_rel_array_size; rti++)
+ {
+ RelOptInfo *brel = root->simple_rel_array[rti];
+ RelOptInfo *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /* there may be empty slots corresponding to non-baserel RTEs */
+ if (brel == NULL)
+ continue;
+
+ Assert(brel->relid == rti); /* sanity check on array */
+
+ /*
+ * The aggregate push-down feature only makes sense if there are
+ * multiple base rels in the query.
+ */
+ if (!bms_nonempty_difference(root->all_baserels, brel->relids))
+ continue;
+
+ /* ignore RTEs that are "other rels" */
+ if (brel->reloptkind != RELOPT_BASEREL)
+ continue;
+
+ rel_grouped = build_simple_grouped_rel(root, brel->relid, &agg_info);
+ if (rel_grouped)
+ {
+ /* Make the relation available for joining. */
+ add_grouped_rel(root, rel_grouped, agg_info);
+ }
+ }
+}
+
/*
* set_base_rel_pathlists
* Finds all paths available for scanning each base-relation entry.
@@ -496,8 +550,21 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
}
else
{
+ RelOptInfo *rel_grouped;
+ RelAggInfo *agg_info;
+
/* Plain relation */
set_plain_rel_pathlist(root, rel, rte);
+
+ /* Add paths to the grouped relation if one exists. */
+ rel_grouped = find_grouped_rel(root, rel->relids,
+ &agg_info);
+ if (rel_grouped)
+ {
+ generate_grouping_paths(root, rel_grouped, rel,
+ agg_info);
+ set_cheapest(rel_grouped);
+ }
}
break;
case RTE_SUBQUERY:
@@ -2943,6 +3010,80 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
}
}
+/*
+ * generate_grouping_paths
+ * Create partially aggregated paths and add them to grouped relation.
+ *
+ * "rel_plain" is base or join relation whose paths are not grouped.
+ */
+void
+generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
+ RelOptInfo *rel_plain, RelAggInfo *agg_info)
+{
+ ListCell *lc;
+
+ if (IS_DUMMY_REL(rel_plain))
+ {
+ mark_dummy_rel(rel_grouped);
+ return;
+ }
+
+ foreach(lc, rel_plain->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ /*
+ * Since the path originates from the non-grouped relation which is
+ * not aware of the aggregate push-down, we must ensure that it
+ * provides the correct input for aggregation.
+ */
+ path = (Path *) create_projection_path(root, rel_grouped, path,
+ agg_info->agg_input);
+
+ /*
+ * add_grouped_path() will check whether the path has suitable
+ * pathkeys.
+ */
+ add_grouped_path(root, rel_grouped, path, AGG_SORTED, agg_info);
+
+ /*
+ * Repeated creation of hash table (for new parameter values) should
+ * be possible, does not sound like a good idea in terms of
+ * efficiency.
+ */
+ if (path->param_info == NULL)
+ add_grouped_path(root, rel_grouped, path, AGG_HASHED, agg_info);
+ }
+
+ /* Could not generate any grouped paths? */
+ if (rel_grouped->pathlist == NIL)
+ mark_dummy_rel(rel_grouped);
+}
+
+/*
+ * Apply partial aggregation to a subpath and add the AggPath to the pathlist.
+ */
+static void
+add_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ AggStrategy aggstrategy, RelAggInfo *agg_info)
+{
+ Path *agg_path;
+
+
+ if (aggstrategy == AGG_HASHED)
+ agg_path = (Path *) create_agg_hashed_path(root, rel, subpath,
+ agg_info);
+ else if (aggstrategy == AGG_SORTED)
+ agg_path = (Path *) create_agg_sorted_path(root, rel, subpath,
+ agg_info);
+ else
+ elog(ERROR, "unexpected strategy %d", aggstrategy);
+
+ /* Add the grouped path to the list of grouped base paths. */
+ if (agg_path != NULL)
+ add_path(rel, (Path *) agg_path);
+}
+
/*
* make_rel_from_joinlist
* Build access paths using a "joinlist" to guide the join path search.
@@ -3084,6 +3225,7 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
for (lev = 2; lev <= levels_needed; lev++)
{
+ RelOptInfo *rel_grouped;
ListCell *lc;
/*
@@ -3120,6 +3262,11 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
+ /* The same for grouped relation if one exists. */
+ rel_grouped = find_grouped_rel(root, rel->relids, NULL);
+ if (rel_grouped)
+ set_cheapest(rel_grouped);
+
#ifdef OPTIMIZER_DEBUG
debug_print_rel(root, rel);
#endif
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index b976afb69d..fd33b2eeda 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4638,7 +4638,6 @@ set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel)
0,
JOIN_INNER,
NULL);
-
rel->rows = clamp_row_est(nrows);
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
@@ -5609,11 +5608,11 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
foreach(lc, target->exprs)
{
Node *node = (Node *) lfirst(lc);
+ int32 item_width;
if (IsA(node, Var))
{
Var *var = (Var *) node;
- int32 item_width;
/* We should not see any upper-level Vars here */
Assert(var->varlevelsup == 0);
@@ -5644,6 +5643,20 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
Assert(item_width > 0);
tuple_width += item_width;
}
+ else if (IsA(node, Aggref))
+ {
+ /*
+ * If the target is evaluated by AggPath, it'll care of cost
+ * estimate. If the target is above AggPath (typically target of a
+ * join relation that contains grouped relation), the cost of
+ * Aggref should not be accounted for again.
+ *
+ * On the other hand, width is always needed.
+ */
+ item_width = get_typavgwidth(exprType(node), exprTypmod(node));
+ Assert(item_width > 0);
+ tuple_width += item_width;
+ }
else
{
/*
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index b99cec00cb..81970fed15 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -2841,6 +2841,136 @@ is_redundant_derived_clause(RestrictInfo *rinfo, List *clauselist)
return false;
}
+/*
+ * translate_expression_to_rels
+ * If the appropriate equivalence classes exist, replace vars in
+ * gvi->gvexpr with vars whose varno is equal to relid. Return NULL if
+ * translation is not possible or needed.
+ *
+ * Note: Currently we only translate Var expressions. This is subject to
+ * change as the aggregate push-down feature gets enhanced.
+ */
+GroupedVarInfo *
+translate_expression_to_rel(PlannerInfo *root, GroupedVarInfo *gvi,
+ Index relid)
+{
+ Var *var;
+ ListCell *l1;
+ bool found_orig = false;
+ Var *var_translated = NULL;
+ GroupedVarInfo *result;
+
+ /* Can't do anything w/o equivalence classes. */
+ if (root->eq_classes == NIL)
+ return NULL;
+
+ var = castNode(Var, gvi->gvexpr);
+
+ /*
+ * Do we need to translate the var?
+ */
+ if (var->varno == relid)
+ return NULL;
+
+ /*
+ * Find the replacement var.
+ */
+ foreach(l1, root->eq_classes)
+ {
+ EquivalenceClass *ec = lfirst_node(EquivalenceClass, l1);
+ ListCell *l2;
+
+ /* TODO Check if any other EC kind should be ignored. */
+ if (ec->ec_has_volatile || ec->ec_below_outer_join || ec->ec_broken)
+ continue;
+
+ /* Single-element EC can hardly help in translations. */
+ if (list_length(ec->ec_members) == 1)
+ continue;
+
+ /*
+ * Collect all vars of this EC and their varnos.
+ *
+ * ec->ec_relids does not help because we're only interested in a
+ * subset of EC members.
+ */
+ foreach(l2, ec->ec_members)
+ {
+ EquivalenceMember *em = lfirst_node(EquivalenceMember, l2);
+ Var *ec_var;
+
+ /*
+ * The grouping expressions derived here are used to evaluate
+ * possibility to push aggregation down to RELOPT_BASEREL or
+ * RELOPT_JOINREL relations, and to construct reltargets for the
+ * grouped rels. We're not interested at the moment whether the
+ * relations do have children.
+ */
+ if (em->em_is_child)
+ continue;
+
+ if (!IsA(em->em_expr, Var))
+ continue;
+
+ ec_var = castNode(Var, em->em_expr);
+ if (equal(ec_var, var))
+ found_orig = true;
+ else if (ec_var->varno == relid)
+ var_translated = ec_var;
+
+ if (found_orig && var_translated)
+ {
+ /*
+ * The replacement Var must have the same data type, otherwise
+ * the values are not guaranteed to be grouped in the same way
+ * as values of the original Var.
+ */
+ if (ec_var->vartype != var->vartype)
+ return NULL;
+
+ break;
+ }
+ }
+
+ if (found_orig)
+ {
+ /*
+ * The same expression probably does not exist in multiple ECs.
+ */
+ if (var_translated == NULL)
+ {
+ /*
+ * Failed to translate the expression.
+ */
+ return NULL;
+ }
+ else
+ {
+ /* Success. */
+ break;
+ }
+ }
+ else
+ {
+ /*
+ * Vars of the requested relid can be in the next ECs too.
+ */
+ var_translated = NULL;
+ }
+ }
+
+ if (!found_orig)
+ return NULL;
+
+ result = makeNode(GroupedVarInfo);
+ memcpy(result, gvi, sizeof(GroupedVarInfo));
+
+ result->gv_eval_at = bms_make_singleton(relid);
+ result->gvexpr = (Expr *) var_translated;
+
+ return result;
+}
+
/*
* is_redundant_with_indexclauses
* Test whether rinfo is redundant with any clause in the IndexClause
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 2d343cd293..1b0dea491a 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -21,6 +21,7 @@
#include "optimizer/paths.h"
#include "partitioning/partbounds.h"
#include "utils/memutils.h"
+#include "utils/selfuncs.h"
static void make_rels_by_clause_joins(PlannerInfo *root,
@@ -35,6 +36,10 @@ static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
+static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1,
+ RelOptInfo *rel2,
+ RelAggInfo *agg_info,
+ RelOptInfo *rel_agg_input);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist);
@@ -669,21 +674,20 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
return true;
}
-
/*
- * make_join_rel
- * Find or create a join RelOptInfo that represents the join of
- * the two given rels, and add to it path information for paths
- * created with the two rels as outer and inner rel.
- * (The join rel may already contain paths generated from other
- * pairs of rels that add up to the same set of base rels.)
+ * make_join_rel_common
+ * The workhorse of make_join_rel().
+ *
+ * 'agg_info' contains the reltarget of grouped relation and everything we
+ * need to aggregate the join result. If NULL, then the join relation should
+ * not be grouped.
*
- * NB: will return NULL if attempted join is not valid. This can happen
- * when working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
+ * 'rel_agg_input' describes the AggPath input relation if the join output
+ * should be aggregated. If NULL is passed, do not aggregate the join output.
*/
-RelOptInfo *
-make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+static RelOptInfo *
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ RelAggInfo *agg_info, RelOptInfo *rel_agg_input)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -744,7 +748,7 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* goes with this particular joining.
*/
joinrel = build_join_rel(root, joinrelids, rel1, rel2, sjinfo,
- &restrictlist);
+ &restrictlist, agg_info);
/*
* If we've already proven this join is empty, we needn't consider any
@@ -757,14 +761,175 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
}
/* Add paths to the join relation. */
- populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
- restrictlist);
+ if (rel_agg_input == NULL)
+ {
+ /*
+ * Simply join the input relations, whether both are plain or one of
+ * them is grouped.
+ */
+ populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
+ restrictlist);
+ }
+ else
+ {
+ /* The join relation is grouped. */
+ Assert(agg_info != NULL);
+
+ /*
+ * Apply partial aggregation to the paths of rel_agg_input and add the
+ * resulting paths to joinrel.
+ */
+ generate_grouping_paths(root, joinrel, rel_agg_input, agg_info);
+ }
bms_free(joinrelids);
return joinrel;
}
+/*
+ * make_join_rel_combined
+ * Join grouped relation to non-grouped one.
+ */
+static void
+make_join_rel_combined(PlannerInfo *root, RelOptInfo *rel1,
+ RelOptInfo *rel2,
+ RelAggInfo *agg_info)
+{
+ RelOptInfo *rel1_grouped;
+ RelOptInfo *rel2_grouped;
+ bool rel1_grouped_useful = false;
+ bool rel2_grouped_useful = false;
+
+ /* Retrieve the grouped relations. */
+ rel1_grouped = find_grouped_rel(root, rel1->relids, NULL);
+ rel2_grouped = find_grouped_rel(root, rel2->relids, NULL);
+
+ /*
+ * Dummy rel may indicate a join relation that is able to generate grouped
+ * paths as such (i.e. it has valid agg_info), but for which the path
+ * actually could not be created (e.g. only AGG_HASHED strategy was
+ * possible but work_mem was not sufficient for hash table).
+ */
+ rel1_grouped_useful = rel1_grouped != NULL && !IS_DUMMY_REL(rel1_grouped);
+ rel2_grouped_useful = rel2_grouped != NULL && !IS_DUMMY_REL(rel2_grouped);
+
+ /* Nothing to do if there's no grouped relation. */
+ if (!rel1_grouped_useful && !rel2_grouped_useful)
+ return;
+
+ /*
+ * At maximum one input rel can be grouped (here we don't care if any rel
+ * is eventually dummy, the existence of grouped rel indicates that
+ * aggregates can be pushed down to it). If both were grouped, then
+ * grouping of one side would change the occurrence of the other side's
+ * aggregate transient states on the input of the final aggregation. This
+ * can be handled by adjusting the transient states, but it's not worth
+ * the effort because it's hard to find a use case for this kind of join.
+ *
+ * XXX If the join of two grouped rels is implemented someday, note that
+ * both rels can have aggregates, so it'd be hard to join grouped rel to
+ * non-grouped here: 1) such a "mixed join" would require a special
+ * target, 2) both AGGSPLIT_FINAL_DESERIAL and AGGSPLIT_SIMPLE aggregates
+ * could appear in the target of the final aggregation node, originating
+ * from the grouped and the non-grouped input rel respectively.
+ */
+ if (rel1_grouped && rel2_grouped)
+ return;
+
+ if (rel1_grouped_useful)
+ make_join_rel_common(root, rel1_grouped, rel2, agg_info, NULL);
+ else if (rel2_grouped_useful)
+ make_join_rel_common(root, rel1, rel2_grouped, agg_info, NULL);
+}
+
+/*
+ * make_join_rel
+ * Find or create a join RelOptInfo that represents the join of
+ * the two given rels, and add to it path information for paths
+ * created with the two rels as outer and inner rel.
+ * (The join rel may already contain paths generated from other
+ * pairs of rels that add up to the same set of base rels.)
+ *
+ * In addition to creating an ordinary join relation, try to create a
+ * grouped one. There are two strategies to achieve that: join a grouped
+ * relation to plain one, or join two plain relations and apply partial
+ * aggregation to the result.
+ *
+ * NB: will return NULL if attempted join is not valid. This can happen when
+ * working with outer joins, or with IN or EXISTS clauses that have been
+ * turned into joins. Besides that, NULL is also returned if caller is
+ * interested in a grouped relation but it could not be created.
+ *
+ * Only the plain relation is returned; if grouped relation exists, it can be
+ * retrieved using find_grouped_rel().
+ */
+RelOptInfo *
+make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+{
+ Relids joinrelids;
+ RelAggInfo *agg_info = NULL;
+ RelOptInfo *joinrel,
+ *joinrel_plain;
+
+ /* 1) form the plain join. */
+ joinrel = make_join_rel_common(root, rel1, rel2, NULL, NULL);
+ joinrel_plain = joinrel;
+
+ if (joinrel_plain == NULL)
+ return joinrel_plain;
+
+ /*
+ * We're done if there are no grouping expressions nor aggregates.
+ */
+ if (root->grouped_var_list == NIL)
+ return joinrel_plain;
+
+ joinrelids = bms_union(rel1->relids, rel2->relids);
+ joinrel = find_grouped_rel(root, joinrelids, &agg_info);
+
+ if (joinrel != NULL)
+ {
+ /*
+ * If the same grouped joinrel was already formed, just with the base
+ * rels divided between rel1 and rel2 in a different way, the matching
+ * agg_info should already be there.
+ */
+ Assert(agg_info != NULL);
+ }
+ else
+ {
+ /*
+ * agg_info must be created from scratch.
+ */
+ agg_info = create_rel_agg_info(root, joinrel_plain);
+
+ /* Cannot we build grouped join? */
+ if (agg_info == NULL)
+ return joinrel_plain;
+
+ /*
+ * The number of aggregate input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ agg_info->input_rows = joinrel_plain->rows;
+ }
+
+ /*
+ * 2) join two plain rels and aggregate the join paths. Aggregate
+ * push-down only makes sense if the join is not the top-level one.
+ */
+ if (bms_nonempty_difference(root->all_baserels, joinrelids))
+ make_join_rel_common(root, rel1, rel2, agg_info, joinrel_plain);
+
+ /*
+ * 3) combine plain and grouped relations.
+ */
+ make_join_rel_combined(root, rel1, rel2, agg_info);
+
+ return joinrel_plain;
+}
+
/*
* populate_joinrel_with_paths
* Add paths to the given joinrel for given pair of joining relations. The
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index e978b491f6..b10cdc8368 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -14,6 +14,7 @@
*/
#include "postgres.h"
+#include "access/nbtree.h"
#include "catalog/pg_class.h"
#include "catalog/pg_type.h"
#include "nodes/makefuncs.h"
@@ -33,6 +34,7 @@
#include "parser/analyze.h"
#include "rewrite/rewriteManip.h"
#include "utils/lsyscache.h"
+#include "utils/typcache.h"
/* These parameters are set by GUC */
int from_collapse_limit;
@@ -47,6 +49,8 @@ typedef struct PostponedQual
} PostponedQual;
+static void create_aggregate_grouped_var_infos(PlannerInfo *root);
+static void create_grouping_expr_grouped_var_infos(PlannerInfo *root);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
@@ -272,6 +276,292 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
}
}
+/*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate as well as for
+ * each possible grouping expression.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+extern void
+setup_aggregate_pushdown(PlannerInfo *root)
+{
+ ListCell *lc;
+
+ /*
+ * Isn't user interested in the aggregate push-down feature?
+ */
+ if (!enable_agg_pushdown)
+ return;
+
+ /* The feature can only be applied to grouped aggregation. */
+ if (!root->parse->groupClause)
+ return;
+
+ /*
+ * Grouping sets require multiple different groupings but the base
+ * relation can only generate one.
+ */
+ if (root->parse->groupingSets)
+ return;
+
+ /*
+ * SRF is not allowed in the aggregate argument and we don't even want it
+ * in the GROUP BY clause, so forbid it in general. It needs to be
+ * analyzed if evaluation of a GROUP BY clause containing SRF below the
+ * query targetlist would be correct. Currently it does not seem to be an
+ * important use case.
+ */
+ if (root->parse->hasTargetSRFs)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_aggregate_grouped_var_infos(root);
+
+ /* Isn't there any aggregate to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Create GroupedVarInfo per grouping expression. */
+ create_grouping_expr_grouped_var_infos(root);
+
+ /* Isn't there any useful grouping expression for aggregate push-down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /*
+ * Now that we know that grouping can be pushed down, search for the
+ * maximum sortgroupref. The base relations may need it if extra grouping
+ * expressions get added to them.
+ */
+ Assert(root->max_sortgroupref == 0);
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc);
+
+ if (te->ressortgroupref > root->max_sortgroupref)
+ root->max_sortgroupref = te->ressortgroupref;
+ }
+}
+
+/*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ */
+static void
+create_aggregate_grouped_var_infos(PlannerInfo *root)
+{
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES);
+
+ /*
+ * Although GroupingFunc is related to root->parse->groupingSets, this
+ * field does not necessarily reflect its presence.
+ */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, GroupingFunc))
+ return;
+ }
+
+ /*
+ * Aggregates within the HAVING clause need to be processed in the same
+ * way as those in the main targetlist.
+ *
+ * Note that the contained aggregates will be pushed down, but the
+ * containing HAVING clause must be ignored until the aggregation is
+ * finalized.
+ */
+ if (root->parse->havingQual != NULL)
+ {
+ List *having_exprs;
+
+ having_exprs = pull_var_clause((Node *) root->parse->havingQual,
+ PVC_INCLUDE_AGGREGATES);
+ if (having_exprs != NIL)
+ tlist_exprs = list_concat(tlist_exprs, having_exprs);
+ }
+
+ if (tlist_exprs == NIL)
+ return;
+
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ /*
+ * tlist_exprs may also contain Vars, but we only need Aggrefs.
+ */
+ if (IsA(expr, Var))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct)
+ {
+ /*
+ * Aggregation push-down is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ gvi = lfirst_node(GroupedVarInfo, lc2);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ gvi = makeNode(GroupedVarInfo);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ gvi->gv_eval_at = NULL;
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+}
+
+/*
+ * Create GroupedVarInfo for each expression usable as grouping key.
+ *
+ * In addition to the expressions of the query targetlist, group_pathkeys is
+ * also considered the source of grouping expressions. That increases the
+ * chance to get the relation output grouped.
+ */
+static void
+create_grouping_expr_grouped_var_infos(PlannerInfo *root)
+{
+ ListCell *l1,
+ *l2;
+ List *exprs = NIL;
+ List *sortgrouprefs = NIL;
+
+ /*
+ * Make sure GroupedVarInfo exists for each expression usable as grouping
+ * key.
+ */
+ foreach(l1, root->parse->groupClause)
+ {
+ SortGroupClause *sgClause;
+ TargetEntry *te;
+ Index sortgroupref;
+ TypeCacheEntry *tce;
+ Oid equalimageproc;
+
+ sgClause = lfirst_node(SortGroupClause, l1);
+ te = get_sortgroupclause_tle(sgClause, root->processed_tlist);
+ sortgroupref = te->ressortgroupref;
+
+ Assert(sortgroupref > 0);
+
+ /*
+ * Non-zero sortgroupref does not necessarily imply grouping
+ * expression: data can also be sorted by aggregate.
+ */
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ /*
+ * The aggregate push-down feature currently supports only plain Vars
+ * as grouping expressions.
+ */
+ if (!IsA(te->expr, Var))
+ {
+ root->grouped_var_list = NIL;
+ return;
+ }
+
+ /*
+ * Aggregate push-down is only possible if equality of grouping keys
+ * per the equality operator implies bitwise equality. Otherwise, if
+ * we put keys of different byte images into the same group, we lose
+ * some information that may be needed to evaluate join clauses above
+ * the pushed-down aggregate node, or the WHERE clause.
+ *
+ * For example, the NUMERIC data type is not supported because values
+ * that fall into the same group according to the equality operator
+ * (e.g. 0 and 0.0) can have different scale.
+ */
+ tce = lookup_type_cache(exprType((Node *) te->expr),
+ TYPECACHE_BTREE_OPFAMILY);
+ if (!OidIsValid(tce->btree_opf) ||
+ !OidIsValid(tce->btree_opintype))
+ goto fail;
+
+ equalimageproc = get_opfamily_proc(tce->btree_opf,
+ tce->btree_opintype,
+ tce->btree_opintype,
+ BTEQUALIMAGE_PROC);
+ if (!OidIsValid(equalimageproc) ||
+ !DatumGetBool(OidFunctionCall1Coll(equalimageproc,
+ tce->typcollation,
+ ObjectIdGetDatum(tce->btree_opintype))))
+ goto fail;
+
+ exprs = lappend(exprs, te->expr);
+ sortgrouprefs = lappend_int(sortgrouprefs, sortgroupref);
+ }
+
+ /*
+ * Construct GroupedVarInfo for each expression.
+ */
+ forboth(l1, exprs, l2, sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, l1);
+ int sortgroupref = lfirst_int(l2);
+ GroupedVarInfo *gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvexpr = (Expr *) copyObject(var);
+ gvi->sortgroupref = sortgroupref;
+
+ /* Find out where the expression should be evaluated. */
+ gvi->gv_eval_at = bms_make_singleton(var->varno);
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ return;
+
+fail:
+ root->grouped_var_list = NIL;
+}
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 5fa33ec200..cb82d3b8de 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -66,6 +66,7 @@ query_planner(PlannerInfo *root,
* here.
*/
root->join_rel_list = makeNode(RelInfoList);
+ root->agg_info_list = makeNode(RelInfoList);
root->join_rel_level = NULL;
root->join_cur_level = 0;
root->canon_pathkeys = NIL;
@@ -74,6 +75,7 @@ query_planner(PlannerInfo *root,
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
@@ -252,6 +254,16 @@ query_planner(PlannerInfo *root,
*/
extract_restriction_or_clauses(root);
+ /*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, setup
+ * root->grouped_var_list.
+ *
+ * The base relations should be fully initialized now, so that we have
+ * enough info to decide whether grouping is possible.
+ */
+ setup_aggregate_pushdown(root);
+
/*
* Now expand appendrels by adding "otherrels" for their children. We
* delay this to the end so that we have as much information as possible
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 357850624c..6b6c7e396f 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -625,6 +625,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
memset(root->upper_rels, 0, sizeof(root->upper_rels));
memset(root->upper_targets, 0, sizeof(root->upper_targets));
root->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
root->grouping_map = NULL;
root->minmax_aggs = NIL;
root->qual_security_level = 0;
@@ -3893,7 +3894,8 @@ create_grouping_paths(PlannerInfo *root,
else
extra.patype = PARTITIONWISE_AGGREGATE_NONE;
- create_ordinary_grouping_paths(root, input_rel, grouped_rel,
+ create_ordinary_grouping_paths(root, input_rel,
+ grouped_rel,
agg_costs, gd, &extra,
&partially_grouped_rel);
}
@@ -4100,11 +4102,11 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
bool force_rel_creation;
/*
- * If we're doing partitionwise aggregation at this level, force
- * creation of a partially_grouped_rel so we can add partitionwise
- * paths to it.
+ * If we're doing partitionwise aggregation at this level or if
+ * aggregate push-down succeeded to create some paths, force creation
+ * of a partially_grouped_rel so we can add the related paths to it.
*/
- force_rel_creation = (patype == PARTITIONWISE_AGGREGATE_PARTIAL);
+ force_rel_creation = patype == PARTITIONWISE_AGGREGATE_PARTIAL;
partially_grouped_rel =
create_partial_grouping_paths(root,
@@ -4137,10 +4139,14 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
/* Gather any partially grouped partial paths. */
if (partially_grouped_rel && partially_grouped_rel->partial_pathlist)
- {
gather_grouping_paths(root, partially_grouped_rel);
+
+ /*
+ * The non-partial paths can come either from the Gather above or from
+ * aggregate push-down.
+ */
+ if (partially_grouped_rel && partially_grouped_rel->pathlist)
set_cheapest(partially_grouped_rel);
- }
/*
* Estimate number of groups.
@@ -6869,6 +6875,13 @@ create_partial_grouping_paths(PlannerInfo *root,
bool can_hash = (extra->flags & GROUPING_CAN_USE_HASH) != 0;
bool can_sort = (extra->flags & GROUPING_CAN_USE_SORT) != 0;
+ /*
+ * The output relation could have been already created due to aggregate
+ * push-down.
+ */
+ partially_grouped_rel = find_grouped_rel(root, input_rel->relids, NULL);
+ Assert(enable_agg_pushdown || partially_grouped_rel == NULL);
+
/*
* Consider whether we should generate partially aggregated non-partial
* paths. We can only do this if we have a non-partial path, and only if
@@ -6895,16 +6908,18 @@ create_partial_grouping_paths(PlannerInfo *root,
*/
if (cheapest_total_path == NULL &&
cheapest_partial_path == NULL &&
- !force_rel_creation)
+ !force_rel_creation &&
+ partially_grouped_rel == NULL)
return NULL;
/*
* Build a new upper relation to represent the result of partially
* aggregating the rows from the input relation.
*/
- partially_grouped_rel = fetch_upper_rel(root,
- UPPERREL_PARTIAL_GROUP_AGG,
- grouped_rel->relids);
+ if (partially_grouped_rel == NULL)
+ partially_grouped_rel = fetch_upper_rel(root,
+ UPPERREL_PARTIAL_GROUP_AGG,
+ grouped_rel->relids);
partially_grouped_rel->consider_parallel =
grouped_rel->consider_parallel;
partially_grouped_rel->reloptkind = grouped_rel->reloptkind;
@@ -6918,10 +6933,14 @@ create_partial_grouping_paths(PlannerInfo *root,
* emit the same tlist as regular aggregate paths, because (1) we must
* include Vars and Aggrefs needed in HAVING, which might not appear in
* the result tlist, and (2) the Aggrefs must be set in partial mode.
+ *
+ * If the target was already created for the sake of aggregate push-down,
+ * it should be compatible with what we'd create here.
*/
- partially_grouped_rel->reltarget =
- make_partial_grouping_target(root, grouped_rel->reltarget,
- extra->havingQual);
+ if (partially_grouped_rel->reltarget->exprs == NIL)
+ partially_grouped_rel->reltarget =
+ make_partial_grouping_target(root, grouped_rel->reltarget,
+ extra->havingQual);
if (!extra->partial_costs_set)
{
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index baefe0e946..026c098d11 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -2457,6 +2457,39 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, Aggref))
+ {
+ Aggref *aggref = castNode(Aggref, node);
+
+ /*
+ * The upper plan targetlist can contain Aggref whose value has
+ * already been evaluated by the subplan. However this can only happen
+ * with specific value of aggsplit.
+ */
+ if (aggref->aggsplit == AGGSPLIT_INITIAL_SERIAL)
+ {
+ /* See if the Aggref has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ }
+
+ /* No referent found for Aggref */
+ elog(ERROR, "Aggref not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 14521728c6..d34065d61e 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -923,6 +923,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
memset(subroot->upper_rels, 0, sizeof(subroot->upper_rels));
memset(subroot->upper_targets, 0, sizeof(subroot->upper_targets));
subroot->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
subroot->grouping_map = NULL;
subroot->minmax_aggs = NIL;
subroot->qual_security_level = 0;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index e845a4b1ae..437d6d3f42 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2528,8 +2528,7 @@ create_projection_path(PlannerInfo *root,
pathnode->path.pathtype = T_Result;
pathnode->path.parent = rel;
pathnode->path.pathtarget = target;
- /* For now, assume we are above any joins, so no parameterization */
- pathnode->path.param_info = NULL;
+ pathnode->path.param_info = subpath->param_info;
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe &&
@@ -3020,6 +3019,146 @@ create_agg_path(PlannerInfo *root,
return pathnode;
}
+/*
+ * Apply AGG_SORTED aggregation path to subpath if it's suitably sorted.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+AggPath *
+create_agg_sorted_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ RelAggInfo *agg_info)
+{
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ ListCell *lc1;
+ List *key_subset = NIL;
+ AggPath *result = NULL;
+
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, (Node *) agg_exprs, aggsplit, &agg_costs);
+
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until the
+ * final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath, target,
+ AGG_SORTED, aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+
+ /* The agg path should require no fewer parameters than the plain one. */
+ result->path.param_info = subpath->param_info;
+
+ return result;
+}
+
+/*
+ * Apply AGG_HASHED aggregation to subpath.
+ */
+AggPath *
+create_agg_hashed_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, RelAggInfo *agg_info)
+{
+ bool can_hash;
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ double hashaggtablesize;
+ Query *parse = root->parse;
+ AggPath *result = NULL;
+
+ /* Do not try to create hash table for each parameter value. */
+ Assert(subpath->param_info == NULL);
+
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, agg_exprs, aggsplit, &agg_costs);
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until
+ * the final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath,
+ target,
+ AGG_HASHED,
+ aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+ }
+ }
+
+ return result;
+}
+
/*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index a95e6364ae..7460794b46 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -18,17 +18,23 @@
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
+#include "catalog/pg_class_d.h"
+#include "catalog/pg_constraint.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
+#include "optimizer/optimizer.h"
#include "optimizer/inherit.h"
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/placeholder.h"
#include "optimizer/plancat.h"
+#include "optimizer/planner.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+#include "parser/parse_oper.h"
#include "utils/hsearch.h"
+#include "utils/selfuncs.h"
#include "utils/lsyscache.h"
@@ -76,6 +82,11 @@ static void build_child_join_reltarget(PlannerInfo *root,
RelOptInfo *childrel,
int nappinfos,
AppendRelInfo **appinfos);
+static bool init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p);
+static bool is_var_in_aggref_only(PlannerInfo *root, Var *var);
+static bool is_var_needed_by_join(PlannerInfo *root, Var *var, RelOptInfo *rel);
/*
@@ -370,6 +381,109 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
return rel;
}
+/*
+ * build_simple_grouped_rel
+ * Construct a new RelOptInfo for a grouped base relation out of an
+ * existing non-grouped relation. On success, pointer to the corresponding
+ * RelAggInfo is stored in *agg_info_p in addition to returning the grouped
+ * relation.
+ */
+RelOptInfo *
+build_simple_grouped_rel(PlannerInfo *root, int relid,
+ RelAggInfo **agg_info_p)
+{
+ RangeTblEntry *rte;
+ RelOptInfo *rel_plain,
+ *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /* Isn't there any grouping expression to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return NULL;
+
+ rel_plain = root->simple_rel_array[relid];
+
+ /* Caller should only pass rti that represents base relation. */
+ Assert(rel_plain != NULL);
+
+ /*
+ * Not all RTE kinds are supported when grouping is considered.
+ *
+ * TODO Consider relaxing some of these restrictions.
+ */
+ rte = root->simple_rte_array[rel_plain->relid];
+ if (rte->rtekind != RTE_RELATION ||
+ rte->relkind == RELKIND_FOREIGN_TABLE ||
+ rte->tablesample != NULL)
+ return NULL;
+
+ /*
+ * Grouped append relation is not supported yet.
+ */
+ if (rte->inh)
+ return NULL;
+
+ /*
+ * Currently we do not support child relations ("other rels").
+ */
+ if (rel_plain->reloptkind != RELOPT_BASEREL)
+ return NULL;
+
+ /*
+ * Prepare the information we need for aggregation of the rel contents.
+ */
+ agg_info = create_rel_agg_info(root, rel_plain);
+ if (agg_info == NULL)
+ return NULL;
+
+ /*
+ * TODO Consider if 1) a flat copy is o.k., 2) it's safer in terms of
+ * adding new fields to RelOptInfo) to copy everything and then reset some
+ * fields, or to zero the structure and copy individual fields.
+ */
+ rel_grouped = makeNode(RelOptInfo);
+ memcpy(rel_grouped, rel_plain, sizeof(RelOptInfo));
+
+ /*
+ * Note on consider_startup: while the AGG_HASHED strategy needs the whole
+ * relation, AGG_SORTED does not. Therefore we do not force
+ * consider_startup to false.
+ */
+
+ /*
+ * Set the appropriate target for grouped paths.
+ *
+ * reltarget should match the target of partially aggregated paths.
+ */
+ rel_grouped->reltarget = agg_info->target;
+
+ /*
+ * Grouped paths must not be mixed with the plain ones.
+ */
+ rel_grouped->pathlist = NIL;
+ rel_grouped->partial_pathlist = NIL;
+ rel_grouped->cheapest_startup_path = NULL;
+ rel_grouped->cheapest_total_path = NULL;
+ rel_grouped->cheapest_unique_path = NULL;
+ rel_grouped->cheapest_parameterized_paths = NIL;
+
+ /*
+ * The number of aggregation input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ agg_info->input_rows = rel_plain->rows;
+
+ /*
+ * The number of output rows is supposedly different (lower) due to
+ * grouping.
+ */
+ rel_grouped->rows = estimate_num_groups(root, agg_info->group_exprs,
+ agg_info->input_rows, NULL);
+
+ *agg_info_p = agg_info;
+ return rel_grouped;
+}
+
/*
* find_base_rel
* Find a base or other relation entry, which must already exist.
@@ -484,9 +598,13 @@ find_rel_info(RelInfoList *list, Relids relids)
void *item = lfirst(l);
Relids item_relids = NULL;
- Assert(IsA(item, RelOptInfo));
+ Assert(IsA(item, RelOptInfo) || IsA(item, RelAggInfo));
+
+ if (IsA(item, RelOptInfo))
+ item_relids = ((RelOptInfo *) item)->relids;
+ else if (IsA(item, RelAggInfo))
+ item_relids = ((RelAggInfo *) item)->relids;
- item_relids = ((RelOptInfo *) item)->relids;
if (bms_equal(item_relids, relids))
return item;
}
@@ -514,7 +632,7 @@ find_join_rel(PlannerInfo *root, Relids relids)
static void
add_rel_info(RelInfoList *list, void *data)
{
- Assert(IsA(data, RelOptInfo));
+ Assert(IsA(data, RelOptInfo) || IsA(data, RelAggInfo));
/* GEQO requires us to append the new joinrel to the end of the list! */
list->items = lappend(list->items, data);
@@ -526,7 +644,11 @@ add_rel_info(RelInfoList *list, void *data)
RelInfoEntry *hentry;
bool found;
- relids = ((RelOptInfo *) data)->relids;
+ if (IsA(data, RelOptInfo))
+ relids = ((RelOptInfo *) data)->relids;
+ else if (IsA(data, RelAggInfo))
+ relids = ((RelAggInfo *) data)->relids;
+
hentry = (RelInfoEntry *) hash_search(list->hash,
&relids,
HASH_ENTER,
@@ -547,6 +669,63 @@ add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
add_rel_info(root->join_rel_list, joinrel);
}
+/*
+ * add_grouped_rel
+ * Add grouped base or join relation to the list of grouped relations in
+ * the given PlannerInfo. Also add the corresponding RelAggInfo to
+ * agg_info_list.
+ */
+void
+add_grouped_rel(PlannerInfo *root, RelOptInfo *rel, RelAggInfo *agg_info)
+{
+ add_rel_info(&root->upper_rels[UPPERREL_PARTIAL_GROUP_AGG], rel);
+ add_rel_info(root->agg_info_list, agg_info);
+}
+
+/*
+ * find_grouped_rel
+ * Returns grouped relation entry (base or join relation) corresponding to
+ * 'relids' or NULL if none exists.
+ *
+ * If agg_info_p is a valid pointer, then pointer to RelAggInfo that
+ * corresponds to the relation returned is assigned to *agg_info_p.
+ *
+ * The call fetch_upper_rel(root, UPPERREL_PARTIAL_GROUP_AGG, ...) should
+ * return the same relation if it exists, however the behavior is different if
+ * the relation is not there. find_grouped_rel() should be used in
+ * query_planner() and subroutines.
+ */
+RelOptInfo *
+find_grouped_rel(PlannerInfo *root, Relids relids, RelAggInfo **agg_info_p)
+{
+ RelOptInfo *rel;
+
+ rel = (RelOptInfo *) find_rel_info(&root->upper_rels[UPPERREL_PARTIAL_GROUP_AGG],
+ relids);
+ if (rel == NULL)
+ {
+ if (agg_info_p)
+ *agg_info_p = NULL;
+
+ return NULL;
+ }
+
+ /* Is caller interested in RelAggInfo? */
+ if (agg_info_p)
+ {
+ RelAggInfo *agg_info;
+
+ agg_info = (RelAggInfo *) find_rel_info(root->agg_info_list, relids);
+
+ /* The relation exists, so the agg_info should be there too. */
+ Assert(agg_info != NULL);
+
+ *agg_info_p = agg_info;
+ }
+
+ return rel;
+}
+
/*
* set_foreign_rel_properties
* Set up foreign-join fields if outer and inner relation are foreign
@@ -609,6 +788,7 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* 'restrictlist_ptr': result variable. If not NULL, *restrictlist_ptr
* receives the list of RestrictInfo nodes that apply to this
* particular pair of joinable relations.
+ * 'agg_info' indicates that grouped join relation should be created.
*
* restrictlist_ptr makes the routine's API a little grotty, but it saves
* duplicated calculation of the restrictlist...
@@ -619,10 +799,12 @@ build_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr)
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info)
{
RelOptInfo *joinrel;
List *restrictlist;
+ bool grouped = agg_info != NULL;
/* This function should be used only for join between parents. */
Assert(!IS_OTHER_REL(outer_rel) && !IS_OTHER_REL(inner_rel));
@@ -630,7 +812,8 @@ build_join_rel(PlannerInfo *root,
/*
* See if we already have a joinrel for this set of base rels.
*/
- joinrel = find_join_rel(root, joinrelids);
+ joinrel = !grouped ? find_join_rel(root, joinrelids) :
+ find_grouped_rel(root, joinrelids, NULL);
if (joinrel)
{
@@ -725,9 +908,21 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ if (!grouped)
+ {
+ joinrel->reltarget = create_empty_pathtarget();
+ build_joinrel_tlist(root, joinrel, outer_rel);
+ build_joinrel_tlist(root, joinrel, inner_rel);
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ }
+ else
+ {
+ /*
+ * The target for grouped join should already have its cost and width
+ * computed, see create_rel_agg_info().
+ */
+ joinrel->reltarget = agg_info->target;
+ }
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -759,49 +954,75 @@ build_join_rel(PlannerInfo *root,
joinrel->has_eclass_joins = has_relevant_eclass_joinclause(root, joinrel);
/* Store the partition information. */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- sjinfo->jointype);
+ if (!grouped)
+ build_joinrel_partition_info(joinrel, outer_rel, inner_rel,
+ restrictlist, sjinfo->jointype);
- /*
- * Set estimates of the joinrel's size.
- */
- set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
- sjinfo, restrictlist);
+ if (!grouped)
+ {
+ /*
+ * Set estimates of the joinrel's size.
+ */
+ set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
+ sjinfo, restrictlist);
- /*
- * Set the consider_parallel flag if this joinrel could potentially be
- * scanned within a parallel worker. If this flag is false for either
- * inner_rel or outer_rel, then it must be false for the joinrel also.
- * Even if both are true, there might be parallel-restricted expressions
- * in the targetlist or quals.
- *
- * Note that if there are more than two rels in this relation, they could
- * be divided between inner_rel and outer_rel in any arbitrary way. We
- * assume this doesn't matter, because we should hit all the same baserels
- * and joinclauses while building up to this joinrel no matter which we
- * take; therefore, we should make the same decision here however we get
- * here.
- */
- if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
- is_parallel_safe(root, (Node *) restrictlist) &&
- is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
- joinrel->consider_parallel = true;
+ /*
+ * Set the consider_parallel flag if this joinrel could potentially be
+ * scanned within a parallel worker. If this flag is false for either
+ * inner_rel or outer_rel, then it must be false for the joinrel also.
+ * Even if both are true, there might be parallel-restricted
+ * expressions in the targetlist or quals.
+ *
+ * Note that if there are more than two rels in this relation, they
+ * could be divided between inner_rel and outer_rel in any arbitrary
+ * way. We assume this doesn't matter, because we should hit all the
+ * same baserels and joinclauses while building up to this joinrel no
+ * matter which we take; therefore, we should make the same decision
+ * here however we get here.
+ */
+ if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
+ is_parallel_safe(root, (Node *) restrictlist) &&
+ is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
+ joinrel->consider_parallel = true;
+ }
+ else
+ {
+ /*
+ * Grouping essentially changes the number of rows.
+ *
+ * XXX We do not distinguish whether two plain rels are joined and the
+ * result is aggregated, or the aggregation has been already applied
+ * to one of the input rels. Is this worth extra effort, e.g.
+ * maintaining a separate RelOptInfo for each case (one difficulty
+ * that would introduce is construction of AppendPath)?
+ */
+ joinrel->rows = estimate_num_groups(root, agg_info->group_exprs,
+ agg_info->input_rows, NULL);
+ }
/* Add the joinrel to the PlannerInfo. */
- add_join_rel(root, joinrel);
+ if (!grouped)
+ add_join_rel(root, joinrel);
+ else
+ add_grouped_rel(root, joinrel, agg_info);
/*
- * Also, if dynamic-programming join search is active, add the new joinrel
- * to the appropriate sublist. Note: you might think the Assert on number
- * of members should be for equality, but some of the level 1 rels might
- * have been joinrels already, so we can only assert <=.
+ * Also, if dynamic-programming join search is active, add the new
+ * joinrelset to the appropriate sublist. Note: you might think the
+ * Assert on number of members should be for equality, but some of the
+ * level 1 rels might have been joinrels already, so we can only assert
+ * <=.
+ *
+ * Do noting for grouped relation as it's stored aside from
+ * join_rel_level.
*/
- if (root->join_rel_level)
+ if (root->join_rel_level && !grouped)
{
Assert(root->join_cur_level > 0);
- Assert(root->join_cur_level <= bms_num_members(joinrel->relids));
+ Assert(root->join_cur_level <= bms_num_members(joinrelids));
root->join_rel_level[root->join_cur_level] =
- lappend(root->join_rel_level[root->join_cur_level], joinrel);
+ lappend(root->join_rel_level[root->join_cur_level],
+ joinrel);
}
return joinrel;
@@ -2059,3 +2280,621 @@ build_child_join_reltarget(PlannerInfo *root,
childrel->reltarget->cost.per_tuple = parentrel->reltarget->cost.per_tuple;
childrel->reltarget->width = parentrel->reltarget->width;
}
+
+/*
+ * Check if the relation can produce grouped paths and return the information
+ * it'll need for it. The passed relation is the non-grouped one which has the
+ * reltarget already constructed.
+ */
+RelAggInfo *
+create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel)
+{
+ List *gvis;
+ List *aggregates = NIL;
+ bool found_other_rel_agg;
+ ListCell *lc;
+ RelAggInfo *result;
+ PathTarget *agg_input;
+ PathTarget *target = NULL;
+ List *grp_exprs_extra = NIL;
+ List *group_clauses_final;
+ int i;
+
+ /*
+ * The function shouldn't have been called if there's no opportunity for
+ * aggregation push-down.
+ */
+ Assert(root->grouped_var_list != NIL);
+
+ /*
+ * The current implementation of aggregation push-down cannot handle
+ * PlaceHolderVar (PHV).
+ *
+ * If we knew that the PHV should be evaluated in this target (and of
+ * course, if its expression matched some Aggref argument), we'd just let
+ * init_grouping_targets add that Aggref. On the other hand, if we knew
+ * that the PHV is evaluated below the current rel, we could ignore it
+ * because the referencing Aggref would take care of propagation of the
+ * value to upper joins.
+ *
+ * The problem is that the same PHV can be evaluated in the target of the
+ * current rel or in that of lower rel --- depending on the input paths.
+ * For example, consider rel->relids = {A, B, C} and if ph_eval_at = {B,
+ * C}. Path "A JOIN (B JOIN C)" implies that the PHV is evaluated by the
+ * "(B JOIN C)", while path "(A JOIN B) JOIN C" evaluates the PHV itself.
+ */
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *expr = lfirst(lc);
+
+ if (IsA(expr, PlaceHolderVar))
+ return NULL;
+ }
+
+ if (IS_SIMPLE_REL(rel))
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return NULL;
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the Agg plan
+ * would receive different input at the base rel level.
+ *
+ * XXX For RELOPT_JOINREL, do not return if all the joins that can set any
+ * entry of the grouped target (do we need to postpone this check until
+ * the grouped target is available, and init_grouping_targets take care?)
+ * of this rel to NULL are provably below rel. (It's ok if rel is one of
+ * these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return NULL;
+
+ /*
+ * Use equivalence classes to generate additional grouping expressions for
+ * the current rel. Without these we might not be able to apply
+ * aggregation to the relation result set.
+ *
+ * It's important that create_grouping_expr_grouped_var_infos has
+ * processed the explicit grouping columns by now. If the grouping clause
+ * contains multiple expressions belonging to the same EC, the original
+ * (i.e. not derived) one should be preferred when we build grouping
+ * target for a relation. Otherwise we have a problem when trying to match
+ * target entries to grouping clauses during plan creation, see
+ * get_grouping_expression().
+ */
+ gvis = list_copy(root->grouped_var_list);
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+ int relid = -1;
+
+ /* Only interested in grouping expressions. */
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ while ((relid = bms_next_member(rel->relids, relid)) >= 0)
+ {
+ GroupedVarInfo *gvi_trans;
+
+ gvi_trans = translate_expression_to_rel(root, gvi, relid);
+ if (gvi_trans != NULL)
+ gvis = lappend(gvis, gvi_trans);
+ }
+ }
+
+ /*
+ * Check if some aggregates or grouping expressions can be evaluated in
+ * this relation's target, and collect all vars referenced by these
+ * aggregates / grouping expressions;
+ */
+ found_other_rel_agg = false;
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which includes
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * init_grouping_targets will handle plain Var grouping
+ * expressions because it needs to look them up in
+ * grouped_var_list anyway.
+ */
+ if (IsA(gvi->gvexpr, Var))
+ continue;
+
+ /*
+ * Currently, GroupedVarInfo only handles Vars and Aggrefs.
+ */
+ Assert(IsA(gvi->gvexpr, Aggref));
+
+ gvi->agg_partial = (Aggref *) copyObject(gvi->gvexpr);
+ mark_partial_aggref(gvi->agg_partial, AGGSPLIT_INITIAL_SERIAL);
+
+ /*
+ * Accept the aggregate.
+ */
+ aggregates = lappend(aggregates, gvi);
+ }
+ else if (IsA(gvi->gvexpr, Aggref))
+ {
+ /*
+ * Remember that there is at least one aggregate expression that
+ * needs something else than this rel.
+ */
+ found_other_rel_agg = true;
+
+ /*
+ * This condition effectively terminates creation of the
+ * RelAggInfo, so there's no reason to check the next
+ * GroupedVarInfo.
+ */
+ break;
+ }
+ }
+
+ /*
+ * Grouping makes little sense w/o aggregate function and w/o grouping
+ * expressions.
+ */
+ if (aggregates == NIL)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Give up if some other aggregate(s) need relations other than the
+ * current one.
+ *
+ * If the aggregate needs the current rel plus anything else, then the
+ * problem is that grouping of the current relation could make some input
+ * variables unavailable for the "higher aggregate", and it'd also
+ * decrease the number of input rows the "higher aggregate" receives.
+ *
+ * If the aggregate does not even need the current rel, then neither the
+ * current rel nor anything else should be grouped because we do not
+ * support join of two grouped relations.
+ */
+ if (found_other_rel_agg)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Create target for grouped paths as well as one for the input paths of
+ * the aggregation paths.
+ */
+ target = create_empty_pathtarget();
+ agg_input = create_empty_pathtarget();
+
+ /*
+ * Cannot suitable targets for the aggregation push-down be derived?
+ */
+ if (!init_grouping_targets(root, rel, target, agg_input, gvis,
+ &grp_exprs_extra))
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ list_free(gvis);
+
+ /*
+ * Aggregation push-down makes no sense w/o grouping expressions.
+ */
+ if ((list_length(target->exprs) + list_length(grp_exprs_extra)) == 0)
+ return NULL;
+
+ group_clauses_final = root->parse->groupClause;
+
+ /*
+ * If the aggregation target should have extra grouping expressions (in
+ * order to emit input vars for join conditions), add them now. This step
+ * includes assignment of tleSortGroupRef's which we can generate now.
+ */
+ if (list_length(grp_exprs_extra) > 0)
+ {
+ Index sortgroupref;
+
+ /*
+ * We'll have to add some clauses, but query group clause must be
+ * preserved.
+ */
+ group_clauses_final = list_copy(group_clauses_final);
+
+ /*
+ * Always start at root->max_sortgroupref. The extra grouping
+ * expressions aren't used during the final aggregation, so the
+ * sortgroupref values don't need to be unique across the query. Thus
+ * we don't have to increase root->max_sortgroupref, which makes
+ * recognition of the extra grouping expressions pretty easy.
+ */
+ sortgroupref = root->max_sortgroupref;
+
+ /*
+ * Generate the SortGroupClause's and add the expressions to the
+ * target.
+ */
+ foreach(lc, grp_exprs_extra)
+ {
+ Var *var = lfirst_node(Var, lc);
+ SortGroupClause *cl = makeNode(SortGroupClause);
+
+ /*
+ * Initialize the SortGroupClause.
+ *
+ * As the final aggregation will not use this grouping expression,
+ * we don't care whether sortop is < or >. The value of
+ * nulls_first should not matter for the same reason.
+ */
+ cl->tleSortGroupRef = ++sortgroupref;
+ get_sort_group_operators(var->vartype,
+ false, true, false,
+ &cl->sortop, &cl->eqop, NULL,
+ &cl->hashable);
+ group_clauses_final = lappend(group_clauses_final, cl);
+ add_column_to_pathtarget(target, (Expr *) var,
+ cl->tleSortGroupRef);
+
+ /*
+ * The aggregation input target must emit this var too.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) var,
+ cl->tleSortGroupRef);
+ }
+ }
+
+ /*
+ * Add aggregates to the grouping target.
+ */
+ foreach(lc, aggregates)
+ {
+ GroupedVarInfo *gvi;
+
+ gvi = lfirst_node(GroupedVarInfo, lc);
+ add_column_to_pathtarget(target, (Expr *) gvi->agg_partial,
+ gvi->sortgroupref);
+ }
+
+ /*
+ * Build a list of grouping expressions and a list of the corresponding
+ * SortGroupClauses.
+ */
+ i = 0;
+ result = makeNode(RelAggInfo);
+ foreach(lc, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(lc);
+
+ if (IsA(texpr, Aggref))
+ {
+ /*
+ * Once we see Aggref, no grouping expressions should follow.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides being an aggregate, the target expression should have no
+ * other reason then being a column of a relation functionally
+ * dependent on the GROUP BY clause. So it's not actually a grouping
+ * column.
+ */
+ if (sortgroupref == 0)
+ continue;
+
+ /*
+ * group_clause_final contains the "local" clauses, so this search
+ * should succeed.
+ */
+ cl = get_sortgroupref_clause(sortgroupref, group_clauses_final);
+
+ result->group_clauses = list_append_unique(result->group_clauses,
+ cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ result->group_exprs = list_append_unique(result->group_exprs,
+ texpr);
+ }
+
+ /*
+ * Since neither target nor agg_input is supposed to be identical to the
+ * source reltarget, compute the width and cost again.
+ *
+ * target does not yet contain aggregates, but these will be accounted by
+ * AggPath.
+ */
+ set_pathtarget_cost_width(root, target);
+ set_pathtarget_cost_width(root, agg_input);
+
+ result->relids = bms_copy(rel->relids);
+ result->target = target;
+ result->agg_input = agg_input;
+
+ /* Finally collect the aggregates. */
+ while (lc != NULL)
+ {
+ Aggref *aggref = lfirst_node(Aggref, lc);
+
+ /*
+ * Partial aggregation is what the grouped paths should do.
+ */
+ result->agg_exprs = lappend(result->agg_exprs, aggref);
+ lc = lnext(target->exprs, lc);
+ }
+
+ /* The "input_rows" field should be set by caller. */
+ return result;
+}
+
+/*
+ * Initialize target for grouped paths (target) as well as a target for paths
+ * that generate input for aggregation (agg_input).
+ *
+ * group_exprs_extra_p receives a list of Var nodes for which we need to
+ * construct SortGroupClause. Those vars will then be used as additional
+ * grouping expressions, for the sake of join clauses.
+ *
+ * gvis a list of GroupedVarInfo's possibly useful for rel.
+ *
+ * Return true iff the targets could be initialized.
+ */
+static bool
+init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p)
+{
+ ListCell *lc;
+ List *possibly_dependent = NIL;
+ Var *tvar;
+
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Index sortgroupref;
+
+ /*
+ * Given that PlaceHolderVar currently prevents us from doing
+ * aggregation push-down, the source target cannot contain anything
+ * more complex than a Var.
+ */
+ tvar = lfirst_node(Var, lc);
+
+ sortgroupref = get_expression_sortgroupref((Expr *) tvar, gvis);
+ if (sortgroupref > 0)
+ {
+ /*
+ * If the target expression can be used as the grouping key, we
+ * don't have to worry whether it can be emitted by the AggPath
+ * pushed down to relation / join.
+ */
+ add_column_to_pathtarget(target, (Expr *) tvar, sortgroupref);
+
+ /*
+ * As for agg_input, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) tvar, sortgroupref);
+ }
+ else
+ {
+ /*
+ * Another reason we might need this variable is that some
+ * aggregate pushed down to this relation references it. In such a
+ * case, add that var to agg_input, but not to "target". However,
+ * if the aggregate is not the only reason for the var to be in
+ * the target, some more checks need to be performed below.
+ */
+ if (is_var_in_aggref_only(root, tvar))
+ add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
+ else if (is_var_needed_by_join(root, tvar, rel))
+ {
+ /*
+ * The variable is needed for a join, however it's neither in
+ * the GROUP BY clause nor can it be derived from it using EC.
+ * (Otherwise it would have to be added to the targets above.)
+ * We need to construct special SortGroupClause for that
+ * variable.
+ *
+ * Note that its tleSortGroupRef needs to be unique within
+ * agg_input, so we need to postpone creation of the
+ * SortGroupClause's until we're done with the iteration of
+ * rel->reltarget->exprs. Also it makes sense for the caller
+ * to do some more check before it starts to create those
+ * SortGroupClause's.
+ */
+ *group_exprs_extra_p = lappend(*group_exprs_extra_p, tvar);
+ }
+ else
+ {
+ /*
+ * The Var can be functionally dependent on another expression
+ * of the target, but we cannot check until the other
+ * expressions are in the target.
+ */
+ possibly_dependent = lappend(possibly_dependent, tvar);
+ }
+ }
+ }
+
+ /*
+ * Now we can check whether the expression is functionally dependent on
+ * another one.
+ */
+ foreach(lc, possibly_dependent)
+ {
+ List *deps = NIL;
+ RangeTblEntry *rte;
+
+ tvar = lfirst_node(Var, lc);
+ rte = root->simple_rte_array[tvar->varno];
+
+ /*
+ * Check if the Var can be in the grouping key even though it's not
+ * mentioned by the GROUP BY clause (and could not be derived using
+ * ECs).
+ */
+ if (check_functional_grouping(rte->relid, tvar->varno,
+ tvar->varlevelsup,
+ target->exprs, &deps))
+ {
+ /*
+ * The var shouldn't be actually used for grouping key evaluation
+ * (instead, the one this depends on will be), so sortgroupref
+ * should not be important.
+ */
+ add_new_column_to_pathtarget(target, (Expr *) tvar);
+ add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
+ }
+ else
+ {
+ /*
+ * As long as the query is semantically correct, arriving here
+ * means that the var is referenced by a generic grouping
+ * expression but not referenced by any join.
+ *
+ * If the aggregate push-down will support generic grouping
+ * expression sin the future, create_rel_agg_info() will have to
+ * add this variable to "agg_input" target and also add the whole
+ * generic expression to "target".
+ */
+ return false;
+ }
+ }
+
+ return true;
+}
+
+/*
+ * Check whether given variable appears in Aggref(s) which we consider usable
+ * at relation / join level, and only in the Aggref(s).
+ */
+static bool
+is_var_in_aggref_only(PlannerInfo *root, Var *var)
+{
+ ListCell *lc;
+ bool found = false;
+
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+ ListCell *lc2;
+ List *vars;
+
+ if (!IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (!bms_is_member(var->varno, gvi->gv_eval_at))
+ continue;
+
+ /*
+ * XXX Consider some sort of caching.
+ */
+ vars = pull_var_clause((Node *) gvi->gvexpr, PVC_RECURSE_AGGREGATES);
+ foreach(lc2, vars)
+ {
+ Var *v = lfirst_node(Var, lc2);
+
+ if (equal(v, var))
+ {
+ found = true;
+ break;
+ }
+
+ }
+ list_free(vars);
+
+ if (found)
+ break;
+ }
+
+ /* No aggregate references the Var? */
+ if (!found)
+ return false;
+
+ /* Does the Var appear in the target outside aggregates? */
+ found = false;
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc);
+
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ if (equal(te->expr, var))
+ return false;
+
+ }
+
+ /* The Var is in aggregate(s) and only there. */
+ return true;
+}
+
+/*
+ * Check if given variable is needed by joins above the current rel?
+ *
+ * Consider pushing the aggregate avg(b.y) down to relation "b" for the
+ * following query:
+ *
+ * SELECT a.i, avg(b.y)
+ * FROM a JOIN b ON b.j = a.i
+ * GROUP BY a.i;
+ *
+ * If we aggregate the "b" relation alone, the column "b.j" needs to be used
+ * as the grouping key because otherwise it cannot find its way to the input
+ * of the join expression.
+ */
+static bool
+is_var_needed_by_join(PlannerInfo *root, Var *var, RelOptInfo *rel)
+{
+ Relids relids_no_top;
+ int ndx;
+ RelOptInfo *baserel;
+
+ /*
+ * The relids we're not interested in do include 0, which is the top-level
+ * targetlist. The only reason for relids to contain 0 should be that
+ * arg_var is referenced either by aggregate or by grouping expression,
+ * but right now we're interested in the *other* reasons. (As soon
+ * aggregation is pushed down, the aggregates in the query targetlist no
+ * longer need direct reference to arg_var anyway.)
+ */
+
+ relids_no_top = bms_copy(rel->relids);
+ bms_add_member(relids_no_top, 0);
+
+ baserel = find_base_rel(root, var->varno);
+ ndx = var->varattno - baserel->min_attr;
+ if (bms_nonempty_difference(baserel->attr_needed[ndx],
+ relids_no_top))
+ return true;
+
+ return false;
+}
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
index 02a3c6b165..5011afc033 100644
--- a/src/backend/optimizer/util/tlist.c
+++ b/src/backend/optimizer/util/tlist.c
@@ -825,6 +825,37 @@ apply_pathtarget_labeling_to_tlist(List *tlist, PathTarget *target)
}
}
+/*
+ * Return sortgroupref if expr can be used as the grouping expression in an
+ * AggPath at relation or join level, or 0 if it can't.
+ *
+ * gvis a list of a list of GroupedVarInfo's available for the query,
+ * including those derived using equivalence classes.
+ */
+Index
+get_expression_sortgroupref(Expr *expr, List *gvis)
+{
+ ListCell *lc;
+
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (equal(gvi->gvexpr, expr))
+ {
+ Assert(gvi->sortgroupref > 0);
+
+ return gvi->sortgroupref;
+ }
+ }
+
+ /* The expression cannot be used as grouping key. */
+ return 0;
+}
+
/*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 2f3e0a70e0..05ea045b1e 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -1100,6 +1100,16 @@ static struct config_bool ConfigureNamesBool[] =
false,
NULL, NULL, NULL
},
+ {
+ {"enable_agg_pushdown", PGC_USERSET, QUERY_TUNING_METHOD,
+ gettext_noop("Enables aggregate push-down."),
+ NULL,
+ GUC_EXPLAIN
+ },
+ &enable_agg_pushdown,
+ false,
+ NULL, NULL, NULL
+ },
{
{"enable_parallel_append", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables the planner's use of parallel append plans."),
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index be5ab273f0..62134356a0 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -226,6 +226,7 @@ typedef enum NodeTag
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_RelAggInfo,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
@@ -271,6 +272,7 @@ typedef enum NodeTag
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 44374de796..98be8b7de2 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -304,6 +304,8 @@ struct PlannerInfo
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
@@ -321,6 +323,12 @@ struct PlannerInfo
/* Use fetch_upper_rel() to get any particular upper rel */
RelInfoList upper_rels[UPPERREL_FINAL + 1]; /* upper-rel RelOptInfos */
+ /*
+ * One instance of RelAggInfo per item of the
+ * upper_rels[UPPERREL_PARTIAL_GROUP_AGG] list.
+ */
+ struct RelInfoList *agg_info_list; /* list of grouped relation
+ * RelAggInfos */
/* Result tlists chosen by grouping_planner for upper-stage processing */
struct PathTarget *upper_targets[UPPERREL_FINAL + 1];
@@ -336,6 +344,12 @@ struct PlannerInfo
*/
List *processed_tlist;
+ /*
+ * The maximum ressortgroupref among target entries in processed_list.
+ * Useful when adding extra grouping expressions for partial aggregation.
+ */
+ int max_sortgroupref;
+
/* Fields filled during create_plan() for use in setrefs.c */
AttrNumber *grouping_map; /* for GroupingFunc fixup */
List *minmax_aggs; /* List of MinMaxAggInfos */
@@ -787,6 +801,60 @@ typedef struct RelOptInfo
((rel)->part_scheme && (rel)->boundinfo && (rel)->nparts > 0 && \
(rel)->part_rels && (rel)->partexprs && (rel)->nullable_partexprs)
+/*
+ * RelAggInfo
+ * Information needed to create grouped paths for base rels and joins.
+ *
+ * "relids" is the set of base-relation identifiers, just like with
+ * RelOptInfo.
+ *
+ * "target" will be used as pathtarget if partial aggregation is applied to
+ * base relation or join. The same target will also --- if the relation is a
+ * join --- be used to joinin grouped path to a non-grouped one. This target
+ * can contain plain-Var grouping expressions and Aggref nodes.
+ *
+ * Note: There's a convention that Aggref expressions are supposed to follow
+ * the other expressions of the target. Iterations of ->exprs may rely on this
+ * arrangement.
+ *
+ * "agg_input" contains Vars used either as grouping expressions or aggregate
+ * arguments. Paths providing the aggregation plan with input data should use
+ * this target. The only difference from reltarget of the non-grouped relation
+ * is that some items can have sortgroupref initialized.
+ *
+ * "input_rows" is the estimated number of input rows for AggPath. It's
+ * actually just a workspace for users of the structure, i.e. not initialized
+ * when instance of the structure is created.
+ *
+ * "group_clauses" and "group_exprs" are lists of SortGroupClause and the
+ * corresponding grouping expressions respectively.
+ *
+ * "agg_exprs" is a list of Aggref nodes for the aggregation of the relation's
+ * paths.
+ *
+ * "rel_grouped" is the relation containing the partially aggregated paths.
+ */
+typedef struct RelAggInfo
+{
+ NodeTag type;
+
+ Relids relids; /* Base rels contained in this grouped rel. */
+
+ struct PathTarget *target; /* Target for grouped paths. */
+
+ struct PathTarget *agg_input; /* pathtarget of paths that generate input
+ * for aggregation paths. */
+
+ double input_rows;
+
+ List *group_clauses;
+ List *group_exprs;
+
+ List *agg_exprs; /* Aggref expressions. */
+
+ RelOptInfo *rel_grouped; /* Grouped relation. */
+} RelAggInfo;
+
/*
* IndexOptInfo
* Per-index information for planning/optimization
@@ -2323,6 +2391,27 @@ typedef struct PlaceHolderInfo
int32 ph_width; /* estimated attribute width */
} PlaceHolderInfo;
+/*
+ * GroupedVarInfo exists for each expression that can be used as an aggregate
+ * or grouping expression evaluated below a join.
+ *
+ * TODO Rename, perhaps to GroupedTargetEntry? (Also rename the variables of
+ * this type.)
+ */
+typedef struct GroupedVarInfo
+{
+ NodeTag type;
+
+ Expr *gvexpr; /* the represented expression. */
+ Aggref *agg_partial; /* if gvexpr is aggregate, agg_partial is the
+ * corresponding partial aggregate */
+ Index sortgroupref; /* If gvexpr is a grouping expression, this is
+ * the tleSortGroupRef of the corresponding
+ * SortGroupClause. */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+} GroupedVarInfo;
+
/*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
index b7456e3e59..0464531e25 100644
--- a/src/include/optimizer/clauses.h
+++ b/src/include/optimizer/clauses.h
@@ -55,4 +55,6 @@ extern void CommuteOpExpr(OpExpr *clause);
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
+extern GroupedVarInfo *translate_expression_to_rel(PlannerInfo *root,
+ GroupedVarInfo *gvi, Index relid);
#endif /* CLAUSES_H */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 715a24ad29..47afa7ed1a 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -216,6 +216,14 @@ extern AggPath *create_agg_path(PlannerInfo *root,
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+extern AggPath *create_agg_sorted_path(PlannerInfo *root,
+ RelOptInfo *rel,
+ Path *subpath,
+ RelAggInfo *agg_info);
+extern AggPath *create_agg_hashed_path(PlannerInfo *root,
+ RelOptInfo *rel,
+ Path *subpath,
+ RelAggInfo *agg_info);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
@@ -287,14 +295,21 @@ extern void setup_simple_rel_arrays(PlannerInfo *root);
extern void expand_planner_arrays(PlannerInfo *root, int add_size);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
+extern RelOptInfo *build_simple_grouped_rel(PlannerInfo *root, int relid,
+ RelAggInfo **agg_info_p);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
+extern void add_grouped_rel(PlannerInfo *root, RelOptInfo *rel,
+ RelAggInfo *agg_info);
+extern RelOptInfo *find_grouped_rel(PlannerInfo *root, Relids relids,
+ RelAggInfo **agg_info_p);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr);
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info);
extern Relids min_join_parameterization(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
@@ -320,5 +335,5 @@ extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo, JoinType jointype);
-
+extern RelAggInfo *create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index 10b6e81079..ee34442772 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -21,6 +21,7 @@
* allpaths.c
*/
extern PGDLLIMPORT bool enable_geqo;
+extern PGDLLIMPORT bool enable_agg_pushdown;
extern PGDLLIMPORT int geqo_threshold;
extern PGDLLIMPORT int min_parallel_table_scan_size;
extern PGDLLIMPORT int min_parallel_index_scan_size;
@@ -56,6 +57,11 @@ extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
extern void generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
+extern void generate_grouping_paths(PlannerInfo *root,
+ RelOptInfo *rel_grouped,
+ RelOptInfo *rel_plain,
+ RelAggInfo *agg_info);
+
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages, int max_workers);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index f3cefe67b8..c2ab6b2115 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -72,6 +72,7 @@ extern void add_other_rels_to_query(PlannerInfo *root);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
Relids where_needed, bool create_new_ph);
+extern void setup_aggregate_pushdown(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
index 1d4c7da545..e2547732d6 100644
--- a/src/include/optimizer/tlist.h
+++ b/src/include/optimizer/tlist.h
@@ -50,8 +50,10 @@ extern void split_pathtarget_at_srfs(PlannerInfo *root,
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+/* TODO Find the best location for this one. */
+extern Index get_expression_sortgroupref(Expr *expr, List *gvis);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
-
#endif /* TLIST_H */
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
new file mode 100644
index 0000000000..413abc147e
--- /dev/null
+++ b/src/test/regress/expected/agg_pushdown.out
@@ -0,0 +1,215 @@
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+CREATE INDEX ON agg_pushdown_child1(parent);
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+ANALYZE;
+SET enable_agg_pushdown TO on;
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+--------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(10 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(11 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (p.i = c1.parent)
+ -> Sort
+ Sort Key: p.i
+ -> Seq Scan on agg_pushdown_parent p
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(12 rows)
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(8 rows)
+
+SET enable_seqscan TO on;
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.x = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(11 rows)
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: ((k = c1.j) AND (parent = c1.parent))
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(13 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+----------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Hash
+ -> Seq Scan on agg_pushdown_child2 c2
+(15 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: ((c1.j = c2.k) AND (c1.parent = c2.parent))
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(13 rows)
+
diff --git a/src/test/regress/expected/sysviews.out b/src/test/regress/expected/sysviews.out
index a126f0ad61..f169afb15c 100644
--- a/src/test/regress/expected/sysviews.out
+++ b/src/test/regress/expected/sysviews.out
@@ -72,6 +72,7 @@ select count(*) >= 0 as ok from pg_prepared_xacts;
select name, setting from pg_settings where name like 'enable%';
name | setting
--------------------------------+---------
+ enable_agg_pushdown | off
enable_bitmapscan | on
enable_gathermerge | on
enable_groupingsets_hash_disk | off
@@ -92,7 +93,7 @@ select name, setting from pg_settings where name like 'enable%';
enable_seqscan | on
enable_sort | on
enable_tidscan | on
-(20 rows)
+(21 rows)
-- Test that the pg_timezone_names and pg_timezone_abbrevs views are
-- more-or-less working. We can't test their contents in any great detail
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 8ba4136220..7c27fc730d 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -145,6 +145,7 @@ test: stats_ext
test: collate.linux.utf8
test: select_parallel
test: write_parallel
+test: agg_pushdown
test: publication
test: subscription
test: select_views
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
new file mode 100644
index 0000000000..6bbbc7a8a1
--- /dev/null
+++ b/src/test/regress/sql/agg_pushdown.sql
@@ -0,0 +1,114 @@
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+
+CREATE INDEX ON agg_pushdown_child1(parent);
+
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+ANALYZE;
+
+SET enable_agg_pushdown TO on;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_seqscan TO on;
+
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
--
2.20.1
v16-0003-Use-also-partial-paths-as-the-input-for-grouped-path.patchtext/x-diffDownload
From 066f0c0b60a738cb6766542b91c894905b672af2 Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Mon, 18 May 2020 14:28:31 +0200
Subject: [PATCH 3/3] Use also partial paths as the input for grouped paths.
---
src/backend/optimizer/path/allpaths.c | 46 +++++-
src/backend/optimizer/util/relnode.c | 46 +++---
src/test/regress/expected/agg_pushdown.out | 157 +++++++++++++++++++++
src/test/regress/sql/agg_pushdown.sql | 66 +++++++++
4 files changed, 285 insertions(+), 30 deletions(-)
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 84a918dc58..e0b5418de4 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -128,7 +128,7 @@ static void set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void add_grouped_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath, AggStrategy aggstrategy,
- RelAggInfo *agg_info);
+ RelAggInfo *agg_info, bool partial);
static RelOptInfo *make_rel_from_joinlist(PlannerInfo *root, List *joinlist);
static bool subquery_is_pushdown_safe(Query *subquery, Query *topquery,
pushdown_safety_info *safetyInfo);
@@ -3021,6 +3021,7 @@ generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
RelOptInfo *rel_plain, RelAggInfo *agg_info)
{
ListCell *lc;
+ Path *path;
if (IS_DUMMY_REL(rel_plain))
{
@@ -3030,7 +3031,7 @@ generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
foreach(lc, rel_plain->pathlist)
{
- Path *path = (Path *) lfirst(lc);
+ path = (Path *) lfirst(lc);
/*
* Since the path originates from the non-grouped relation which is
@@ -3044,7 +3045,8 @@ generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
* add_grouped_path() will check whether the path has suitable
* pathkeys.
*/
- add_grouped_path(root, rel_grouped, path, AGG_SORTED, agg_info);
+ add_grouped_path(root, rel_grouped, path, AGG_SORTED, agg_info,
+ false);
/*
* Repeated creation of hash table (for new parameter values) should
@@ -3052,12 +3054,38 @@ generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
* efficiency.
*/
if (path->param_info == NULL)
- add_grouped_path(root, rel_grouped, path, AGG_HASHED, agg_info);
+ add_grouped_path(root, rel_grouped, path, AGG_HASHED, agg_info,
+ false);
}
/* Could not generate any grouped paths? */
if (rel_grouped->pathlist == NIL)
+ {
mark_dummy_rel(rel_grouped);
+ return;
+ }
+
+ /*
+ * Almost the same for partial paths.
+ *
+ * The difference is that parameterized paths are never created, see
+ * add_partial_path() for explanation.
+ */
+ foreach(lc, rel_plain->partial_pathlist)
+ {
+ path = (Path *) lfirst(lc);
+
+ if (path->param_info != NULL)
+ continue;
+
+ path = (Path *) create_projection_path(root, rel_grouped, path,
+ agg_info->agg_input);
+
+ add_grouped_path(root, rel_grouped, path, AGG_SORTED, agg_info,
+ true);
+ add_grouped_path(root, rel_grouped, path, AGG_HASHED, agg_info,
+ true);
+ }
}
/*
@@ -3065,7 +3093,8 @@ generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
*/
static void
add_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
- AggStrategy aggstrategy, RelAggInfo *agg_info)
+ AggStrategy aggstrategy, RelAggInfo *agg_info,
+ bool partial)
{
Path *agg_path;
@@ -3081,7 +3110,12 @@ add_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
/* Add the grouped path to the list of grouped base paths. */
if (agg_path != NULL)
- add_path(rel, (Path *) agg_path);
+ {
+ if (!partial)
+ add_path(rel, (Path *) agg_path);
+ else
+ add_partial_path(rel, (Path *) agg_path);
+ }
}
/*
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 7460794b46..4164f4b0e4 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -958,33 +958,12 @@ build_join_rel(PlannerInfo *root,
build_joinrel_partition_info(joinrel, outer_rel, inner_rel,
restrictlist, sjinfo->jointype);
+ /*
+ * Set estimates of the joinrel's size.
+ */
if (!grouped)
- {
- /*
- * Set estimates of the joinrel's size.
- */
set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
sjinfo, restrictlist);
-
- /*
- * Set the consider_parallel flag if this joinrel could potentially be
- * scanned within a parallel worker. If this flag is false for either
- * inner_rel or outer_rel, then it must be false for the joinrel also.
- * Even if both are true, there might be parallel-restricted
- * expressions in the targetlist or quals.
- *
- * Note that if there are more than two rels in this relation, they
- * could be divided between inner_rel and outer_rel in any arbitrary
- * way. We assume this doesn't matter, because we should hit all the
- * same baserels and joinclauses while building up to this joinrel no
- * matter which we take; therefore, we should make the same decision
- * here however we get here.
- */
- if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
- is_parallel_safe(root, (Node *) restrictlist) &&
- is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
- joinrel->consider_parallel = true;
- }
else
{
/*
@@ -1000,6 +979,25 @@ build_join_rel(PlannerInfo *root,
agg_info->input_rows, NULL);
}
+ /*
+ * Set the consider_parallel flag if this joinrel could potentially be
+ * scanned within a parallel worker. If this flag is false for either
+ * inner_rel or outer_rel, then it must be false for the joinrel also.
+ * Even if both are true, there might be parallel-restricted expressions
+ * in the targetlist or quals.
+ *
+ * Note that if there are more than two rels in this relation, they could
+ * be divided between inner_rel and outer_rel in any arbitrary way. We
+ * assume this doesn't matter, because we should hit all the same baserels
+ * and joinclauses while building up to this joinrel no matter which we
+ * take; therefore, we should make the same decision here however we get
+ * here.
+ */
+ if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
+ is_parallel_safe(root, (Node *) restrictlist) &&
+ is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
+ joinrel->consider_parallel = true;
+
/* Add the joinrel to the PlannerInfo. */
if (!grouped)
add_join_rel(root, joinrel);
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
index 413abc147e..66d36d122e 100644
--- a/src/test/regress/expected/agg_pushdown.out
+++ b/src/test/regress/expected/agg_pushdown.out
@@ -91,6 +91,7 @@ AS c1 ON c1.parent = p.i GROUP BY p.i;
-> Seq Scan on agg_pushdown_child1 c1
(12 rows)
+-- Restore the default values.
SET enable_nestloop TO on;
SET enable_hashjoin TO on;
-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
@@ -213,3 +214,159 @@ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
-> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
(13 rows)
+-- Most of the tests above with parallel query processing enforced.
+SET min_parallel_index_scan_size = 0;
+SET min_parallel_table_scan_size = 0;
+SET parallel_setup_cost = 0;
+SET parallel_tuple_cost = 0;
+-- Partially aggregate a single relation.
+--
+-- Nestloop join.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 1
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Parallel Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(10 rows)
+
+-- Hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Gather
+ Workers Planned: 1
+ -> Parallel Hash Join
+ Hash Cond: (c1.parent = p.i)
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Parallel Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Parallel Hash
+ -> Parallel Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(13 rows)
+
+-- Merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 1
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Parallel Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(10 rows)
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Parallel Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: ((k = c1.j) AND (parent = c1.parent))
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(15 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 1
+ -> Sort
+ Sort Key: p.i
+ -> Parallel Hash Join
+ Hash Cond: (c1.parent = p.i)
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Parallel Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Parallel Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Parallel Hash
+ -> Parallel Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Parallel Hash
+ -> Parallel Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(17 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 2
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: ((c1.j = c2.k) AND (c1.parent = c2.parent))
+ -> Parallel Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(15 rows)
+
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
index 6bbbc7a8a1..49ba6dd67c 100644
--- a/src/test/regress/sql/agg_pushdown.sql
+++ b/src/test/regress/sql/agg_pushdown.sql
@@ -61,6 +61,7 @@ EXPLAIN (COSTS off)
SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
AS c1 ON c1.parent = p.i GROUP BY p.i;
+-- Restore the default values.
SET enable_nestloop TO on;
SET enable_hashjoin TO on;
@@ -112,3 +113,68 @@ EXPLAIN (COSTS off)
SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- Most of the tests above with parallel query processing enforced.
+SET min_parallel_index_scan_size = 0;
+SET min_parallel_table_scan_size = 0;
+SET parallel_setup_cost = 0;
+SET parallel_tuple_cost = 0;
+
+-- Partially aggregate a single relation.
+--
+-- Nestloop join.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- Hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- Merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
--
2.20.1
Re: WIP: Aggregation push-down
On 19 May 2020, at 10:17, Antonin Houska <ah@cybertec.at> wrote:
The next version is attached.
This version now fails to apply to HEAD, with what looks like like a trivial
error in the expected test output. Can you please submit a rebased version so
we can see it run in the patch tester CI? I'm marking the entry as Waiting on
Author in the meantime.
cheers ./daniel
Re: WIP: Aggregation push-down
On Thu, 2020-07-02 at 14:39 +0200, Daniel Gustafsson wrote:
This version now fails to apply to HEAD, with what looks like like a trivial
error in the expected test output. Can you please submit a rebased version so
we can see it run in the patch tester CI? I'm marking the entry as Waiting on
Author in the meantime.
I have rebased the patch against current HEAD, it passes "make installcheck".
Yours,
Laurenz Albe
Attachments:
v17-0001-Introduce-RelInfoList-structure.patchtext/x-patch; charset=UTF-8; name=v17-0001-Introduce-RelInfoList-structure.patchDownload
From 67fa4b73d1f8483d561b7a16907e9edbf00215c9 Mon Sep 17 00:00:00 2001
From: Laurenz Albe <laurenz.albe@cybertec.at>
Date: Fri, 3 Jul 2020 10:20:07 +0200
Subject: [PATCH 1/3] Introduce RelInfoList structure.
This patch puts join_rel_list and join_rel_hash fields of PlannerInfo
structure into a new structure RelInfoList. It also adjusts add_join_rel() and
find_join_rel() functions so they only call add_rel_info() and find_rel_info()
respectively.
fetch_upper_rel() now uses the new API and the hash table as well because the
list stored in root->upper_rels[UPPERREL_PARTIAL_GROUP_AGG] will contain many
relations as soon as the aggregate push-down feature is added.
---
contrib/postgres_fdw/postgres_fdw.c | 3 +-
src/backend/nodes/outfuncs.c | 11 ++
src/backend/optimizer/geqo/geqo_eval.c | 12 +-
src/backend/optimizer/plan/planmain.c | 3 +-
src/backend/optimizer/util/relnode.c | 182 +++++++++++++++----------
src/include/nodes/nodes.h | 1 +
src/include/nodes/pathnodes.h | 30 ++--
7 files changed, 149 insertions(+), 93 deletions(-)
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 9fc53cad68..a46834a377 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -5251,7 +5251,8 @@ foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel, JoinType jointype,
*/
Assert(fpinfo->relation_index == 0); /* shouldn't be set yet */
fpinfo->relation_index =
- list_length(root->parse->rtable) + list_length(root->join_rel_list);
+ list_length(root->parse->rtable) +
+ list_length(root->join_rel_list->items);
return true;
}
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index e2f177515d..dbd36408c3 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2315,6 +2315,14 @@ _outRelOptInfo(StringInfo str, const RelOptInfo *node)
WRITE_NODE_FIELD(partitioned_child_rels);
}
+static void
+_outRelInfoList(StringInfo str, const RelInfoList *node)
+{
+ WRITE_NODE_TYPE("RELOPTINFOLIST");
+
+ WRITE_NODE_FIELD(items);
+}
+
static void
_outIndexOptInfo(StringInfo str, const IndexOptInfo *node)
{
@@ -4112,6 +4120,9 @@ outNode(StringInfo str, const void *obj)
case T_RelOptInfo:
_outRelOptInfo(str, obj);
break;
+ case T_RelInfoList:
+ _outRelInfoList(str, obj);
+ break;
case T_IndexOptInfo:
_outIndexOptInfo(str, obj);
break;
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index ff33acc7b6..6b04ab3c2a 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -92,11 +92,11 @@ geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
*
* join_rel_level[] shouldn't be in use, so just Assert it isn't.
*/
- savelength = list_length(root->join_rel_list);
- savehash = root->join_rel_hash;
+ savelength = list_length(root->join_rel_list->items);
+ savehash = root->join_rel_list->hash;
Assert(root->join_rel_level == NULL);
- root->join_rel_hash = NULL;
+ root->join_rel_list->hash = NULL;
/* construct the best path for the given combination of relations */
joinrel = gimme_tree(root, tour, num_gene);
@@ -121,9 +121,9 @@ geqo_eval(PlannerInfo *root, Gene *tour, int num_gene)
* Restore join_rel_list to its former state, and put back original
* hashtable if any.
*/
- root->join_rel_list = list_truncate(root->join_rel_list,
- savelength);
- root->join_rel_hash = savehash;
+ root->join_rel_list->items = list_truncate(root->join_rel_list->items,
+ savelength);
+ root->join_rel_list->hash = savehash;
/* release all the memory acquired within gimme_tree */
MemoryContextSwitchTo(oldcxt);
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 62dfc6d44a..5fa33ec200 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -65,8 +65,7 @@ query_planner(PlannerInfo *root,
* NOTE: append_rel_list was set up by subquery_planner, so do not touch
* here.
*/
- root->join_rel_list = NIL;
- root->join_rel_hash = NULL;
+ root->join_rel_list = makeNode(RelInfoList);
root->join_rel_level = NULL;
root->join_cur_level = 0;
root->canon_pathkeys = NIL;
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index a203e6f1ff..a95e6364ae 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -32,11 +32,15 @@
#include "utils/lsyscache.h"
-typedef struct JoinHashEntry
+/*
+ * An entry of a hash table that we use to make lookup for RelOptInfo
+ * structures more efficient.
+ */
+typedef struct RelInfoEntry
{
- Relids join_relids; /* hash key --- MUST BE FIRST */
- RelOptInfo *join_rel;
-} JoinHashEntry;
+ Relids relids; /* hash key --- MUST BE FIRST */
+ void *data;
+} RelInfoEntry;
static void build_joinrel_tlist(PlannerInfo *root, RelOptInfo *joinrel,
RelOptInfo *input_rel);
@@ -390,11 +394,11 @@ find_base_rel(PlannerInfo *root, int relid)
}
/*
- * build_join_rel_hash
- * Construct the auxiliary hash table for join relations.
+ * build_rel_hash
+ * Construct the auxiliary hash table for relation specific data.
*/
static void
-build_join_rel_hash(PlannerInfo *root)
+build_rel_hash(RelInfoList *list)
{
HTAB *hashtab;
HASHCTL hash_ctl;
@@ -403,47 +407,53 @@ build_join_rel_hash(PlannerInfo *root)
/* Create the hash table */
MemSet(&hash_ctl, 0, sizeof(hash_ctl));
hash_ctl.keysize = sizeof(Relids);
- hash_ctl.entrysize = sizeof(JoinHashEntry);
+ hash_ctl.entrysize = sizeof(RelInfoEntry);
hash_ctl.hash = bitmap_hash;
hash_ctl.match = bitmap_match;
hash_ctl.hcxt = CurrentMemoryContext;
- hashtab = hash_create("JoinRelHashTable",
+ hashtab = hash_create("RelHashTable",
256L,
&hash_ctl,
HASH_ELEM | HASH_FUNCTION | HASH_COMPARE | HASH_CONTEXT);
/* Insert all the already-existing joinrels */
- foreach(l, root->join_rel_list)
+ foreach(l, list->items)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
- JoinHashEntry *hentry;
+ void *item = lfirst(l);
+ RelInfoEntry *hentry;
bool found;
+ Relids relids;
+
+ Assert(IsA(item, RelOptInfo));
+ relids = ((RelOptInfo *) item)->relids;
- hentry = (JoinHashEntry *) hash_search(hashtab,
- &(rel->relids),
- HASH_ENTER,
- &found);
+ hentry = (RelInfoEntry *) hash_search(hashtab,
+ &relids,
+ HASH_ENTER,
+ &found);
Assert(!found);
- hentry->join_rel = rel;
+ hentry->data = item;
}
- root->join_rel_hash = hashtab;
+ list->hash = hashtab;
}
/*
- * find_join_rel
- * Returns relation entry corresponding to 'relids' (a set of RT indexes),
- * or NULL if none exists. This is for join relations.
+ * find_rel_info
+ * Find a base or join relation entry.
*/
-RelOptInfo *
-find_join_rel(PlannerInfo *root, Relids relids)
+static void *
+find_rel_info(RelInfoList *list, Relids relids)
{
+ if (list == NULL)
+ return NULL;
+
/*
* Switch to using hash lookup when list grows "too long". The threshold
* is arbitrary and is known only here.
*/
- if (!root->join_rel_hash && list_length(root->join_rel_list) > 32)
- build_join_rel_hash(root);
+ if (!list->hash && list_length(list->items) > 32)
+ build_rel_hash(list);
/*
* Use either hashtable lookup or linear search, as appropriate.
@@ -453,34 +463,90 @@ find_join_rel(PlannerInfo *root, Relids relids)
* so would force relids out of a register and thus probably slow down the
* list-search case.
*/
- if (root->join_rel_hash)
+ if (list->hash)
{
Relids hashkey = relids;
- JoinHashEntry *hentry;
+ RelInfoEntry *hentry;
- hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
- &hashkey,
- HASH_FIND,
- NULL);
+ hentry = (RelInfoEntry *) hash_search(list->hash,
+ &hashkey,
+ HASH_FIND,
+ NULL);
if (hentry)
- return hentry->join_rel;
+ return hentry->data;
}
else
{
ListCell *l;
- foreach(l, root->join_rel_list)
+ foreach(l, list->items)
{
- RelOptInfo *rel = (RelOptInfo *) lfirst(l);
+ void *item = lfirst(l);
+ Relids item_relids = NULL;
+
+ Assert(IsA(item, RelOptInfo));
- if (bms_equal(rel->relids, relids))
- return rel;
+ item_relids = ((RelOptInfo *) item)->relids;
+ if (bms_equal(item_relids, relids))
+ return item;
}
}
return NULL;
}
+/*
+ * find_join_rel
+ * Returns relation entry corresponding to 'relids' (a set of RT indexes),
+ * or NULL if none exists. This is for join relations.
+ */
+RelOptInfo *
+find_join_rel(PlannerInfo *root, Relids relids)
+{
+ return (RelOptInfo *) find_rel_info(root->join_rel_list, relids);
+}
+
+/*
+ * add_rel_info
+ * Add relation specific info to a list, and also add it to the auxiliary
+ * hashtable if there is one.
+ */
+static void
+add_rel_info(RelInfoList *list, void *data)
+{
+ Assert(IsA(data, RelOptInfo));
+
+ /* GEQO requires us to append the new joinrel to the end of the list! */
+ list->items = lappend(list->items, data);
+
+ /* store it into the auxiliary hashtable if there is one. */
+ if (list->hash)
+ {
+ Relids relids;
+ RelInfoEntry *hentry;
+ bool found;
+
+ relids = ((RelOptInfo *) data)->relids;
+ hentry = (RelInfoEntry *) hash_search(list->hash,
+ &relids,
+ HASH_ENTER,
+ &found);
+ Assert(!found);
+ hentry->data = data;
+ }
+}
+
+/*
+ * add_join_rel
+ * Add given join relation to the list of join relations in the given
+ * PlannerInfo.
+ */
+static void
+add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
+{
+ add_rel_info(root->join_rel_list, joinrel);
+}
+
/*
* set_foreign_rel_properties
* Set up foreign-join fields if outer and inner relation are foreign
@@ -531,32 +597,6 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
}
}
-/*
- * add_join_rel
- * Add given join relation to the list of join relations in the given
- * PlannerInfo. Also add it to the auxiliary hashtable if there is one.
- */
-static void
-add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
-{
- /* GEQO requires us to append the new joinrel to the end of the list! */
- root->join_rel_list = lappend(root->join_rel_list, joinrel);
-
- /* store it into the auxiliary hashtable if there is one. */
- if (root->join_rel_hash)
- {
- JoinHashEntry *hentry;
- bool found;
-
- hentry = (JoinHashEntry *) hash_search(root->join_rel_hash,
- &(joinrel->relids),
- HASH_ENTER,
- &found);
- Assert(!found);
- hentry->join_rel = joinrel;
- }
-}
-
/*
* build_join_rel
* Returns relation entry corresponding to the union of two given rels,
@@ -1191,22 +1231,14 @@ subbuild_joinrel_joinlist(RelOptInfo *joinrel,
RelOptInfo *
fetch_upper_rel(PlannerInfo *root, UpperRelationKind kind, Relids relids)
{
+ RelInfoList *list = &root->upper_rels[kind];
RelOptInfo *upperrel;
- ListCell *lc;
-
- /*
- * For the moment, our indexing data structure is just a List for each
- * relation kind. If we ever get so many of one kind that this stops
- * working well, we can improve it. No code outside this function should
- * assume anything about how to find a particular upperrel.
- */
/* If we already made this upperrel for the query, return it */
- foreach(lc, root->upper_rels[kind])
+ if (list)
{
- upperrel = (RelOptInfo *) lfirst(lc);
-
- if (bms_equal(upperrel->relids, relids))
+ upperrel = find_rel_info(list, relids);
+ if (upperrel)
return upperrel;
}
@@ -1225,7 +1257,7 @@ fetch_upper_rel(PlannerInfo *root, UpperRelationKind kind, Relids relids)
upperrel->cheapest_unique_path = NULL;
upperrel->cheapest_parameterized_paths = NIL;
- root->upper_rels[kind] = lappend(root->upper_rels[kind], upperrel);
+ add_rel_info(&root->upper_rels[kind], upperrel);
return upperrel;
}
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 381d84b4e4..be5ab273f0 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -222,6 +222,7 @@ typedef enum NodeTag
T_PlannerInfo,
T_PlannerGlobal,
T_RelOptInfo,
+ T_RelInfoList,
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 485d1b06c9..44374de796 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -93,6 +93,23 @@ typedef enum InheritanceKind
INHKIND_PARTITIONED
} InheritanceKind;
+/*
+ * Hashed list to store relation specific info and to retrieve it by relids.
+ *
+ * For small problems we just scan the list to do lookups, but when there are
+ * many relations we build a hash table for faster lookups. The hash table is
+ * present and valid when rel_hash is not NULL. Note that we still maintain
+ * the list even when using the hash table for lookups; this simplifies life
+ * for GEQO.
+ */
+typedef struct RelInfoList
+{
+ NodeTag type;
+
+ List *items;
+ struct HTAB *hash;
+} RelInfoList;
+
/*----------
* PlannerGlobal
* Global information for planning/optimization
@@ -236,15 +253,9 @@ struct PlannerInfo
/*
* join_rel_list is a list of all join-relation RelOptInfos we have
- * considered in this planning run. For small problems we just scan the
- * list to do lookups, but when there are many join relations we build a
- * hash table for faster lookups. The hash table is present and valid
- * when join_rel_hash is not NULL. Note that we still maintain the list
- * even when using the hash table for lookups; this simplifies life for
- * GEQO.
+ * considered in this planning run.
*/
- List *join_rel_list; /* list of join-relation RelOptInfos */
- struct HTAB *join_rel_hash; /* optional hashtable for join relations */
+ struct RelInfoList *join_rel_list; /* list of join-relation RelOptInfos */
/*
* When doing a dynamic-programming-style join search, join_rel_level[k]
@@ -308,7 +319,8 @@ struct PlannerInfo
List *initial_rels; /* RelOptInfos we are now trying to join */
/* Use fetch_upper_rel() to get any particular upper rel */
- List *upper_rels[UPPERREL_FINAL + 1]; /* upper-rel RelOptInfos */
+ RelInfoList upper_rels[UPPERREL_FINAL + 1]; /* upper-rel RelOptInfos */
+
/* Result tlists chosen by grouping_planner for upper-stage processing */
struct PathTarget *upper_targets[UPPERREL_FINAL + 1];
--
2.21.3
v17-0002-Aggregate-push-down-basic-functionality.patchtext/x-patch; charset=UTF-8; name=v17-0002-Aggregate-push-down-basic-functionality.patchDownload
From 437b3c2e9295ab27c6b61ea21ac631fd135477b1 Mon Sep 17 00:00:00 2001
From: Laurenz Albe <laurenz.albe@cybertec.at>
Date: Fri, 3 Jul 2020 10:21:25 +0200
Subject: [PATCH 2/3] Aggregate push-down - basic functionality.
With this patch, partial aggregation can be applied to a base relation or to a
join, and the resulting "grouped" relations can be joined to other "plain"
relations. Once all tables are joined, the aggregation is finalized. See
README for more information.
The next patches will enable the aggregate push-down feature for parallel
query processing, for partitioned tables and for foreign tables.
---
src/backend/nodes/copyfuncs.c | 20 +-
src/backend/nodes/outfuncs.c | 34 +
src/backend/optimizer/README | 69 ++
src/backend/optimizer/path/allpaths.c | 147 ++++
src/backend/optimizer/path/costsize.c | 17 +-
src/backend/optimizer/path/equivclass.c | 130 +++
src/backend/optimizer/path/joinrels.c | 195 ++++-
src/backend/optimizer/plan/initsplan.c | 290 +++++++
src/backend/optimizer/plan/planmain.c | 12 +
src/backend/optimizer/plan/planner.c | 47 +-
src/backend/optimizer/plan/setrefs.c | 33 +
src/backend/optimizer/prep/prepjointree.c | 1 +
src/backend/optimizer/util/pathnode.c | 143 +++-
src/backend/optimizer/util/relnode.c | 923 ++++++++++++++++++++-
src/backend/optimizer/util/tlist.c | 31 +
src/backend/utils/misc/guc.c | 10 +
src/include/nodes/nodes.h | 2 +
src/include/nodes/pathnodes.h | 89 ++
src/include/optimizer/clauses.h | 2 +
src/include/optimizer/pathnode.h | 19 +-
src/include/optimizer/paths.h | 6 +
src/include/optimizer/planmain.h | 1 +
src/include/optimizer/tlist.h | 4 +-
src/test/regress/expected/agg_pushdown.out | 215 +++++
src/test/regress/expected/sysviews.out | 3 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/agg_pushdown.sql | 114 +++
27 files changed, 2477 insertions(+), 81 deletions(-)
create mode 100644 src/test/regress/expected/agg_pushdown.out
create mode 100644 src/test/regress/sql/agg_pushdown.sql
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index d8cf87e6d0..c5ec1e7e9c 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2251,8 +2251,8 @@ _copyOnConflictExpr(const OnConflictExpr *from)
/* ****************************************************************
* pathnodes.h copy functions
*
- * We don't support copying RelOptInfo, IndexOptInfo, or Path nodes.
- * There are some subsidiary structs that are useful to copy, though.
+ * We don't support copying RelOptInfo, IndexOptInfo, RelAggInfo or Path
+ * nodes. There are some subsidiary structs that are useful to copy, though.
* ****************************************************************
*/
@@ -2395,6 +2395,19 @@ _copyPlaceHolderInfo(const PlaceHolderInfo *from)
return newnode;
}
+static GroupedVarInfo *
+_copyGroupedVarInfo(const GroupedVarInfo *from)
+{
+ GroupedVarInfo *newnode = makeNode(GroupedVarInfo);
+
+ COPY_NODE_FIELD(gvexpr);
+ COPY_NODE_FIELD(agg_partial);
+ COPY_SCALAR_FIELD(sortgroupref);
+ COPY_SCALAR_FIELD(gv_eval_at);
+
+ return newnode;
+}
+
/* ****************************************************************
* parsenodes.h copy functions
* ****************************************************************
@@ -5167,6 +5180,9 @@ copyObjectImpl(const void *from)
case T_PlaceHolderInfo:
retval = _copyPlaceHolderInfo(from);
break;
+ case T_GroupedVarInfo:
+ retval = _copyGroupedVarInfo(from);
+ break;
/*
* VALUE NODES
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index dbd36408c3..c02bcac698 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2223,6 +2223,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_BITMAPSET_FIELD(all_baserels);
WRITE_BITMAPSET_FIELD(nullable_baserels);
WRITE_NODE_FIELD(join_rel_list);
+ WRITE_NODE_FIELD(agg_info_list);
WRITE_INT_FIELD(join_cur_level);
WRITE_NODE_FIELD(init_plans);
WRITE_NODE_FIELD(cte_plan_ids);
@@ -2237,6 +2238,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(append_rel_list);
WRITE_NODE_FIELD(rowMarks);
WRITE_NODE_FIELD(placeholder_list);
+ WRITE_NODE_FIELD(grouped_var_list);
WRITE_NODE_FIELD(fkey_list);
WRITE_NODE_FIELD(query_pathkeys);
WRITE_NODE_FIELD(group_pathkeys);
@@ -2244,6 +2246,7 @@ _outPlannerInfo(StringInfo str, const PlannerInfo *node)
WRITE_NODE_FIELD(distinct_pathkeys);
WRITE_NODE_FIELD(sort_pathkeys);
WRITE_NODE_FIELD(processed_tlist);
+ WRITE_INT_FIELD(max_sortgroupref);
WRITE_NODE_FIELD(minmax_aggs);
WRITE_FLOAT_FIELD(total_table_pages, "%.0f");
WRITE_FLOAT_FIELD(tuple_fraction, "%.4f");
@@ -2465,6 +2468,20 @@ _outParamPathInfo(StringInfo str, const ParamPathInfo *node)
WRITE_NODE_FIELD(ppi_clauses);
}
+static void
+_outRelAggInfo(StringInfo str, const RelAggInfo *node)
+{
+ WRITE_NODE_TYPE("RELAGGINFO");
+
+ WRITE_BITMAPSET_FIELD(relids);
+ WRITE_NODE_FIELD(target);
+ WRITE_NODE_FIELD(agg_input);
+ WRITE_FLOAT_FIELD(input_rows, "%.0f");
+ WRITE_NODE_FIELD(group_clauses);
+ WRITE_NODE_FIELD(group_exprs);
+ WRITE_NODE_FIELD(agg_exprs);
+}
+
static void
_outRestrictInfo(StringInfo str, const RestrictInfo *node)
{
@@ -2566,6 +2583,17 @@ _outPlaceHolderInfo(StringInfo str, const PlaceHolderInfo *node)
WRITE_INT_FIELD(ph_width);
}
+static void
+_outGroupedVarInfo(StringInfo str, const GroupedVarInfo *node)
+{
+ WRITE_NODE_TYPE("GROUPEDVARINFO");
+
+ WRITE_NODE_FIELD(gvexpr);
+ WRITE_NODE_FIELD(agg_partial);
+ WRITE_UINT_FIELD(sortgroupref);
+ WRITE_BITMAPSET_FIELD(gv_eval_at);
+}
+
static void
_outMinMaxAggInfo(StringInfo str, const MinMaxAggInfo *node)
{
@@ -4144,6 +4172,9 @@ outNode(StringInfo str, const void *obj)
case T_ParamPathInfo:
_outParamPathInfo(str, obj);
break;
+ case T_RelAggInfo:
+ _outRelAggInfo(str, obj);
+ break;
case T_RestrictInfo:
_outRestrictInfo(str, obj);
break;
@@ -4162,6 +4193,9 @@ outNode(StringInfo str, const void *obj)
case T_PlaceHolderInfo:
_outPlaceHolderInfo(str, obj);
break;
+ case T_GroupedVarInfo:
+ _outGroupedVarInfo(str, obj);
+ break;
case T_MinMaxAggInfo:
_outMinMaxAggInfo(str, obj);
break;
diff --git a/src/backend/optimizer/README b/src/backend/optimizer/README
index d174b8cb73..a73a46ae31 100644
--- a/src/backend/optimizer/README
+++ b/src/backend/optimizer/README
@@ -1154,3 +1154,72 @@ breaking down aggregation or grouping over a partitioned relation into
aggregation or grouping over its partitions is called partitionwise
aggregation. Especially when the partition keys match the GROUP BY clause,
this can be significantly faster than the regular method.
+
+Aggregate push-down
+-------------------
+
+The obvious way to evaluate aggregates is to evaluate the FROM clause of the
+SQL query (this is what query_planner does) and use the resulting paths as the
+input of Agg node. However, if the groups are large enough, it may be more
+efficient to apply the partial aggregation to the output of base relation
+scan, and finalize it when we have all relations of the query joined:
+
+ EXPLAIN
+ SELECT a.i, avg(b.y)
+ FROM a JOIN b ON b.j = a.i
+ GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Seq Scan on b
+ -> Index Only Scan using a_pkey on a
+ Index Cond: (i = b.j)
+
+Thus the join above the partial aggregate node receives fewer input rows, and
+so the number of outer-to-inner pairs of tuples to be checked can be
+significantly lower, which can in turn lead to considerably lower join cost.
+
+Note that there's often no GROUP BY expression to be used for the partial
+aggregation, so we use equivalence classes to derive grouping expression: in
+the example above, the grouping key "b.j" was derived from "a.i".
+
+Also note that in this case the partial aggregate uses the "b.j" as grouping
+column although the column does not appear in the query target list. The point
+is that "b.j" is needed to evaluate the join condition, and there's no other
+way for the partial aggregate to emit its values.
+
+Besides base relation, the aggregation can also be pushed down to join:
+
+ EXPLAIN
+ SELECT a.i, avg(b.y + c.v)
+ FROM a JOIN b ON b.j = a.i
+ JOIN c ON c.k = a.i
+ WHERE b.j = c.k GROUP BY a.i;
+
+ Finalize HashAggregate
+ Group Key: a.i
+ -> Hash Join
+ Hash Cond: (b.j = a.i)
+ -> Partial HashAggregate
+ Group Key: b.j
+ -> Hash Join
+ Hash Cond: (b.j = c.k)
+ -> Seq Scan on b
+ -> Hash
+ -> Seq Scan on c
+ -> Hash
+ -> Seq Scan on a
+
+Whether the Agg node is created out of base relation or out of join, it's
+added to a separate RelOptInfo that we call "grouped relation". Grouped
+relation can be joined to a non-grouped relation, which results in a grouped
+relation too. Join of two grouped relations does not seem to be very useful
+and is currently not supported.
+
+If query_planner produces a grouped relation that contains valid paths, these
+are simply added to the UPPERREL_PARTIAL_GROUP_AGG relation. Further
+processing of these paths then does not differ from processing of other
+partially grouped paths.
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index d984da25d7..84a918dc58 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -60,6 +60,7 @@ typedef struct pushdown_safety_info
/* These parameters are set by GUC */
bool enable_geqo = false; /* just in case GUC doesn't set it */
+bool enable_agg_pushdown;
int geqo_threshold;
int min_parallel_table_scan_size;
int min_parallel_index_scan_size;
@@ -73,6 +74,7 @@ join_search_hook_type join_search_hook = NULL;
static void set_base_rel_consider_startup(PlannerInfo *root);
static void set_base_rel_sizes(PlannerInfo *root);
+static void setup_base_grouped_rels(PlannerInfo *root);
static void set_base_rel_pathlists(PlannerInfo *root);
static void set_rel_size(PlannerInfo *root, RelOptInfo *rel,
Index rti, RangeTblEntry *rte);
@@ -124,6 +126,9 @@ static void set_result_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
+static void add_grouped_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, AggStrategy aggstrategy,
+ RelAggInfo *agg_info);
static RelOptInfo *make_rel_from_joinlist(PlannerInfo *root, List *joinlist);
static bool subquery_is_pushdown_safe(Query *subquery, Query *topquery,
pushdown_safety_info *safetyInfo);
@@ -184,6 +189,13 @@ make_one_rel(PlannerInfo *root, List *joinlist)
*/
set_base_rel_sizes(root);
+ /*
+ * Now that the sizes are known, we can estimate the sizes of the grouped
+ * relations.
+ */
+ if (root->grouped_var_list)
+ setup_base_grouped_rels(root);
+
/*
* We should now have size estimates for every actual table involved in
* the query, and we also know which if any have been deleted from the
@@ -324,6 +336,48 @@ set_base_rel_sizes(PlannerInfo *root)
}
}
+/*
+ * setup_based_grouped_rels
+ * For each "plain" relation build a grouped relation if aggregate pushdown
+ * is possible and if this relation is suitable for partial aggregation.
+ */
+static void
+setup_base_grouped_rels(PlannerInfo *root)
+{
+ Index rti;
+
+ for (rti = 1; rti < root->simple_rel_array_size; rti++)
+ {
+ RelOptInfo *brel = root->simple_rel_array[rti];
+ RelOptInfo *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /* there may be empty slots corresponding to non-baserel RTEs */
+ if (brel == NULL)
+ continue;
+
+ Assert(brel->relid == rti); /* sanity check on array */
+
+ /*
+ * The aggregate push-down feature only makes sense if there are
+ * multiple base rels in the query.
+ */
+ if (!bms_nonempty_difference(root->all_baserels, brel->relids))
+ continue;
+
+ /* ignore RTEs that are "other rels" */
+ if (brel->reloptkind != RELOPT_BASEREL)
+ continue;
+
+ rel_grouped = build_simple_grouped_rel(root, brel->relid, &agg_info);
+ if (rel_grouped)
+ {
+ /* Make the relation available for joining. */
+ add_grouped_rel(root, rel_grouped, agg_info);
+ }
+ }
+}
+
/*
* set_base_rel_pathlists
* Finds all paths available for scanning each base-relation entry.
@@ -496,8 +550,21 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
}
else
{
+ RelOptInfo *rel_grouped;
+ RelAggInfo *agg_info;
+
/* Plain relation */
set_plain_rel_pathlist(root, rel, rte);
+
+ /* Add paths to the grouped relation if one exists. */
+ rel_grouped = find_grouped_rel(root, rel->relids,
+ &agg_info);
+ if (rel_grouped)
+ {
+ generate_grouping_paths(root, rel_grouped, rel,
+ agg_info);
+ set_cheapest(rel_grouped);
+ }
}
break;
case RTE_SUBQUERY:
@@ -2943,6 +3010,80 @@ generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel, bool override_r
}
}
+/*
+ * generate_grouping_paths
+ * Create partially aggregated paths and add them to grouped relation.
+ *
+ * "rel_plain" is base or join relation whose paths are not grouped.
+ */
+void
+generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
+ RelOptInfo *rel_plain, RelAggInfo *agg_info)
+{
+ ListCell *lc;
+
+ if (IS_DUMMY_REL(rel_plain))
+ {
+ mark_dummy_rel(rel_grouped);
+ return;
+ }
+
+ foreach(lc, rel_plain->pathlist)
+ {
+ Path *path = (Path *) lfirst(lc);
+
+ /*
+ * Since the path originates from the non-grouped relation which is
+ * not aware of the aggregate push-down, we must ensure that it
+ * provides the correct input for aggregation.
+ */
+ path = (Path *) create_projection_path(root, rel_grouped, path,
+ agg_info->agg_input);
+
+ /*
+ * add_grouped_path() will check whether the path has suitable
+ * pathkeys.
+ */
+ add_grouped_path(root, rel_grouped, path, AGG_SORTED, agg_info);
+
+ /*
+ * Repeated creation of hash table (for new parameter values) should
+ * be possible, does not sound like a good idea in terms of
+ * efficiency.
+ */
+ if (path->param_info == NULL)
+ add_grouped_path(root, rel_grouped, path, AGG_HASHED, agg_info);
+ }
+
+ /* Could not generate any grouped paths? */
+ if (rel_grouped->pathlist == NIL)
+ mark_dummy_rel(rel_grouped);
+}
+
+/*
+ * Apply partial aggregation to a subpath and add the AggPath to the pathlist.
+ */
+static void
+add_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ AggStrategy aggstrategy, RelAggInfo *agg_info)
+{
+ Path *agg_path;
+
+
+ if (aggstrategy == AGG_HASHED)
+ agg_path = (Path *) create_agg_hashed_path(root, rel, subpath,
+ agg_info);
+ else if (aggstrategy == AGG_SORTED)
+ agg_path = (Path *) create_agg_sorted_path(root, rel, subpath,
+ agg_info);
+ else
+ elog(ERROR, "unexpected strategy %d", aggstrategy);
+
+ /* Add the grouped path to the list of grouped base paths. */
+ if (agg_path != NULL)
+ add_path(rel, (Path *) agg_path);
+}
+
/*
* make_rel_from_joinlist
* Build access paths using a "joinlist" to guide the join path search.
@@ -3084,6 +3225,7 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
for (lev = 2; lev <= levels_needed; lev++)
{
+ RelOptInfo *rel_grouped;
ListCell *lc;
/*
@@ -3120,6 +3262,11 @@ standard_join_search(PlannerInfo *root, int levels_needed, List *initial_rels)
/* Find and save the cheapest paths for this rel */
set_cheapest(rel);
+ /* The same for grouped relation if one exists. */
+ rel_grouped = find_grouped_rel(root, rel->relids, NULL);
+ if (rel_grouped)
+ set_cheapest(rel_grouped);
+
#ifdef OPTIMIZER_DEBUG
debug_print_rel(root, rel);
#endif
diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c
index 4ff3c7a2fd..9e415fd881 100644
--- a/src/backend/optimizer/path/costsize.c
+++ b/src/backend/optimizer/path/costsize.c
@@ -4637,7 +4637,6 @@ set_baserel_size_estimates(PlannerInfo *root, RelOptInfo *rel)
0,
JOIN_INNER,
NULL);
-
rel->rows = clamp_row_est(nrows);
cost_qual_eval(&rel->baserestrictcost, rel->baserestrictinfo, root);
@@ -5608,11 +5607,11 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
foreach(lc, target->exprs)
{
Node *node = (Node *) lfirst(lc);
+ int32 item_width;
if (IsA(node, Var))
{
Var *var = (Var *) node;
- int32 item_width;
/* We should not see any upper-level Vars here */
Assert(var->varlevelsup == 0);
@@ -5643,6 +5642,20 @@ set_pathtarget_cost_width(PlannerInfo *root, PathTarget *target)
Assert(item_width > 0);
tuple_width += item_width;
}
+ else if (IsA(node, Aggref))
+ {
+ /*
+ * If the target is evaluated by AggPath, it'll care of cost
+ * estimate. If the target is above AggPath (typically target of a
+ * join relation that contains grouped relation), the cost of
+ * Aggref should not be accounted for again.
+ *
+ * On the other hand, width is always needed.
+ */
+ item_width = get_typavgwidth(exprType(node), exprTypmod(node));
+ Assert(item_width > 0);
+ tuple_width += item_width;
+ }
else
{
/*
diff --git a/src/backend/optimizer/path/equivclass.c b/src/backend/optimizer/path/equivclass.c
index b99cec00cb..81970fed15 100644
--- a/src/backend/optimizer/path/equivclass.c
+++ b/src/backend/optimizer/path/equivclass.c
@@ -2841,6 +2841,136 @@ is_redundant_derived_clause(RestrictInfo *rinfo, List *clauselist)
return false;
}
+/*
+ * translate_expression_to_rels
+ * If the appropriate equivalence classes exist, replace vars in
+ * gvi->gvexpr with vars whose varno is equal to relid. Return NULL if
+ * translation is not possible or needed.
+ *
+ * Note: Currently we only translate Var expressions. This is subject to
+ * change as the aggregate push-down feature gets enhanced.
+ */
+GroupedVarInfo *
+translate_expression_to_rel(PlannerInfo *root, GroupedVarInfo *gvi,
+ Index relid)
+{
+ Var *var;
+ ListCell *l1;
+ bool found_orig = false;
+ Var *var_translated = NULL;
+ GroupedVarInfo *result;
+
+ /* Can't do anything w/o equivalence classes. */
+ if (root->eq_classes == NIL)
+ return NULL;
+
+ var = castNode(Var, gvi->gvexpr);
+
+ /*
+ * Do we need to translate the var?
+ */
+ if (var->varno == relid)
+ return NULL;
+
+ /*
+ * Find the replacement var.
+ */
+ foreach(l1, root->eq_classes)
+ {
+ EquivalenceClass *ec = lfirst_node(EquivalenceClass, l1);
+ ListCell *l2;
+
+ /* TODO Check if any other EC kind should be ignored. */
+ if (ec->ec_has_volatile || ec->ec_below_outer_join || ec->ec_broken)
+ continue;
+
+ /* Single-element EC can hardly help in translations. */
+ if (list_length(ec->ec_members) == 1)
+ continue;
+
+ /*
+ * Collect all vars of this EC and their varnos.
+ *
+ * ec->ec_relids does not help because we're only interested in a
+ * subset of EC members.
+ */
+ foreach(l2, ec->ec_members)
+ {
+ EquivalenceMember *em = lfirst_node(EquivalenceMember, l2);
+ Var *ec_var;
+
+ /*
+ * The grouping expressions derived here are used to evaluate
+ * possibility to push aggregation down to RELOPT_BASEREL or
+ * RELOPT_JOINREL relations, and to construct reltargets for the
+ * grouped rels. We're not interested at the moment whether the
+ * relations do have children.
+ */
+ if (em->em_is_child)
+ continue;
+
+ if (!IsA(em->em_expr, Var))
+ continue;
+
+ ec_var = castNode(Var, em->em_expr);
+ if (equal(ec_var, var))
+ found_orig = true;
+ else if (ec_var->varno == relid)
+ var_translated = ec_var;
+
+ if (found_orig && var_translated)
+ {
+ /*
+ * The replacement Var must have the same data type, otherwise
+ * the values are not guaranteed to be grouped in the same way
+ * as values of the original Var.
+ */
+ if (ec_var->vartype != var->vartype)
+ return NULL;
+
+ break;
+ }
+ }
+
+ if (found_orig)
+ {
+ /*
+ * The same expression probably does not exist in multiple ECs.
+ */
+ if (var_translated == NULL)
+ {
+ /*
+ * Failed to translate the expression.
+ */
+ return NULL;
+ }
+ else
+ {
+ /* Success. */
+ break;
+ }
+ }
+ else
+ {
+ /*
+ * Vars of the requested relid can be in the next ECs too.
+ */
+ var_translated = NULL;
+ }
+ }
+
+ if (!found_orig)
+ return NULL;
+
+ result = makeNode(GroupedVarInfo);
+ memcpy(result, gvi, sizeof(GroupedVarInfo));
+
+ result->gv_eval_at = bms_make_singleton(relid);
+ result->gvexpr = (Expr *) var_translated;
+
+ return result;
+}
+
/*
* is_redundant_with_indexclauses
* Test whether rinfo is redundant with any clause in the IndexClause
diff --git a/src/backend/optimizer/path/joinrels.c b/src/backend/optimizer/path/joinrels.c
index 2d343cd293..1b0dea491a 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -21,6 +21,7 @@
#include "optimizer/paths.h"
#include "partitioning/partbounds.h"
#include "utils/memutils.h"
+#include "utils/selfuncs.h"
static void make_rels_by_clause_joins(PlannerInfo *root,
@@ -35,6 +36,10 @@ static bool has_legal_joinclause(PlannerInfo *root, RelOptInfo *rel);
static bool restriction_is_constant_false(List *restrictlist,
RelOptInfo *joinrel,
bool only_pushed_down);
+static RelOptInfo *make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1,
+ RelOptInfo *rel2,
+ RelAggInfo *agg_info,
+ RelOptInfo *rel_agg_input);
static void populate_joinrel_with_paths(PlannerInfo *root, RelOptInfo *rel1,
RelOptInfo *rel2, RelOptInfo *joinrel,
SpecialJoinInfo *sjinfo, List *restrictlist);
@@ -669,21 +674,20 @@ join_is_legal(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
return true;
}
-
/*
- * make_join_rel
- * Find or create a join RelOptInfo that represents the join of
- * the two given rels, and add to it path information for paths
- * created with the two rels as outer and inner rel.
- * (The join rel may already contain paths generated from other
- * pairs of rels that add up to the same set of base rels.)
+ * make_join_rel_common
+ * The workhorse of make_join_rel().
+ *
+ * 'agg_info' contains the reltarget of grouped relation and everything we
+ * need to aggregate the join result. If NULL, then the join relation should
+ * not be grouped.
*
- * NB: will return NULL if attempted join is not valid. This can happen
- * when working with outer joins, or with IN or EXISTS clauses that have been
- * turned into joins.
+ * 'rel_agg_input' describes the AggPath input relation if the join output
+ * should be aggregated. If NULL is passed, do not aggregate the join output.
*/
-RelOptInfo *
-make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+static RelOptInfo *
+make_join_rel_common(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2,
+ RelAggInfo *agg_info, RelOptInfo *rel_agg_input)
{
Relids joinrelids;
SpecialJoinInfo *sjinfo;
@@ -744,7 +748,7 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
* goes with this particular joining.
*/
joinrel = build_join_rel(root, joinrelids, rel1, rel2, sjinfo,
- &restrictlist);
+ &restrictlist, agg_info);
/*
* If we've already proven this join is empty, we needn't consider any
@@ -757,14 +761,175 @@ make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
}
/* Add paths to the join relation. */
- populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
- restrictlist);
+ if (rel_agg_input == NULL)
+ {
+ /*
+ * Simply join the input relations, whether both are plain or one of
+ * them is grouped.
+ */
+ populate_joinrel_with_paths(root, rel1, rel2, joinrel, sjinfo,
+ restrictlist);
+ }
+ else
+ {
+ /* The join relation is grouped. */
+ Assert(agg_info != NULL);
+
+ /*
+ * Apply partial aggregation to the paths of rel_agg_input and add the
+ * resulting paths to joinrel.
+ */
+ generate_grouping_paths(root, joinrel, rel_agg_input, agg_info);
+ }
bms_free(joinrelids);
return joinrel;
}
+/*
+ * make_join_rel_combined
+ * Join grouped relation to non-grouped one.
+ */
+static void
+make_join_rel_combined(PlannerInfo *root, RelOptInfo *rel1,
+ RelOptInfo *rel2,
+ RelAggInfo *agg_info)
+{
+ RelOptInfo *rel1_grouped;
+ RelOptInfo *rel2_grouped;
+ bool rel1_grouped_useful = false;
+ bool rel2_grouped_useful = false;
+
+ /* Retrieve the grouped relations. */
+ rel1_grouped = find_grouped_rel(root, rel1->relids, NULL);
+ rel2_grouped = find_grouped_rel(root, rel2->relids, NULL);
+
+ /*
+ * Dummy rel may indicate a join relation that is able to generate grouped
+ * paths as such (i.e. it has valid agg_info), but for which the path
+ * actually could not be created (e.g. only AGG_HASHED strategy was
+ * possible but work_mem was not sufficient for hash table).
+ */
+ rel1_grouped_useful = rel1_grouped != NULL && !IS_DUMMY_REL(rel1_grouped);
+ rel2_grouped_useful = rel2_grouped != NULL && !IS_DUMMY_REL(rel2_grouped);
+
+ /* Nothing to do if there's no grouped relation. */
+ if (!rel1_grouped_useful && !rel2_grouped_useful)
+ return;
+
+ /*
+ * At maximum one input rel can be grouped (here we don't care if any rel
+ * is eventually dummy, the existence of grouped rel indicates that
+ * aggregates can be pushed down to it). If both were grouped, then
+ * grouping of one side would change the occurrence of the other side's
+ * aggregate transient states on the input of the final aggregation. This
+ * can be handled by adjusting the transient states, but it's not worth
+ * the effort because it's hard to find a use case for this kind of join.
+ *
+ * XXX If the join of two grouped rels is implemented someday, note that
+ * both rels can have aggregates, so it'd be hard to join grouped rel to
+ * non-grouped here: 1) such a "mixed join" would require a special
+ * target, 2) both AGGSPLIT_FINAL_DESERIAL and AGGSPLIT_SIMPLE aggregates
+ * could appear in the target of the final aggregation node, originating
+ * from the grouped and the non-grouped input rel respectively.
+ */
+ if (rel1_grouped && rel2_grouped)
+ return;
+
+ if (rel1_grouped_useful)
+ make_join_rel_common(root, rel1_grouped, rel2, agg_info, NULL);
+ else if (rel2_grouped_useful)
+ make_join_rel_common(root, rel1, rel2_grouped, agg_info, NULL);
+}
+
+/*
+ * make_join_rel
+ * Find or create a join RelOptInfo that represents the join of
+ * the two given rels, and add to it path information for paths
+ * created with the two rels as outer and inner rel.
+ * (The join rel may already contain paths generated from other
+ * pairs of rels that add up to the same set of base rels.)
+ *
+ * In addition to creating an ordinary join relation, try to create a
+ * grouped one. There are two strategies to achieve that: join a grouped
+ * relation to plain one, or join two plain relations and apply partial
+ * aggregation to the result.
+ *
+ * NB: will return NULL if attempted join is not valid. This can happen when
+ * working with outer joins, or with IN or EXISTS clauses that have been
+ * turned into joins. Besides that, NULL is also returned if caller is
+ * interested in a grouped relation but it could not be created.
+ *
+ * Only the plain relation is returned; if grouped relation exists, it can be
+ * retrieved using find_grouped_rel().
+ */
+RelOptInfo *
+make_join_rel(PlannerInfo *root, RelOptInfo *rel1, RelOptInfo *rel2)
+{
+ Relids joinrelids;
+ RelAggInfo *agg_info = NULL;
+ RelOptInfo *joinrel,
+ *joinrel_plain;
+
+ /* 1) form the plain join. */
+ joinrel = make_join_rel_common(root, rel1, rel2, NULL, NULL);
+ joinrel_plain = joinrel;
+
+ if (joinrel_plain == NULL)
+ return joinrel_plain;
+
+ /*
+ * We're done if there are no grouping expressions nor aggregates.
+ */
+ if (root->grouped_var_list == NIL)
+ return joinrel_plain;
+
+ joinrelids = bms_union(rel1->relids, rel2->relids);
+ joinrel = find_grouped_rel(root, joinrelids, &agg_info);
+
+ if (joinrel != NULL)
+ {
+ /*
+ * If the same grouped joinrel was already formed, just with the base
+ * rels divided between rel1 and rel2 in a different way, the matching
+ * agg_info should already be there.
+ */
+ Assert(agg_info != NULL);
+ }
+ else
+ {
+ /*
+ * agg_info must be created from scratch.
+ */
+ agg_info = create_rel_agg_info(root, joinrel_plain);
+
+ /* Cannot we build grouped join? */
+ if (agg_info == NULL)
+ return joinrel_plain;
+
+ /*
+ * The number of aggregate input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ agg_info->input_rows = joinrel_plain->rows;
+ }
+
+ /*
+ * 2) join two plain rels and aggregate the join paths. Aggregate
+ * push-down only makes sense if the join is not the top-level one.
+ */
+ if (bms_nonempty_difference(root->all_baserels, joinrelids))
+ make_join_rel_common(root, rel1, rel2, agg_info, joinrel_plain);
+
+ /*
+ * 3) combine plain and grouped relations.
+ */
+ make_join_rel_combined(root, rel1, rel2, agg_info);
+
+ return joinrel_plain;
+}
+
/*
* populate_joinrel_with_paths
* Add paths to the given joinrel for given pair of joining relations. The
diff --git a/src/backend/optimizer/plan/initsplan.c b/src/backend/optimizer/plan/initsplan.c
index e978b491f6..b10cdc8368 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -14,6 +14,7 @@
*/
#include "postgres.h"
+#include "access/nbtree.h"
#include "catalog/pg_class.h"
#include "catalog/pg_type.h"
#include "nodes/makefuncs.h"
@@ -33,6 +34,7 @@
#include "parser/analyze.h"
#include "rewrite/rewriteManip.h"
#include "utils/lsyscache.h"
+#include "utils/typcache.h"
/* These parameters are set by GUC */
int from_collapse_limit;
@@ -47,6 +49,8 @@ typedef struct PostponedQual
} PostponedQual;
+static void create_aggregate_grouped_var_infos(PlannerInfo *root);
+static void create_grouping_expr_grouped_var_infos(PlannerInfo *root);
static void extract_lateral_references(PlannerInfo *root, RelOptInfo *brel,
Index rtindex);
static List *deconstruct_recurse(PlannerInfo *root, Node *jtnode,
@@ -272,6 +276,292 @@ add_vars_to_targetlist(PlannerInfo *root, List *vars,
}
}
+/*
+ * Add GroupedVarInfo to grouped_var_list for each aggregate as well as for
+ * each possible grouping expression.
+ *
+ * root->group_pathkeys must be setup before this function is called.
+ */
+extern void
+setup_aggregate_pushdown(PlannerInfo *root)
+{
+ ListCell *lc;
+
+ /*
+ * Isn't user interested in the aggregate push-down feature?
+ */
+ if (!enable_agg_pushdown)
+ return;
+
+ /* The feature can only be applied to grouped aggregation. */
+ if (!root->parse->groupClause)
+ return;
+
+ /*
+ * Grouping sets require multiple different groupings but the base
+ * relation can only generate one.
+ */
+ if (root->parse->groupingSets)
+ return;
+
+ /*
+ * SRF is not allowed in the aggregate argument and we don't even want it
+ * in the GROUP BY clause, so forbid it in general. It needs to be
+ * analyzed if evaluation of a GROUP BY clause containing SRF below the
+ * query targetlist would be correct. Currently it does not seem to be an
+ * important use case.
+ */
+ if (root->parse->hasTargetSRFs)
+ return;
+
+ /* Create GroupedVarInfo per (distinct) aggregate. */
+ create_aggregate_grouped_var_infos(root);
+
+ /* Isn't there any aggregate to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /* Create GroupedVarInfo per grouping expression. */
+ create_grouping_expr_grouped_var_infos(root);
+
+ /* Isn't there any useful grouping expression for aggregate push-down? */
+ if (root->grouped_var_list == NIL)
+ return;
+
+ /*
+ * Now that we know that grouping can be pushed down, search for the
+ * maximum sortgroupref. The base relations may need it if extra grouping
+ * expressions get added to them.
+ */
+ Assert(root->max_sortgroupref == 0);
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc);
+
+ if (te->ressortgroupref > root->max_sortgroupref)
+ root->max_sortgroupref = te->ressortgroupref;
+ }
+}
+
+/*
+ * Create GroupedVarInfo for each distinct aggregate.
+ *
+ * If any aggregate is not suitable, set root->grouped_var_list to NIL and
+ * return.
+ */
+static void
+create_aggregate_grouped_var_infos(PlannerInfo *root)
+{
+ List *tlist_exprs;
+ ListCell *lc;
+
+ Assert(root->grouped_var_list == NIL);
+
+ tlist_exprs = pull_var_clause((Node *) root->processed_tlist,
+ PVC_INCLUDE_AGGREGATES);
+
+ /*
+ * Although GroupingFunc is related to root->parse->groupingSets, this
+ * field does not necessarily reflect its presence.
+ */
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+
+ if (IsA(expr, GroupingFunc))
+ return;
+ }
+
+ /*
+ * Aggregates within the HAVING clause need to be processed in the same
+ * way as those in the main targetlist.
+ *
+ * Note that the contained aggregates will be pushed down, but the
+ * containing HAVING clause must be ignored until the aggregation is
+ * finalized.
+ */
+ if (root->parse->havingQual != NULL)
+ {
+ List *having_exprs;
+
+ having_exprs = pull_var_clause((Node *) root->parse->havingQual,
+ PVC_INCLUDE_AGGREGATES);
+ if (having_exprs != NIL)
+ tlist_exprs = list_concat(tlist_exprs, having_exprs);
+ }
+
+ if (tlist_exprs == NIL)
+ return;
+
+ foreach(lc, tlist_exprs)
+ {
+ Expr *expr = (Expr *) lfirst(lc);
+ Aggref *aggref;
+ ListCell *lc2;
+ GroupedVarInfo *gvi;
+ bool exists;
+
+ /*
+ * tlist_exprs may also contain Vars, but we only need Aggrefs.
+ */
+ if (IsA(expr, Var))
+ continue;
+
+ aggref = castNode(Aggref, expr);
+
+ /* TODO Think if (some of) these can be handled. */
+ if (aggref->aggvariadic ||
+ aggref->aggdirectargs || aggref->aggorder ||
+ aggref->aggdistinct)
+ {
+ /*
+ * Aggregation push-down is not useful if at least one aggregate
+ * cannot be evaluated below the top-level join.
+ *
+ * XXX Is it worth freeing the GroupedVarInfos and their subtrees?
+ */
+ root->grouped_var_list = NIL;
+ break;
+ }
+
+ /* Does GroupedVarInfo for this aggregate already exist? */
+ exists = false;
+ foreach(lc2, root->grouped_var_list)
+ {
+ gvi = lfirst_node(GroupedVarInfo, lc2);
+
+ if (equal(expr, gvi->gvexpr))
+ {
+ exists = true;
+ break;
+ }
+ }
+
+ /* Construct a new GroupedVarInfo if does not exist yet. */
+ if (!exists)
+ {
+ Relids relids;
+
+ gvi = makeNode(GroupedVarInfo);
+ gvi->gvexpr = (Expr *) copyObject(aggref);
+
+ /* Find out where the aggregate should be evaluated. */
+ relids = pull_varnos((Node *) aggref);
+ if (!bms_is_empty(relids))
+ gvi->gv_eval_at = relids;
+ else
+ gvi->gv_eval_at = NULL;
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ }
+
+ list_free(tlist_exprs);
+}
+
+/*
+ * Create GroupedVarInfo for each expression usable as grouping key.
+ *
+ * In addition to the expressions of the query targetlist, group_pathkeys is
+ * also considered the source of grouping expressions. That increases the
+ * chance to get the relation output grouped.
+ */
+static void
+create_grouping_expr_grouped_var_infos(PlannerInfo *root)
+{
+ ListCell *l1,
+ *l2;
+ List *exprs = NIL;
+ List *sortgrouprefs = NIL;
+
+ /*
+ * Make sure GroupedVarInfo exists for each expression usable as grouping
+ * key.
+ */
+ foreach(l1, root->parse->groupClause)
+ {
+ SortGroupClause *sgClause;
+ TargetEntry *te;
+ Index sortgroupref;
+ TypeCacheEntry *tce;
+ Oid equalimageproc;
+
+ sgClause = lfirst_node(SortGroupClause, l1);
+ te = get_sortgroupclause_tle(sgClause, root->processed_tlist);
+ sortgroupref = te->ressortgroupref;
+
+ Assert(sortgroupref > 0);
+
+ /*
+ * Non-zero sortgroupref does not necessarily imply grouping
+ * expression: data can also be sorted by aggregate.
+ */
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ /*
+ * The aggregate push-down feature currently supports only plain Vars
+ * as grouping expressions.
+ */
+ if (!IsA(te->expr, Var))
+ {
+ root->grouped_var_list = NIL;
+ return;
+ }
+
+ /*
+ * Aggregate push-down is only possible if equality of grouping keys
+ * per the equality operator implies bitwise equality. Otherwise, if
+ * we put keys of different byte images into the same group, we lose
+ * some information that may be needed to evaluate join clauses above
+ * the pushed-down aggregate node, or the WHERE clause.
+ *
+ * For example, the NUMERIC data type is not supported because values
+ * that fall into the same group according to the equality operator
+ * (e.g. 0 and 0.0) can have different scale.
+ */
+ tce = lookup_type_cache(exprType((Node *) te->expr),
+ TYPECACHE_BTREE_OPFAMILY);
+ if (!OidIsValid(tce->btree_opf) ||
+ !OidIsValid(tce->btree_opintype))
+ goto fail;
+
+ equalimageproc = get_opfamily_proc(tce->btree_opf,
+ tce->btree_opintype,
+ tce->btree_opintype,
+ BTEQUALIMAGE_PROC);
+ if (!OidIsValid(equalimageproc) ||
+ !DatumGetBool(OidFunctionCall1Coll(equalimageproc,
+ tce->typcollation,
+ ObjectIdGetDatum(tce->btree_opintype))))
+ goto fail;
+
+ exprs = lappend(exprs, te->expr);
+ sortgrouprefs = lappend_int(sortgrouprefs, sortgroupref);
+ }
+
+ /*
+ * Construct GroupedVarInfo for each expression.
+ */
+ forboth(l1, exprs, l2, sortgrouprefs)
+ {
+ Var *var = lfirst_node(Var, l1);
+ int sortgroupref = lfirst_int(l2);
+ GroupedVarInfo *gvi = makeNode(GroupedVarInfo);
+
+ gvi->gvexpr = (Expr *) copyObject(var);
+ gvi->sortgroupref = sortgroupref;
+
+ /* Find out where the expression should be evaluated. */
+ gvi->gv_eval_at = bms_make_singleton(var->varno);
+
+ root->grouped_var_list = lappend(root->grouped_var_list, gvi);
+ }
+ return;
+
+fail:
+ root->grouped_var_list = NIL;
+}
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planmain.c b/src/backend/optimizer/plan/planmain.c
index 5fa33ec200..cb82d3b8de 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -66,6 +66,7 @@ query_planner(PlannerInfo *root,
* here.
*/
root->join_rel_list = makeNode(RelInfoList);
+ root->agg_info_list = makeNode(RelInfoList);
root->join_rel_level = NULL;
root->join_cur_level = 0;
root->canon_pathkeys = NIL;
@@ -74,6 +75,7 @@ query_planner(PlannerInfo *root,
root->full_join_clauses = NIL;
root->join_info_list = NIL;
root->placeholder_list = NIL;
+ root->grouped_var_list = NIL;
root->fkey_list = NIL;
root->initial_rels = NIL;
@@ -252,6 +254,16 @@ query_planner(PlannerInfo *root,
*/
extract_restriction_or_clauses(root);
+ /*
+ * If the query result can be grouped, check if any grouping can be
+ * performed below the top-level join. If so, setup
+ * root->grouped_var_list.
+ *
+ * The base relations should be fully initialized now, so that we have
+ * enough info to decide whether grouping is possible.
+ */
+ setup_aggregate_pushdown(root);
+
/*
* Now expand appendrels by adding "otherrels" for their children. We
* delay this to the end so that we have as much information as possible
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 4131019fc9..6fe12d08ce 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -625,6 +625,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
memset(root->upper_rels, 0, sizeof(root->upper_rels));
memset(root->upper_targets, 0, sizeof(root->upper_targets));
root->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
root->grouping_map = NULL;
root->minmax_aggs = NIL;
root->qual_security_level = 0;
@@ -3893,7 +3894,8 @@ create_grouping_paths(PlannerInfo *root,
else
extra.patype = PARTITIONWISE_AGGREGATE_NONE;
- create_ordinary_grouping_paths(root, input_rel, grouped_rel,
+ create_ordinary_grouping_paths(root, input_rel,
+ grouped_rel,
agg_costs, gd, &extra,
&partially_grouped_rel);
}
@@ -4100,11 +4102,11 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
bool force_rel_creation;
/*
- * If we're doing partitionwise aggregation at this level, force
- * creation of a partially_grouped_rel so we can add partitionwise
- * paths to it.
+ * If we're doing partitionwise aggregation at this level or if
+ * aggregate push-down succeeded to create some paths, force creation
+ * of a partially_grouped_rel so we can add the related paths to it.
*/
- force_rel_creation = (patype == PARTITIONWISE_AGGREGATE_PARTIAL);
+ force_rel_creation = patype == PARTITIONWISE_AGGREGATE_PARTIAL;
partially_grouped_rel =
create_partial_grouping_paths(root,
@@ -4137,10 +4139,14 @@ create_ordinary_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
/* Gather any partially grouped partial paths. */
if (partially_grouped_rel && partially_grouped_rel->partial_pathlist)
- {
gather_grouping_paths(root, partially_grouped_rel);
+
+ /*
+ * The non-partial paths can come either from the Gather above or from
+ * aggregate push-down.
+ */
+ if (partially_grouped_rel && partially_grouped_rel->pathlist)
set_cheapest(partially_grouped_rel);
- }
/*
* Estimate number of groups.
@@ -6868,6 +6874,13 @@ create_partial_grouping_paths(PlannerInfo *root,
bool can_hash = (extra->flags & GROUPING_CAN_USE_HASH) != 0;
bool can_sort = (extra->flags & GROUPING_CAN_USE_SORT) != 0;
+ /*
+ * The output relation could have been already created due to aggregate
+ * push-down.
+ */
+ partially_grouped_rel = find_grouped_rel(root, input_rel->relids, NULL);
+ Assert(enable_agg_pushdown || partially_grouped_rel == NULL);
+
/*
* Consider whether we should generate partially aggregated non-partial
* paths. We can only do this if we have a non-partial path, and only if
@@ -6894,16 +6907,18 @@ create_partial_grouping_paths(PlannerInfo *root,
*/
if (cheapest_total_path == NULL &&
cheapest_partial_path == NULL &&
- !force_rel_creation)
+ !force_rel_creation &&
+ partially_grouped_rel == NULL)
return NULL;
/*
* Build a new upper relation to represent the result of partially
* aggregating the rows from the input relation.
*/
- partially_grouped_rel = fetch_upper_rel(root,
- UPPERREL_PARTIAL_GROUP_AGG,
- grouped_rel->relids);
+ if (partially_grouped_rel == NULL)
+ partially_grouped_rel = fetch_upper_rel(root,
+ UPPERREL_PARTIAL_GROUP_AGG,
+ grouped_rel->relids);
partially_grouped_rel->consider_parallel =
grouped_rel->consider_parallel;
partially_grouped_rel->reloptkind = grouped_rel->reloptkind;
@@ -6917,10 +6932,14 @@ create_partial_grouping_paths(PlannerInfo *root,
* emit the same tlist as regular aggregate paths, because (1) we must
* include Vars and Aggrefs needed in HAVING, which might not appear in
* the result tlist, and (2) the Aggrefs must be set in partial mode.
+ *
+ * If the target was already created for the sake of aggregate push-down,
+ * it should be compatible with what we'd create here.
*/
- partially_grouped_rel->reltarget =
- make_partial_grouping_target(root, grouped_rel->reltarget,
- extra->havingQual);
+ if (partially_grouped_rel->reltarget->exprs == NIL)
+ partially_grouped_rel->reltarget =
+ make_partial_grouping_target(root, grouped_rel->reltarget,
+ extra->havingQual);
if (!extra->partial_costs_set)
{
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index baefe0e946..026c098d11 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -2457,6 +2457,39 @@ fix_join_expr_mutator(Node *node, fix_join_expr_context *context)
/* No referent found for Var */
elog(ERROR, "variable not found in subplan target lists");
}
+ if (IsA(node, Aggref))
+ {
+ Aggref *aggref = castNode(Aggref, node);
+
+ /*
+ * The upper plan targetlist can contain Aggref whose value has
+ * already been evaluated by the subplan. However this can only happen
+ * with specific value of aggsplit.
+ */
+ if (aggref->aggsplit == AGGSPLIT_INITIAL_SERIAL)
+ {
+ /* See if the Aggref has bubbled up from a lower plan node */
+ if (context->outer_itlist && context->outer_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->outer_itlist,
+ OUTER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ if (context->inner_itlist && context->inner_itlist->has_non_vars)
+ {
+ newvar = search_indexed_tlist_for_non_var((Expr *) node,
+ context->inner_itlist,
+ INNER_VAR);
+ if (newvar)
+ return (Node *) newvar;
+ }
+ }
+
+ /* No referent found for Aggref */
+ elog(ERROR, "Aggref not found in subplan target lists");
+ }
if (IsA(node, PlaceHolderVar))
{
PlaceHolderVar *phv = (PlaceHolderVar *) node;
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
index 14521728c6..d34065d61e 100644
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -923,6 +923,7 @@ pull_up_simple_subquery(PlannerInfo *root, Node *jtnode, RangeTblEntry *rte,
memset(subroot->upper_rels, 0, sizeof(subroot->upper_rels));
memset(subroot->upper_targets, 0, sizeof(subroot->upper_targets));
subroot->processed_tlist = NIL;
+ root->max_sortgroupref = 0;
subroot->grouping_map = NULL;
subroot->minmax_aggs = NIL;
subroot->qual_security_level = 0;
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index e845a4b1ae..437d6d3f42 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -2528,8 +2528,7 @@ create_projection_path(PlannerInfo *root,
pathnode->path.pathtype = T_Result;
pathnode->path.parent = rel;
pathnode->path.pathtarget = target;
- /* For now, assume we are above any joins, so no parameterization */
- pathnode->path.param_info = NULL;
+ pathnode->path.param_info = subpath->param_info;
pathnode->path.parallel_aware = false;
pathnode->path.parallel_safe = rel->consider_parallel &&
subpath->parallel_safe &&
@@ -3020,6 +3019,146 @@ create_agg_path(PlannerInfo *root,
return pathnode;
}
+/*
+ * Apply AGG_SORTED aggregation path to subpath if it's suitably sorted.
+ *
+ * NULL is returned if sorting of subpath output is not suitable.
+ */
+AggPath *
+create_agg_sorted_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
+ RelAggInfo *agg_info)
+{
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ ListCell *lc1;
+ List *key_subset = NIL;
+ AggPath *result = NULL;
+
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ if (subpath->pathkeys == NIL)
+ return NULL;
+
+ if (!grouping_is_sortable(root->parse->groupClause))
+ return NULL;
+
+ /*
+ * Find all query pathkeys that our relation does affect.
+ */
+ foreach(lc1, root->group_pathkeys)
+ {
+ PathKey *gkey = castNode(PathKey, lfirst(lc1));
+ ListCell *lc2;
+
+ foreach(lc2, subpath->pathkeys)
+ {
+ PathKey *skey = castNode(PathKey, lfirst(lc2));
+
+ if (skey == gkey)
+ {
+ key_subset = lappend(key_subset, gkey);
+ break;
+ }
+ }
+ }
+
+ if (key_subset == NIL)
+ return NULL;
+
+ /* Check if AGG_SORTED is useful for the whole query. */
+ if (!pathkeys_contained_in(key_subset, subpath->pathkeys))
+ return NULL;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, (Node *) agg_exprs, aggsplit, &agg_costs);
+
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until the
+ * final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath, target,
+ AGG_SORTED, aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+
+ /* The agg path should require no fewer parameters than the plain one. */
+ result->path.param_info = subpath->param_info;
+
+ return result;
+}
+
+/*
+ * Apply AGG_HASHED aggregation to subpath.
+ */
+AggPath *
+create_agg_hashed_path(PlannerInfo *root, RelOptInfo *rel,
+ Path *subpath, RelAggInfo *agg_info)
+{
+ bool can_hash;
+ Node *agg_exprs;
+ AggSplit aggsplit;
+ AggClauseCosts agg_costs;
+ PathTarget *target;
+ double dNumGroups;
+ double hashaggtablesize;
+ Query *parse = root->parse;
+ AggPath *result = NULL;
+
+ /* Do not try to create hash table for each parameter value. */
+ Assert(subpath->param_info == NULL);
+
+ aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ agg_exprs = (Node *) agg_info->agg_exprs;
+ target = agg_info->target;
+
+ MemSet(&agg_costs, 0, sizeof(AggClauseCosts));
+ get_agg_clause_costs(root, agg_exprs, aggsplit, &agg_costs);
+
+ can_hash = (parse->groupClause != NIL &&
+ parse->groupingSets == NIL &&
+ agg_costs.numOrderedAggs == 0 &&
+ grouping_is_hashable(parse->groupClause));
+
+ if (can_hash)
+ {
+ Assert(agg_info->group_exprs != NIL);
+ dNumGroups = estimate_num_groups(root, agg_info->group_exprs,
+ subpath->rows, NULL);
+
+ hashaggtablesize = estimate_hashagg_tablesize(subpath, &agg_costs,
+ dNumGroups);
+
+ if (hashaggtablesize < work_mem * 1024L)
+ {
+ /*
+ * qual is NIL because the HAVING clause cannot be evaluated until
+ * the final value of the aggregate is known.
+ */
+ result = create_agg_path(root, rel, subpath,
+ target,
+ AGG_HASHED,
+ aggsplit,
+ agg_info->group_clauses,
+ NIL,
+ &agg_costs,
+ dNumGroups);
+ }
+ }
+
+ return result;
+}
+
/*
* create_groupingsets_path
* Creates a pathnode that represents performing GROUPING SETS aggregation
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index a95e6364ae..7460794b46 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -18,17 +18,23 @@
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
+#include "catalog/pg_class_d.h"
+#include "catalog/pg_constraint.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
+#include "optimizer/optimizer.h"
#include "optimizer/inherit.h"
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/placeholder.h"
#include "optimizer/plancat.h"
+#include "optimizer/planner.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
+#include "parser/parse_oper.h"
#include "utils/hsearch.h"
+#include "utils/selfuncs.h"
#include "utils/lsyscache.h"
@@ -76,6 +82,11 @@ static void build_child_join_reltarget(PlannerInfo *root,
RelOptInfo *childrel,
int nappinfos,
AppendRelInfo **appinfos);
+static bool init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p);
+static bool is_var_in_aggref_only(PlannerInfo *root, Var *var);
+static bool is_var_needed_by_join(PlannerInfo *root, Var *var, RelOptInfo *rel);
/*
@@ -370,6 +381,109 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo *parent)
return rel;
}
+/*
+ * build_simple_grouped_rel
+ * Construct a new RelOptInfo for a grouped base relation out of an
+ * existing non-grouped relation. On success, pointer to the corresponding
+ * RelAggInfo is stored in *agg_info_p in addition to returning the grouped
+ * relation.
+ */
+RelOptInfo *
+build_simple_grouped_rel(PlannerInfo *root, int relid,
+ RelAggInfo **agg_info_p)
+{
+ RangeTblEntry *rte;
+ RelOptInfo *rel_plain,
+ *rel_grouped;
+ RelAggInfo *agg_info;
+
+ /* Isn't there any grouping expression to be pushed down? */
+ if (root->grouped_var_list == NIL)
+ return NULL;
+
+ rel_plain = root->simple_rel_array[relid];
+
+ /* Caller should only pass rti that represents base relation. */
+ Assert(rel_plain != NULL);
+
+ /*
+ * Not all RTE kinds are supported when grouping is considered.
+ *
+ * TODO Consider relaxing some of these restrictions.
+ */
+ rte = root->simple_rte_array[rel_plain->relid];
+ if (rte->rtekind != RTE_RELATION ||
+ rte->relkind == RELKIND_FOREIGN_TABLE ||
+ rte->tablesample != NULL)
+ return NULL;
+
+ /*
+ * Grouped append relation is not supported yet.
+ */
+ if (rte->inh)
+ return NULL;
+
+ /*
+ * Currently we do not support child relations ("other rels").
+ */
+ if (rel_plain->reloptkind != RELOPT_BASEREL)
+ return NULL;
+
+ /*
+ * Prepare the information we need for aggregation of the rel contents.
+ */
+ agg_info = create_rel_agg_info(root, rel_plain);
+ if (agg_info == NULL)
+ return NULL;
+
+ /*
+ * TODO Consider if 1) a flat copy is o.k., 2) it's safer in terms of
+ * adding new fields to RelOptInfo) to copy everything and then reset some
+ * fields, or to zero the structure and copy individual fields.
+ */
+ rel_grouped = makeNode(RelOptInfo);
+ memcpy(rel_grouped, rel_plain, sizeof(RelOptInfo));
+
+ /*
+ * Note on consider_startup: while the AGG_HASHED strategy needs the whole
+ * relation, AGG_SORTED does not. Therefore we do not force
+ * consider_startup to false.
+ */
+
+ /*
+ * Set the appropriate target for grouped paths.
+ *
+ * reltarget should match the target of partially aggregated paths.
+ */
+ rel_grouped->reltarget = agg_info->target;
+
+ /*
+ * Grouped paths must not be mixed with the plain ones.
+ */
+ rel_grouped->pathlist = NIL;
+ rel_grouped->partial_pathlist = NIL;
+ rel_grouped->cheapest_startup_path = NULL;
+ rel_grouped->cheapest_total_path = NULL;
+ rel_grouped->cheapest_unique_path = NULL;
+ rel_grouped->cheapest_parameterized_paths = NIL;
+
+ /*
+ * The number of aggregation input rows is simply the number of rows of
+ * the non-grouped relation, which should have been estimated by now.
+ */
+ agg_info->input_rows = rel_plain->rows;
+
+ /*
+ * The number of output rows is supposedly different (lower) due to
+ * grouping.
+ */
+ rel_grouped->rows = estimate_num_groups(root, agg_info->group_exprs,
+ agg_info->input_rows, NULL);
+
+ *agg_info_p = agg_info;
+ return rel_grouped;
+}
+
/*
* find_base_rel
* Find a base or other relation entry, which must already exist.
@@ -484,9 +598,13 @@ find_rel_info(RelInfoList *list, Relids relids)
void *item = lfirst(l);
Relids item_relids = NULL;
- Assert(IsA(item, RelOptInfo));
+ Assert(IsA(item, RelOptInfo) || IsA(item, RelAggInfo));
+
+ if (IsA(item, RelOptInfo))
+ item_relids = ((RelOptInfo *) item)->relids;
+ else if (IsA(item, RelAggInfo))
+ item_relids = ((RelAggInfo *) item)->relids;
- item_relids = ((RelOptInfo *) item)->relids;
if (bms_equal(item_relids, relids))
return item;
}
@@ -514,7 +632,7 @@ find_join_rel(PlannerInfo *root, Relids relids)
static void
add_rel_info(RelInfoList *list, void *data)
{
- Assert(IsA(data, RelOptInfo));
+ Assert(IsA(data, RelOptInfo) || IsA(data, RelAggInfo));
/* GEQO requires us to append the new joinrel to the end of the list! */
list->items = lappend(list->items, data);
@@ -526,7 +644,11 @@ add_rel_info(RelInfoList *list, void *data)
RelInfoEntry *hentry;
bool found;
- relids = ((RelOptInfo *) data)->relids;
+ if (IsA(data, RelOptInfo))
+ relids = ((RelOptInfo *) data)->relids;
+ else if (IsA(data, RelAggInfo))
+ relids = ((RelAggInfo *) data)->relids;
+
hentry = (RelInfoEntry *) hash_search(list->hash,
&relids,
HASH_ENTER,
@@ -547,6 +669,63 @@ add_join_rel(PlannerInfo *root, RelOptInfo *joinrel)
add_rel_info(root->join_rel_list, joinrel);
}
+/*
+ * add_grouped_rel
+ * Add grouped base or join relation to the list of grouped relations in
+ * the given PlannerInfo. Also add the corresponding RelAggInfo to
+ * agg_info_list.
+ */
+void
+add_grouped_rel(PlannerInfo *root, RelOptInfo *rel, RelAggInfo *agg_info)
+{
+ add_rel_info(&root->upper_rels[UPPERREL_PARTIAL_GROUP_AGG], rel);
+ add_rel_info(root->agg_info_list, agg_info);
+}
+
+/*
+ * find_grouped_rel
+ * Returns grouped relation entry (base or join relation) corresponding to
+ * 'relids' or NULL if none exists.
+ *
+ * If agg_info_p is a valid pointer, then pointer to RelAggInfo that
+ * corresponds to the relation returned is assigned to *agg_info_p.
+ *
+ * The call fetch_upper_rel(root, UPPERREL_PARTIAL_GROUP_AGG, ...) should
+ * return the same relation if it exists, however the behavior is different if
+ * the relation is not there. find_grouped_rel() should be used in
+ * query_planner() and subroutines.
+ */
+RelOptInfo *
+find_grouped_rel(PlannerInfo *root, Relids relids, RelAggInfo **agg_info_p)
+{
+ RelOptInfo *rel;
+
+ rel = (RelOptInfo *) find_rel_info(&root->upper_rels[UPPERREL_PARTIAL_GROUP_AGG],
+ relids);
+ if (rel == NULL)
+ {
+ if (agg_info_p)
+ *agg_info_p = NULL;
+
+ return NULL;
+ }
+
+ /* Is caller interested in RelAggInfo? */
+ if (agg_info_p)
+ {
+ RelAggInfo *agg_info;
+
+ agg_info = (RelAggInfo *) find_rel_info(root->agg_info_list, relids);
+
+ /* The relation exists, so the agg_info should be there too. */
+ Assert(agg_info != NULL);
+
+ *agg_info_p = agg_info;
+ }
+
+ return rel;
+}
+
/*
* set_foreign_rel_properties
* Set up foreign-join fields if outer and inner relation are foreign
@@ -609,6 +788,7 @@ set_foreign_rel_properties(RelOptInfo *joinrel, RelOptInfo *outer_rel,
* 'restrictlist_ptr': result variable. If not NULL, *restrictlist_ptr
* receives the list of RestrictInfo nodes that apply to this
* particular pair of joinable relations.
+ * 'agg_info' indicates that grouped join relation should be created.
*
* restrictlist_ptr makes the routine's API a little grotty, but it saves
* duplicated calculation of the restrictlist...
@@ -619,10 +799,12 @@ build_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr)
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info)
{
RelOptInfo *joinrel;
List *restrictlist;
+ bool grouped = agg_info != NULL;
/* This function should be used only for join between parents. */
Assert(!IS_OTHER_REL(outer_rel) && !IS_OTHER_REL(inner_rel));
@@ -630,7 +812,8 @@ build_join_rel(PlannerInfo *root,
/*
* See if we already have a joinrel for this set of base rels.
*/
- joinrel = find_join_rel(root, joinrelids);
+ joinrel = !grouped ? find_join_rel(root, joinrelids) :
+ find_grouped_rel(root, joinrelids, NULL);
if (joinrel)
{
@@ -725,9 +908,21 @@ build_join_rel(PlannerInfo *root,
* and inner rels we first try to build it from. But the contents should
* be the same regardless.
*/
- build_joinrel_tlist(root, joinrel, outer_rel);
- build_joinrel_tlist(root, joinrel, inner_rel);
- add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ if (!grouped)
+ {
+ joinrel->reltarget = create_empty_pathtarget();
+ build_joinrel_tlist(root, joinrel, outer_rel);
+ build_joinrel_tlist(root, joinrel, inner_rel);
+ add_placeholders_to_joinrel(root, joinrel, outer_rel, inner_rel);
+ }
+ else
+ {
+ /*
+ * The target for grouped join should already have its cost and width
+ * computed, see create_rel_agg_info().
+ */
+ joinrel->reltarget = agg_info->target;
+ }
/*
* add_placeholders_to_joinrel also took care of adding the ph_lateral
@@ -759,49 +954,75 @@ build_join_rel(PlannerInfo *root,
joinrel->has_eclass_joins = has_relevant_eclass_joinclause(root, joinrel);
/* Store the partition information. */
- build_joinrel_partition_info(joinrel, outer_rel, inner_rel, restrictlist,
- sjinfo->jointype);
+ if (!grouped)
+ build_joinrel_partition_info(joinrel, outer_rel, inner_rel,
+ restrictlist, sjinfo->jointype);
- /*
- * Set estimates of the joinrel's size.
- */
- set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
- sjinfo, restrictlist);
+ if (!grouped)
+ {
+ /*
+ * Set estimates of the joinrel's size.
+ */
+ set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
+ sjinfo, restrictlist);
- /*
- * Set the consider_parallel flag if this joinrel could potentially be
- * scanned within a parallel worker. If this flag is false for either
- * inner_rel or outer_rel, then it must be false for the joinrel also.
- * Even if both are true, there might be parallel-restricted expressions
- * in the targetlist or quals.
- *
- * Note that if there are more than two rels in this relation, they could
- * be divided between inner_rel and outer_rel in any arbitrary way. We
- * assume this doesn't matter, because we should hit all the same baserels
- * and joinclauses while building up to this joinrel no matter which we
- * take; therefore, we should make the same decision here however we get
- * here.
- */
- if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
- is_parallel_safe(root, (Node *) restrictlist) &&
- is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
- joinrel->consider_parallel = true;
+ /*
+ * Set the consider_parallel flag if this joinrel could potentially be
+ * scanned within a parallel worker. If this flag is false for either
+ * inner_rel or outer_rel, then it must be false for the joinrel also.
+ * Even if both are true, there might be parallel-restricted
+ * expressions in the targetlist or quals.
+ *
+ * Note that if there are more than two rels in this relation, they
+ * could be divided between inner_rel and outer_rel in any arbitrary
+ * way. We assume this doesn't matter, because we should hit all the
+ * same baserels and joinclauses while building up to this joinrel no
+ * matter which we take; therefore, we should make the same decision
+ * here however we get here.
+ */
+ if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
+ is_parallel_safe(root, (Node *) restrictlist) &&
+ is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
+ joinrel->consider_parallel = true;
+ }
+ else
+ {
+ /*
+ * Grouping essentially changes the number of rows.
+ *
+ * XXX We do not distinguish whether two plain rels are joined and the
+ * result is aggregated, or the aggregation has been already applied
+ * to one of the input rels. Is this worth extra effort, e.g.
+ * maintaining a separate RelOptInfo for each case (one difficulty
+ * that would introduce is construction of AppendPath)?
+ */
+ joinrel->rows = estimate_num_groups(root, agg_info->group_exprs,
+ agg_info->input_rows, NULL);
+ }
/* Add the joinrel to the PlannerInfo. */
- add_join_rel(root, joinrel);
+ if (!grouped)
+ add_join_rel(root, joinrel);
+ else
+ add_grouped_rel(root, joinrel, agg_info);
/*
- * Also, if dynamic-programming join search is active, add the new joinrel
- * to the appropriate sublist. Note: you might think the Assert on number
- * of members should be for equality, but some of the level 1 rels might
- * have been joinrels already, so we can only assert <=.
+ * Also, if dynamic-programming join search is active, add the new
+ * joinrelset to the appropriate sublist. Note: you might think the
+ * Assert on number of members should be for equality, but some of the
+ * level 1 rels might have been joinrels already, so we can only assert
+ * <=.
+ *
+ * Do noting for grouped relation as it's stored aside from
+ * join_rel_level.
*/
- if (root->join_rel_level)
+ if (root->join_rel_level && !grouped)
{
Assert(root->join_cur_level > 0);
- Assert(root->join_cur_level <= bms_num_members(joinrel->relids));
+ Assert(root->join_cur_level <= bms_num_members(joinrelids));
root->join_rel_level[root->join_cur_level] =
- lappend(root->join_rel_level[root->join_cur_level], joinrel);
+ lappend(root->join_rel_level[root->join_cur_level],
+ joinrel);
}
return joinrel;
@@ -2059,3 +2280,621 @@ build_child_join_reltarget(PlannerInfo *root,
childrel->reltarget->cost.per_tuple = parentrel->reltarget->cost.per_tuple;
childrel->reltarget->width = parentrel->reltarget->width;
}
+
+/*
+ * Check if the relation can produce grouped paths and return the information
+ * it'll need for it. The passed relation is the non-grouped one which has the
+ * reltarget already constructed.
+ */
+RelAggInfo *
+create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel)
+{
+ List *gvis;
+ List *aggregates = NIL;
+ bool found_other_rel_agg;
+ ListCell *lc;
+ RelAggInfo *result;
+ PathTarget *agg_input;
+ PathTarget *target = NULL;
+ List *grp_exprs_extra = NIL;
+ List *group_clauses_final;
+ int i;
+
+ /*
+ * The function shouldn't have been called if there's no opportunity for
+ * aggregation push-down.
+ */
+ Assert(root->grouped_var_list != NIL);
+
+ /*
+ * The current implementation of aggregation push-down cannot handle
+ * PlaceHolderVar (PHV).
+ *
+ * If we knew that the PHV should be evaluated in this target (and of
+ * course, if its expression matched some Aggref argument), we'd just let
+ * init_grouping_targets add that Aggref. On the other hand, if we knew
+ * that the PHV is evaluated below the current rel, we could ignore it
+ * because the referencing Aggref would take care of propagation of the
+ * value to upper joins.
+ *
+ * The problem is that the same PHV can be evaluated in the target of the
+ * current rel or in that of lower rel --- depending on the input paths.
+ * For example, consider rel->relids = {A, B, C} and if ph_eval_at = {B,
+ * C}. Path "A JOIN (B JOIN C)" implies that the PHV is evaluated by the
+ * "(B JOIN C)", while path "(A JOIN B) JOIN C" evaluates the PHV itself.
+ */
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Expr *expr = lfirst(lc);
+
+ if (IsA(expr, PlaceHolderVar))
+ return NULL;
+ }
+
+ if (IS_SIMPLE_REL(rel))
+ {
+ RangeTblEntry *rte = root->simple_rte_array[rel->relid];;
+
+ /*
+ * rtekind != RTE_RELATION case is not supported yet.
+ */
+ if (rte->rtekind != RTE_RELATION)
+ return NULL;
+ }
+
+ /* Caller should only pass base relations or joins. */
+ Assert(rel->reloptkind == RELOPT_BASEREL ||
+ rel->reloptkind == RELOPT_JOINREL);
+
+ /*
+ * If any outer join can set the attribute value to NULL, the Agg plan
+ * would receive different input at the base rel level.
+ *
+ * XXX For RELOPT_JOINREL, do not return if all the joins that can set any
+ * entry of the grouped target (do we need to postpone this check until
+ * the grouped target is available, and init_grouping_targets take care?)
+ * of this rel to NULL are provably below rel. (It's ok if rel is one of
+ * these joins.)
+ */
+ if (bms_overlap(rel->relids, root->nullable_baserels))
+ return NULL;
+
+ /*
+ * Use equivalence classes to generate additional grouping expressions for
+ * the current rel. Without these we might not be able to apply
+ * aggregation to the relation result set.
+ *
+ * It's important that create_grouping_expr_grouped_var_infos has
+ * processed the explicit grouping columns by now. If the grouping clause
+ * contains multiple expressions belonging to the same EC, the original
+ * (i.e. not derived) one should be preferred when we build grouping
+ * target for a relation. Otherwise we have a problem when trying to match
+ * target entries to grouping clauses during plan creation, see
+ * get_grouping_expression().
+ */
+ gvis = list_copy(root->grouped_var_list);
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+ int relid = -1;
+
+ /* Only interested in grouping expressions. */
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ while ((relid = bms_next_member(rel->relids, relid)) >= 0)
+ {
+ GroupedVarInfo *gvi_trans;
+
+ gvi_trans = translate_expression_to_rel(root, gvi, relid);
+ if (gvi_trans != NULL)
+ gvis = lappend(gvis, gvi_trans);
+ }
+ }
+
+ /*
+ * Check if some aggregates or grouping expressions can be evaluated in
+ * this relation's target, and collect all vars referenced by these
+ * aggregates / grouping expressions;
+ */
+ found_other_rel_agg = false;
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ /*
+ * The subset includes gv_eval_at uninitialized, which includes
+ * Aggref.aggstar.
+ */
+ if (bms_is_subset(gvi->gv_eval_at, rel->relids))
+ {
+ /*
+ * init_grouping_targets will handle plain Var grouping
+ * expressions because it needs to look them up in
+ * grouped_var_list anyway.
+ */
+ if (IsA(gvi->gvexpr, Var))
+ continue;
+
+ /*
+ * Currently, GroupedVarInfo only handles Vars and Aggrefs.
+ */
+ Assert(IsA(gvi->gvexpr, Aggref));
+
+ gvi->agg_partial = (Aggref *) copyObject(gvi->gvexpr);
+ mark_partial_aggref(gvi->agg_partial, AGGSPLIT_INITIAL_SERIAL);
+
+ /*
+ * Accept the aggregate.
+ */
+ aggregates = lappend(aggregates, gvi);
+ }
+ else if (IsA(gvi->gvexpr, Aggref))
+ {
+ /*
+ * Remember that there is at least one aggregate expression that
+ * needs something else than this rel.
+ */
+ found_other_rel_agg = true;
+
+ /*
+ * This condition effectively terminates creation of the
+ * RelAggInfo, so there's no reason to check the next
+ * GroupedVarInfo.
+ */
+ break;
+ }
+ }
+
+ /*
+ * Grouping makes little sense w/o aggregate function and w/o grouping
+ * expressions.
+ */
+ if (aggregates == NIL)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Give up if some other aggregate(s) need relations other than the
+ * current one.
+ *
+ * If the aggregate needs the current rel plus anything else, then the
+ * problem is that grouping of the current relation could make some input
+ * variables unavailable for the "higher aggregate", and it'd also
+ * decrease the number of input rows the "higher aggregate" receives.
+ *
+ * If the aggregate does not even need the current rel, then neither the
+ * current rel nor anything else should be grouped because we do not
+ * support join of two grouped relations.
+ */
+ if (found_other_rel_agg)
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ /*
+ * Create target for grouped paths as well as one for the input paths of
+ * the aggregation paths.
+ */
+ target = create_empty_pathtarget();
+ agg_input = create_empty_pathtarget();
+
+ /*
+ * Cannot suitable targets for the aggregation push-down be derived?
+ */
+ if (!init_grouping_targets(root, rel, target, agg_input, gvis,
+ &grp_exprs_extra))
+ {
+ list_free(gvis);
+ return NULL;
+ }
+
+ list_free(gvis);
+
+ /*
+ * Aggregation push-down makes no sense w/o grouping expressions.
+ */
+ if ((list_length(target->exprs) + list_length(grp_exprs_extra)) == 0)
+ return NULL;
+
+ group_clauses_final = root->parse->groupClause;
+
+ /*
+ * If the aggregation target should have extra grouping expressions (in
+ * order to emit input vars for join conditions), add them now. This step
+ * includes assignment of tleSortGroupRef's which we can generate now.
+ */
+ if (list_length(grp_exprs_extra) > 0)
+ {
+ Index sortgroupref;
+
+ /*
+ * We'll have to add some clauses, but query group clause must be
+ * preserved.
+ */
+ group_clauses_final = list_copy(group_clauses_final);
+
+ /*
+ * Always start at root->max_sortgroupref. The extra grouping
+ * expressions aren't used during the final aggregation, so the
+ * sortgroupref values don't need to be unique across the query. Thus
+ * we don't have to increase root->max_sortgroupref, which makes
+ * recognition of the extra grouping expressions pretty easy.
+ */
+ sortgroupref = root->max_sortgroupref;
+
+ /*
+ * Generate the SortGroupClause's and add the expressions to the
+ * target.
+ */
+ foreach(lc, grp_exprs_extra)
+ {
+ Var *var = lfirst_node(Var, lc);
+ SortGroupClause *cl = makeNode(SortGroupClause);
+
+ /*
+ * Initialize the SortGroupClause.
+ *
+ * As the final aggregation will not use this grouping expression,
+ * we don't care whether sortop is < or >. The value of
+ * nulls_first should not matter for the same reason.
+ */
+ cl->tleSortGroupRef = ++sortgroupref;
+ get_sort_group_operators(var->vartype,
+ false, true, false,
+ &cl->sortop, &cl->eqop, NULL,
+ &cl->hashable);
+ group_clauses_final = lappend(group_clauses_final, cl);
+ add_column_to_pathtarget(target, (Expr *) var,
+ cl->tleSortGroupRef);
+
+ /*
+ * The aggregation input target must emit this var too.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) var,
+ cl->tleSortGroupRef);
+ }
+ }
+
+ /*
+ * Add aggregates to the grouping target.
+ */
+ foreach(lc, aggregates)
+ {
+ GroupedVarInfo *gvi;
+
+ gvi = lfirst_node(GroupedVarInfo, lc);
+ add_column_to_pathtarget(target, (Expr *) gvi->agg_partial,
+ gvi->sortgroupref);
+ }
+
+ /*
+ * Build a list of grouping expressions and a list of the corresponding
+ * SortGroupClauses.
+ */
+ i = 0;
+ result = makeNode(RelAggInfo);
+ foreach(lc, target->exprs)
+ {
+ Index sortgroupref = 0;
+ SortGroupClause *cl;
+ Expr *texpr;
+
+ texpr = (Expr *) lfirst(lc);
+
+ if (IsA(texpr, Aggref))
+ {
+ /*
+ * Once we see Aggref, no grouping expressions should follow.
+ */
+ break;
+ }
+
+ /*
+ * Find the clause by sortgroupref.
+ */
+ sortgroupref = target->sortgrouprefs[i++];
+
+ /*
+ * Besides being an aggregate, the target expression should have no
+ * other reason then being a column of a relation functionally
+ * dependent on the GROUP BY clause. So it's not actually a grouping
+ * column.
+ */
+ if (sortgroupref == 0)
+ continue;
+
+ /*
+ * group_clause_final contains the "local" clauses, so this search
+ * should succeed.
+ */
+ cl = get_sortgroupref_clause(sortgroupref, group_clauses_final);
+
+ result->group_clauses = list_append_unique(result->group_clauses,
+ cl);
+
+ /*
+ * Add only unique clauses because of joins (both sides of a join can
+ * point at the same grouping clause). XXX Is it worth adding a bool
+ * argument indicating that we're dealing with join right now?
+ */
+ result->group_exprs = list_append_unique(result->group_exprs,
+ texpr);
+ }
+
+ /*
+ * Since neither target nor agg_input is supposed to be identical to the
+ * source reltarget, compute the width and cost again.
+ *
+ * target does not yet contain aggregates, but these will be accounted by
+ * AggPath.
+ */
+ set_pathtarget_cost_width(root, target);
+ set_pathtarget_cost_width(root, agg_input);
+
+ result->relids = bms_copy(rel->relids);
+ result->target = target;
+ result->agg_input = agg_input;
+
+ /* Finally collect the aggregates. */
+ while (lc != NULL)
+ {
+ Aggref *aggref = lfirst_node(Aggref, lc);
+
+ /*
+ * Partial aggregation is what the grouped paths should do.
+ */
+ result->agg_exprs = lappend(result->agg_exprs, aggref);
+ lc = lnext(target->exprs, lc);
+ }
+
+ /* The "input_rows" field should be set by caller. */
+ return result;
+}
+
+/*
+ * Initialize target for grouped paths (target) as well as a target for paths
+ * that generate input for aggregation (agg_input).
+ *
+ * group_exprs_extra_p receives a list of Var nodes for which we need to
+ * construct SortGroupClause. Those vars will then be used as additional
+ * grouping expressions, for the sake of join clauses.
+ *
+ * gvis a list of GroupedVarInfo's possibly useful for rel.
+ *
+ * Return true iff the targets could be initialized.
+ */
+static bool
+init_grouping_targets(PlannerInfo *root, RelOptInfo *rel,
+ PathTarget *target, PathTarget *agg_input,
+ List *gvis, List **group_exprs_extra_p)
+{
+ ListCell *lc;
+ List *possibly_dependent = NIL;
+ Var *tvar;
+
+ foreach(lc, rel->reltarget->exprs)
+ {
+ Index sortgroupref;
+
+ /*
+ * Given that PlaceHolderVar currently prevents us from doing
+ * aggregation push-down, the source target cannot contain anything
+ * more complex than a Var.
+ */
+ tvar = lfirst_node(Var, lc);
+
+ sortgroupref = get_expression_sortgroupref((Expr *) tvar, gvis);
+ if (sortgroupref > 0)
+ {
+ /*
+ * If the target expression can be used as the grouping key, we
+ * don't have to worry whether it can be emitted by the AggPath
+ * pushed down to relation / join.
+ */
+ add_column_to_pathtarget(target, (Expr *) tvar, sortgroupref);
+
+ /*
+ * As for agg_input, add the original expression but set
+ * sortgroupref in addition.
+ */
+ add_column_to_pathtarget(agg_input, (Expr *) tvar, sortgroupref);
+ }
+ else
+ {
+ /*
+ * Another reason we might need this variable is that some
+ * aggregate pushed down to this relation references it. In such a
+ * case, add that var to agg_input, but not to "target". However,
+ * if the aggregate is not the only reason for the var to be in
+ * the target, some more checks need to be performed below.
+ */
+ if (is_var_in_aggref_only(root, tvar))
+ add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
+ else if (is_var_needed_by_join(root, tvar, rel))
+ {
+ /*
+ * The variable is needed for a join, however it's neither in
+ * the GROUP BY clause nor can it be derived from it using EC.
+ * (Otherwise it would have to be added to the targets above.)
+ * We need to construct special SortGroupClause for that
+ * variable.
+ *
+ * Note that its tleSortGroupRef needs to be unique within
+ * agg_input, so we need to postpone creation of the
+ * SortGroupClause's until we're done with the iteration of
+ * rel->reltarget->exprs. Also it makes sense for the caller
+ * to do some more check before it starts to create those
+ * SortGroupClause's.
+ */
+ *group_exprs_extra_p = lappend(*group_exprs_extra_p, tvar);
+ }
+ else
+ {
+ /*
+ * The Var can be functionally dependent on another expression
+ * of the target, but we cannot check until the other
+ * expressions are in the target.
+ */
+ possibly_dependent = lappend(possibly_dependent, tvar);
+ }
+ }
+ }
+
+ /*
+ * Now we can check whether the expression is functionally dependent on
+ * another one.
+ */
+ foreach(lc, possibly_dependent)
+ {
+ List *deps = NIL;
+ RangeTblEntry *rte;
+
+ tvar = lfirst_node(Var, lc);
+ rte = root->simple_rte_array[tvar->varno];
+
+ /*
+ * Check if the Var can be in the grouping key even though it's not
+ * mentioned by the GROUP BY clause (and could not be derived using
+ * ECs).
+ */
+ if (check_functional_grouping(rte->relid, tvar->varno,
+ tvar->varlevelsup,
+ target->exprs, &deps))
+ {
+ /*
+ * The var shouldn't be actually used for grouping key evaluation
+ * (instead, the one this depends on will be), so sortgroupref
+ * should not be important.
+ */
+ add_new_column_to_pathtarget(target, (Expr *) tvar);
+ add_new_column_to_pathtarget(agg_input, (Expr *) tvar);
+ }
+ else
+ {
+ /*
+ * As long as the query is semantically correct, arriving here
+ * means that the var is referenced by a generic grouping
+ * expression but not referenced by any join.
+ *
+ * If the aggregate push-down will support generic grouping
+ * expression sin the future, create_rel_agg_info() will have to
+ * add this variable to "agg_input" target and also add the whole
+ * generic expression to "target".
+ */
+ return false;
+ }
+ }
+
+ return true;
+}
+
+/*
+ * Check whether given variable appears in Aggref(s) which we consider usable
+ * at relation / join level, and only in the Aggref(s).
+ */
+static bool
+is_var_in_aggref_only(PlannerInfo *root, Var *var)
+{
+ ListCell *lc;
+ bool found = false;
+
+ foreach(lc, root->grouped_var_list)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+ ListCell *lc2;
+ List *vars;
+
+ if (!IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (!bms_is_member(var->varno, gvi->gv_eval_at))
+ continue;
+
+ /*
+ * XXX Consider some sort of caching.
+ */
+ vars = pull_var_clause((Node *) gvi->gvexpr, PVC_RECURSE_AGGREGATES);
+ foreach(lc2, vars)
+ {
+ Var *v = lfirst_node(Var, lc2);
+
+ if (equal(v, var))
+ {
+ found = true;
+ break;
+ }
+
+ }
+ list_free(vars);
+
+ if (found)
+ break;
+ }
+
+ /* No aggregate references the Var? */
+ if (!found)
+ return false;
+
+ /* Does the Var appear in the target outside aggregates? */
+ found = false;
+ foreach(lc, root->processed_tlist)
+ {
+ TargetEntry *te = lfirst_node(TargetEntry, lc);
+
+ if (IsA(te->expr, Aggref))
+ continue;
+
+ if (equal(te->expr, var))
+ return false;
+
+ }
+
+ /* The Var is in aggregate(s) and only there. */
+ return true;
+}
+
+/*
+ * Check if given variable is needed by joins above the current rel?
+ *
+ * Consider pushing the aggregate avg(b.y) down to relation "b" for the
+ * following query:
+ *
+ * SELECT a.i, avg(b.y)
+ * FROM a JOIN b ON b.j = a.i
+ * GROUP BY a.i;
+ *
+ * If we aggregate the "b" relation alone, the column "b.j" needs to be used
+ * as the grouping key because otherwise it cannot find its way to the input
+ * of the join expression.
+ */
+static bool
+is_var_needed_by_join(PlannerInfo *root, Var *var, RelOptInfo *rel)
+{
+ Relids relids_no_top;
+ int ndx;
+ RelOptInfo *baserel;
+
+ /*
+ * The relids we're not interested in do include 0, which is the top-level
+ * targetlist. The only reason for relids to contain 0 should be that
+ * arg_var is referenced either by aggregate or by grouping expression,
+ * but right now we're interested in the *other* reasons. (As soon
+ * aggregation is pushed down, the aggregates in the query targetlist no
+ * longer need direct reference to arg_var anyway.)
+ */
+
+ relids_no_top = bms_copy(rel->relids);
+ bms_add_member(relids_no_top, 0);
+
+ baserel = find_base_rel(root, var->varno);
+ ndx = var->varattno - baserel->min_attr;
+ if (bms_nonempty_difference(baserel->attr_needed[ndx],
+ relids_no_top))
+ return true;
+
+ return false;
+}
diff --git a/src/backend/optimizer/util/tlist.c b/src/backend/optimizer/util/tlist.c
index 02a3c6b165..5011afc033 100644
--- a/src/backend/optimizer/util/tlist.c
+++ b/src/backend/optimizer/util/tlist.c
@@ -825,6 +825,37 @@ apply_pathtarget_labeling_to_tlist(List *tlist, PathTarget *target)
}
}
+/*
+ * Return sortgroupref if expr can be used as the grouping expression in an
+ * AggPath at relation or join level, or 0 if it can't.
+ *
+ * gvis a list of a list of GroupedVarInfo's available for the query,
+ * including those derived using equivalence classes.
+ */
+Index
+get_expression_sortgroupref(Expr *expr, List *gvis)
+{
+ ListCell *lc;
+
+ foreach(lc, gvis)
+ {
+ GroupedVarInfo *gvi = lfirst_node(GroupedVarInfo, lc);
+
+ if (IsA(gvi->gvexpr, Aggref))
+ continue;
+
+ if (equal(gvi->gvexpr, expr))
+ {
+ Assert(gvi->sortgroupref > 0);
+
+ return gvi->sortgroupref;
+ }
+ }
+
+ /* The expression cannot be used as grouping key. */
+ return 0;
+}
+
/*
* split_pathtarget_at_srfs
* Split given PathTarget into multiple levels to position SRFs safely
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 75fc6f11d6..a06dd52a50 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -1081,6 +1081,16 @@ static struct config_bool ConfigureNamesBool[] =
false,
NULL, NULL, NULL
},
+ {
+ {"enable_agg_pushdown", PGC_USERSET, QUERY_TUNING_METHOD,
+ gettext_noop("Enables aggregate push-down."),
+ NULL,
+ GUC_EXPLAIN
+ },
+ &enable_agg_pushdown,
+ false,
+ NULL, NULL, NULL
+ },
{
{"enable_parallel_append", PGC_USERSET, QUERY_TUNING_METHOD,
gettext_noop("Enables the planner's use of parallel append plans."),
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index be5ab273f0..62134356a0 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -226,6 +226,7 @@ typedef enum NodeTag
T_IndexOptInfo,
T_ForeignKeyOptInfo,
T_ParamPathInfo,
+ T_RelAggInfo,
T_Path,
T_IndexPath,
T_BitmapHeapPath,
@@ -271,6 +272,7 @@ typedef enum NodeTag
T_SpecialJoinInfo,
T_AppendRelInfo,
T_PlaceHolderInfo,
+ T_GroupedVarInfo,
T_MinMaxAggInfo,
T_PlannerParamItem,
T_RollupData,
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 44374de796..98be8b7de2 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -304,6 +304,8 @@ struct PlannerInfo
List *placeholder_list; /* list of PlaceHolderInfos */
+ List *grouped_var_list; /* List of GroupedVarInfos. */
+
List *fkey_list; /* list of ForeignKeyOptInfos */
List *query_pathkeys; /* desired pathkeys for query_planner() */
@@ -321,6 +323,12 @@ struct PlannerInfo
/* Use fetch_upper_rel() to get any particular upper rel */
RelInfoList upper_rels[UPPERREL_FINAL + 1]; /* upper-rel RelOptInfos */
+ /*
+ * One instance of RelAggInfo per item of the
+ * upper_rels[UPPERREL_PARTIAL_GROUP_AGG] list.
+ */
+ struct RelInfoList *agg_info_list; /* list of grouped relation
+ * RelAggInfos */
/* Result tlists chosen by grouping_planner for upper-stage processing */
struct PathTarget *upper_targets[UPPERREL_FINAL + 1];
@@ -336,6 +344,12 @@ struct PlannerInfo
*/
List *processed_tlist;
+ /*
+ * The maximum ressortgroupref among target entries in processed_list.
+ * Useful when adding extra grouping expressions for partial aggregation.
+ */
+ int max_sortgroupref;
+
/* Fields filled during create_plan() for use in setrefs.c */
AttrNumber *grouping_map; /* for GroupingFunc fixup */
List *minmax_aggs; /* List of MinMaxAggInfos */
@@ -787,6 +801,60 @@ typedef struct RelOptInfo
((rel)->part_scheme && (rel)->boundinfo && (rel)->nparts > 0 && \
(rel)->part_rels && (rel)->partexprs && (rel)->nullable_partexprs)
+/*
+ * RelAggInfo
+ * Information needed to create grouped paths for base rels and joins.
+ *
+ * "relids" is the set of base-relation identifiers, just like with
+ * RelOptInfo.
+ *
+ * "target" will be used as pathtarget if partial aggregation is applied to
+ * base relation or join. The same target will also --- if the relation is a
+ * join --- be used to joinin grouped path to a non-grouped one. This target
+ * can contain plain-Var grouping expressions and Aggref nodes.
+ *
+ * Note: There's a convention that Aggref expressions are supposed to follow
+ * the other expressions of the target. Iterations of ->exprs may rely on this
+ * arrangement.
+ *
+ * "agg_input" contains Vars used either as grouping expressions or aggregate
+ * arguments. Paths providing the aggregation plan with input data should use
+ * this target. The only difference from reltarget of the non-grouped relation
+ * is that some items can have sortgroupref initialized.
+ *
+ * "input_rows" is the estimated number of input rows for AggPath. It's
+ * actually just a workspace for users of the structure, i.e. not initialized
+ * when instance of the structure is created.
+ *
+ * "group_clauses" and "group_exprs" are lists of SortGroupClause and the
+ * corresponding grouping expressions respectively.
+ *
+ * "agg_exprs" is a list of Aggref nodes for the aggregation of the relation's
+ * paths.
+ *
+ * "rel_grouped" is the relation containing the partially aggregated paths.
+ */
+typedef struct RelAggInfo
+{
+ NodeTag type;
+
+ Relids relids; /* Base rels contained in this grouped rel. */
+
+ struct PathTarget *target; /* Target for grouped paths. */
+
+ struct PathTarget *agg_input; /* pathtarget of paths that generate input
+ * for aggregation paths. */
+
+ double input_rows;
+
+ List *group_clauses;
+ List *group_exprs;
+
+ List *agg_exprs; /* Aggref expressions. */
+
+ RelOptInfo *rel_grouped; /* Grouped relation. */
+} RelAggInfo;
+
/*
* IndexOptInfo
* Per-index information for planning/optimization
@@ -2323,6 +2391,27 @@ typedef struct PlaceHolderInfo
int32 ph_width; /* estimated attribute width */
} PlaceHolderInfo;
+/*
+ * GroupedVarInfo exists for each expression that can be used as an aggregate
+ * or grouping expression evaluated below a join.
+ *
+ * TODO Rename, perhaps to GroupedTargetEntry? (Also rename the variables of
+ * this type.)
+ */
+typedef struct GroupedVarInfo
+{
+ NodeTag type;
+
+ Expr *gvexpr; /* the represented expression. */
+ Aggref *agg_partial; /* if gvexpr is aggregate, agg_partial is the
+ * corresponding partial aggregate */
+ Index sortgroupref; /* If gvexpr is a grouping expression, this is
+ * the tleSortGroupRef of the corresponding
+ * SortGroupClause. */
+ Relids gv_eval_at; /* lowest level we can evaluate the expression
+ * at or NULL if it can happen anywhere. */
+} GroupedVarInfo;
+
/*
* This struct describes one potentially index-optimizable MIN/MAX aggregate
* function. MinMaxAggPath contains a list of these, and if we accept that
diff --git a/src/include/optimizer/clauses.h b/src/include/optimizer/clauses.h
index b7456e3e59..0464531e25 100644
--- a/src/include/optimizer/clauses.h
+++ b/src/include/optimizer/clauses.h
@@ -55,4 +55,6 @@ extern void CommuteOpExpr(OpExpr *clause);
extern Query *inline_set_returning_function(PlannerInfo *root,
RangeTblEntry *rte);
+extern GroupedVarInfo *translate_expression_to_rel(PlannerInfo *root,
+ GroupedVarInfo *gvi, Index relid);
#endif /* CLAUSES_H */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 715a24ad29..47afa7ed1a 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -216,6 +216,14 @@ extern AggPath *create_agg_path(PlannerInfo *root,
List *qual,
const AggClauseCosts *aggcosts,
double numGroups);
+extern AggPath *create_agg_sorted_path(PlannerInfo *root,
+ RelOptInfo *rel,
+ Path *subpath,
+ RelAggInfo *agg_info);
+extern AggPath *create_agg_hashed_path(PlannerInfo *root,
+ RelOptInfo *rel,
+ Path *subpath,
+ RelAggInfo *agg_info);
extern GroupingSetsPath *create_groupingsets_path(PlannerInfo *root,
RelOptInfo *rel,
Path *subpath,
@@ -287,14 +295,21 @@ extern void setup_simple_rel_arrays(PlannerInfo *root);
extern void expand_planner_arrays(PlannerInfo *root, int add_size);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
+extern RelOptInfo *build_simple_grouped_rel(PlannerInfo *root, int relid,
+ RelAggInfo **agg_info_p);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
+extern void add_grouped_rel(PlannerInfo *root, RelOptInfo *rel,
+ RelAggInfo *agg_info);
+extern RelOptInfo *find_grouped_rel(PlannerInfo *root, Relids relids,
+ RelAggInfo **agg_info_p);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
RelOptInfo *inner_rel,
SpecialJoinInfo *sjinfo,
- List **restrictlist_ptr);
+ List **restrictlist_ptr,
+ RelAggInfo *agg_info);
extern Relids min_join_parameterization(PlannerInfo *root,
Relids joinrelids,
RelOptInfo *outer_rel,
@@ -320,5 +335,5 @@ extern RelOptInfo *build_child_join_rel(PlannerInfo *root,
RelOptInfo *outer_rel, RelOptInfo *inner_rel,
RelOptInfo *parent_joinrel, List *restrictlist,
SpecialJoinInfo *sjinfo, JoinType jointype);
-
+extern RelAggInfo *create_rel_agg_info(PlannerInfo *root, RelOptInfo *rel);
#endif /* PATHNODE_H */
diff --git a/src/include/optimizer/paths.h b/src/include/optimizer/paths.h
index 10b6e81079..ee34442772 100644
--- a/src/include/optimizer/paths.h
+++ b/src/include/optimizer/paths.h
@@ -21,6 +21,7 @@
* allpaths.c
*/
extern PGDLLIMPORT bool enable_geqo;
+extern PGDLLIMPORT bool enable_agg_pushdown;
extern PGDLLIMPORT int geqo_threshold;
extern PGDLLIMPORT int min_parallel_table_scan_size;
extern PGDLLIMPORT int min_parallel_index_scan_size;
@@ -56,6 +57,11 @@ extern void generate_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
extern void generate_useful_gather_paths(PlannerInfo *root, RelOptInfo *rel,
bool override_rows);
+extern void generate_grouping_paths(PlannerInfo *root,
+ RelOptInfo *rel_grouped,
+ RelOptInfo *rel_plain,
+ RelAggInfo *agg_info);
+
extern int compute_parallel_worker(RelOptInfo *rel, double heap_pages,
double index_pages, int max_workers);
extern void create_partial_bitmap_paths(PlannerInfo *root, RelOptInfo *rel,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index f3cefe67b8..c2ab6b2115 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -72,6 +72,7 @@ extern void add_other_rels_to_query(PlannerInfo *root);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
Relids where_needed, bool create_new_ph);
+extern void setup_aggregate_pushdown(PlannerInfo *root);
extern void find_lateral_references(PlannerInfo *root);
extern void create_lateral_join_info(PlannerInfo *root);
extern List *deconstruct_jointree(PlannerInfo *root);
diff --git a/src/include/optimizer/tlist.h b/src/include/optimizer/tlist.h
index 1d4c7da545..e2547732d6 100644
--- a/src/include/optimizer/tlist.h
+++ b/src/include/optimizer/tlist.h
@@ -50,8 +50,10 @@ extern void split_pathtarget_at_srfs(PlannerInfo *root,
PathTarget *target, PathTarget *input_target,
List **targets, List **targets_contain_srfs);
+/* TODO Find the best location for this one. */
+extern Index get_expression_sortgroupref(Expr *expr, List *gvis);
+
/* Convenience macro to get a PathTarget with valid cost/width fields */
#define create_pathtarget(root, tlist) \
set_pathtarget_cost_width(root, make_pathtarget_from_tlist(tlist))
-
#endif /* TLIST_H */
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
new file mode 100644
index 0000000000..413abc147e
--- /dev/null
+++ b/src/test/regress/expected/agg_pushdown.out
@@ -0,0 +1,215 @@
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+CREATE INDEX ON agg_pushdown_child1(parent);
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+ANALYZE;
+SET enable_agg_pushdown TO on;
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+--------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(10 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(11 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (p.i = c1.parent)
+ -> Sort
+ Sort Key: p.i
+ -> Seq Scan on agg_pushdown_parent p
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(12 rows)
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(8 rows)
+
+SET enable_seqscan TO on;
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.x = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Seq Scan on agg_pushdown_child1 c1
+(11 rows)
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: ((k = c1.j) AND (parent = c1.parent))
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(13 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+----------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Hash Join
+ Hash Cond: (p.i = c1.parent)
+ -> Seq Scan on agg_pushdown_parent p
+ -> Hash
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Seq Scan on agg_pushdown_child1 c1
+ -> Hash
+ -> Seq Scan on agg_pushdown_child2 c2
+(15 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+---------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: ((c1.j = c2.k) AND (c1.parent = c2.parent))
+ -> Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(13 rows)
+
diff --git a/src/test/regress/expected/sysviews.out b/src/test/regress/expected/sysviews.out
index 01b7786f01..5c15e2847d 100644
--- a/src/test/regress/expected/sysviews.out
+++ b/src/test/regress/expected/sysviews.out
@@ -72,6 +72,7 @@ select count(*) >= 0 as ok from pg_prepared_xacts;
select name, setting from pg_settings where name like 'enable%';
name | setting
--------------------------------+---------
+ enable_agg_pushdown | off
enable_bitmapscan | on
enable_gathermerge | on
enable_hashagg | on
@@ -90,7 +91,7 @@ select name, setting from pg_settings where name like 'enable%';
enable_seqscan | on
enable_sort | on
enable_tidscan | on
-(18 rows)
+(19 rows)
-- Test that the pg_timezone_names and pg_timezone_abbrevs views are
-- more-or-less working. We can't test their contents in any great detail
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 979d926119..71a1c31431 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -146,6 +146,7 @@ test: stats_ext
test: collate.linux.utf8
test: select_parallel
test: write_parallel
+test: agg_pushdown
test: publication
test: subscription
test: select_views
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
new file mode 100644
index 0000000000..6bbbc7a8a1
--- /dev/null
+++ b/src/test/regress/sql/agg_pushdown.sql
@@ -0,0 +1,114 @@
+CREATE TABLE agg_pushdown_parent (
+ i int primary key,
+ x int);
+
+CREATE TABLE agg_pushdown_child1 (
+ j int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (j, parent));
+
+CREATE INDEX ON agg_pushdown_child1(parent);
+
+CREATE TABLE agg_pushdown_child2 (
+ k int,
+ parent int references agg_pushdown_parent,
+ v double precision,
+ PRIMARY KEY (k, parent));;
+
+INSERT INTO agg_pushdown_parent(i, x)
+SELECT n, n
+FROM generate_series(0, 7) AS s(n);
+
+INSERT INTO agg_pushdown_child1(j, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+INSERT INTO agg_pushdown_child2(k, parent, v)
+SELECT 128 * i + n, i, random()
+FROM generate_series(0, 127) AS s(n), agg_pushdown_parent;
+
+ANALYZE;
+
+SET enable_agg_pushdown TO on;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+-- Perform scan of a table, aggregate the result, join it to the other table
+-- and finalize the aggregation.
+--
+-- In addition, check that functionally dependent column "c.x" can be
+-- referenced by SELECT although GROUP BY references "p.i".
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+
+-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
+-- using AGG_SORTED strategy.
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_seqscan TO on;
+
+-- Join "c1" to "p.x" column, i.e. one that is not in the GROUP BY clause. The
+-- planner should still use "c1.parent" as grouping expression for partial
+-- aggregation, although it's not in the same equivalence class as the GROUP
+-- BY expression ("p.i"). The reason to use "c1.parent" for partial
+-- aggregation is that this is the only way for "c1" to provide the join
+-- expression with input data.
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.x GROUP BY p.i;
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
--
2.21.3
v17-0003-Use-also-partial-paths-as-the-input-for-grouped-path.patchtext/x-patch; charset=UTF-8; name*0=v17-0003-Use-also-partial-paths-as-the-input-for-grouped-path.pat; name*1=chDownload
From cb03357f4413484e2c8a8bd96474c610484e81e3 Mon Sep 17 00:00:00 2001
From: Laurenz Albe <laurenz.albe@cybertec.at>
Date: Fri, 3 Jul 2020 10:22:14 +0200
Subject: [PATCH 3/3] Use also partial paths as the input for grouped paths.
---
src/backend/optimizer/path/allpaths.c | 46 +++++-
src/backend/optimizer/util/relnode.c | 46 +++---
src/test/regress/expected/agg_pushdown.out | 157 +++++++++++++++++++++
src/test/regress/sql/agg_pushdown.sql | 66 +++++++++
4 files changed, 285 insertions(+), 30 deletions(-)
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index 84a918dc58..e0b5418de4 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -128,7 +128,7 @@ static void set_worktable_pathlist(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte);
static void add_grouped_path(PlannerInfo *root, RelOptInfo *rel,
Path *subpath, AggStrategy aggstrategy,
- RelAggInfo *agg_info);
+ RelAggInfo *agg_info, bool partial);
static RelOptInfo *make_rel_from_joinlist(PlannerInfo *root, List *joinlist);
static bool subquery_is_pushdown_safe(Query *subquery, Query *topquery,
pushdown_safety_info *safetyInfo);
@@ -3021,6 +3021,7 @@ generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
RelOptInfo *rel_plain, RelAggInfo *agg_info)
{
ListCell *lc;
+ Path *path;
if (IS_DUMMY_REL(rel_plain))
{
@@ -3030,7 +3031,7 @@ generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
foreach(lc, rel_plain->pathlist)
{
- Path *path = (Path *) lfirst(lc);
+ path = (Path *) lfirst(lc);
/*
* Since the path originates from the non-grouped relation which is
@@ -3044,7 +3045,8 @@ generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
* add_grouped_path() will check whether the path has suitable
* pathkeys.
*/
- add_grouped_path(root, rel_grouped, path, AGG_SORTED, agg_info);
+ add_grouped_path(root, rel_grouped, path, AGG_SORTED, agg_info,
+ false);
/*
* Repeated creation of hash table (for new parameter values) should
@@ -3052,12 +3054,38 @@ generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
* efficiency.
*/
if (path->param_info == NULL)
- add_grouped_path(root, rel_grouped, path, AGG_HASHED, agg_info);
+ add_grouped_path(root, rel_grouped, path, AGG_HASHED, agg_info,
+ false);
}
/* Could not generate any grouped paths? */
if (rel_grouped->pathlist == NIL)
+ {
mark_dummy_rel(rel_grouped);
+ return;
+ }
+
+ /*
+ * Almost the same for partial paths.
+ *
+ * The difference is that parameterized paths are never created, see
+ * add_partial_path() for explanation.
+ */
+ foreach(lc, rel_plain->partial_pathlist)
+ {
+ path = (Path *) lfirst(lc);
+
+ if (path->param_info != NULL)
+ continue;
+
+ path = (Path *) create_projection_path(root, rel_grouped, path,
+ agg_info->agg_input);
+
+ add_grouped_path(root, rel_grouped, path, AGG_SORTED, agg_info,
+ true);
+ add_grouped_path(root, rel_grouped, path, AGG_HASHED, agg_info,
+ true);
+ }
}
/*
@@ -3065,7 +3093,8 @@ generate_grouping_paths(PlannerInfo *root, RelOptInfo *rel_grouped,
*/
static void
add_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
- AggStrategy aggstrategy, RelAggInfo *agg_info)
+ AggStrategy aggstrategy, RelAggInfo *agg_info,
+ bool partial)
{
Path *agg_path;
@@ -3081,7 +3110,12 @@ add_grouped_path(PlannerInfo *root, RelOptInfo *rel, Path *subpath,
/* Add the grouped path to the list of grouped base paths. */
if (agg_path != NULL)
- add_path(rel, (Path *) agg_path);
+ {
+ if (!partial)
+ add_path(rel, (Path *) agg_path);
+ else
+ add_partial_path(rel, (Path *) agg_path);
+ }
}
/*
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 7460794b46..4164f4b0e4 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -958,33 +958,12 @@ build_join_rel(PlannerInfo *root,
build_joinrel_partition_info(joinrel, outer_rel, inner_rel,
restrictlist, sjinfo->jointype);
+ /*
+ * Set estimates of the joinrel's size.
+ */
if (!grouped)
- {
- /*
- * Set estimates of the joinrel's size.
- */
set_joinrel_size_estimates(root, joinrel, outer_rel, inner_rel,
sjinfo, restrictlist);
-
- /*
- * Set the consider_parallel flag if this joinrel could potentially be
- * scanned within a parallel worker. If this flag is false for either
- * inner_rel or outer_rel, then it must be false for the joinrel also.
- * Even if both are true, there might be parallel-restricted
- * expressions in the targetlist or quals.
- *
- * Note that if there are more than two rels in this relation, they
- * could be divided between inner_rel and outer_rel in any arbitrary
- * way. We assume this doesn't matter, because we should hit all the
- * same baserels and joinclauses while building up to this joinrel no
- * matter which we take; therefore, we should make the same decision
- * here however we get here.
- */
- if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
- is_parallel_safe(root, (Node *) restrictlist) &&
- is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
- joinrel->consider_parallel = true;
- }
else
{
/*
@@ -1000,6 +979,25 @@ build_join_rel(PlannerInfo *root,
agg_info->input_rows, NULL);
}
+ /*
+ * Set the consider_parallel flag if this joinrel could potentially be
+ * scanned within a parallel worker. If this flag is false for either
+ * inner_rel or outer_rel, then it must be false for the joinrel also.
+ * Even if both are true, there might be parallel-restricted expressions
+ * in the targetlist or quals.
+ *
+ * Note that if there are more than two rels in this relation, they could
+ * be divided between inner_rel and outer_rel in any arbitrary way. We
+ * assume this doesn't matter, because we should hit all the same baserels
+ * and joinclauses while building up to this joinrel no matter which we
+ * take; therefore, we should make the same decision here however we get
+ * here.
+ */
+ if (inner_rel->consider_parallel && outer_rel->consider_parallel &&
+ is_parallel_safe(root, (Node *) restrictlist) &&
+ is_parallel_safe(root, (Node *) joinrel->reltarget->exprs))
+ joinrel->consider_parallel = true;
+
/* Add the joinrel to the PlannerInfo. */
if (!grouped)
add_join_rel(root, joinrel);
diff --git a/src/test/regress/expected/agg_pushdown.out b/src/test/regress/expected/agg_pushdown.out
index 413abc147e..66d36d122e 100644
--- a/src/test/regress/expected/agg_pushdown.out
+++ b/src/test/regress/expected/agg_pushdown.out
@@ -91,6 +91,7 @@ AS c1 ON c1.parent = p.i GROUP BY p.i;
-> Seq Scan on agg_pushdown_child1 c1
(12 rows)
+-- Restore the default values.
SET enable_nestloop TO on;
SET enable_hashjoin TO on;
-- Scan index on agg_pushdown_child1(parent) column and aggregate the result
@@ -213,3 +214,159 @@ c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
-> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
(13 rows)
+-- Most of the tests above with parallel query processing enforced.
+SET min_parallel_index_scan_size = 0;
+SET min_parallel_table_scan_size = 0;
+SET parallel_setup_cost = 0;
+SET parallel_tuple_cost = 0;
+-- Partially aggregate a single relation.
+--
+-- Nestloop join.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 1
+ -> Nested Loop
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Parallel Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(10 rows)
+
+-- Hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Sort
+ Sort Key: p.i
+ -> Gather
+ Workers Planned: 1
+ -> Parallel Hash Join
+ Hash Cond: (c1.parent = p.i)
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Parallel Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Parallel Hash
+ -> Parallel Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(13 rows)
+
+-- Merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 1
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Partial GroupAggregate
+ Group Key: c1.parent
+ -> Parallel Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(10 rows)
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 2
+ -> Sort
+ Sort Key: p.i
+ -> Nested Loop
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Nested Loop
+ -> Parallel Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ Index Cond: ((k = c1.j) AND (parent = c1.parent))
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+ Index Cond: (i = c1.parent)
+(15 rows)
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 1
+ -> Sort
+ Sort Key: p.i
+ -> Parallel Hash Join
+ Hash Cond: (c1.parent = p.i)
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Parallel Hash Join
+ Hash Cond: ((c1.parent = c2.parent) AND (c1.j = c2.k))
+ -> Parallel Index Scan using agg_pushdown_child1_parent_idx on agg_pushdown_child1 c1
+ -> Parallel Hash
+ -> Parallel Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Parallel Hash
+ -> Parallel Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(17 rows)
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------
+ Finalize GroupAggregate
+ Group Key: p.i
+ -> Gather Merge
+ Workers Planned: 2
+ -> Merge Join
+ Merge Cond: (c1.parent = p.i)
+ -> Sort
+ Sort Key: c1.parent
+ -> Partial HashAggregate
+ Group Key: c1.parent
+ -> Merge Join
+ Merge Cond: ((c1.j = c2.k) AND (c1.parent = c2.parent))
+ -> Parallel Index Scan using agg_pushdown_child1_pkey on agg_pushdown_child1 c1
+ -> Index Scan using agg_pushdown_child2_pkey on agg_pushdown_child2 c2
+ -> Index Only Scan using agg_pushdown_parent_pkey on agg_pushdown_parent p
+(15 rows)
+
diff --git a/src/test/regress/sql/agg_pushdown.sql b/src/test/regress/sql/agg_pushdown.sql
index 6bbbc7a8a1..49ba6dd67c 100644
--- a/src/test/regress/sql/agg_pushdown.sql
+++ b/src/test/regress/sql/agg_pushdown.sql
@@ -61,6 +61,7 @@ EXPLAIN (COSTS off)
SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
AS c1 ON c1.parent = p.i GROUP BY p.i;
+-- Restore the default values.
SET enable_nestloop TO on;
SET enable_hashjoin TO on;
@@ -112,3 +113,68 @@ EXPLAIN (COSTS off)
SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- Most of the tests above with parallel query processing enforced.
+SET min_parallel_index_scan_size = 0;
+SET min_parallel_table_scan_size = 0;
+SET parallel_setup_cost = 0;
+SET parallel_tuple_cost = 0;
+
+-- Partially aggregate a single relation.
+--
+-- Nestloop join.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+EXPLAIN (COSTS off)
+SELECT p.x, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- Hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+-- Merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v) FROM agg_pushdown_parent AS p JOIN agg_pushdown_child1
+AS c1 ON c1.parent = p.i GROUP BY p.i;
+
+SET enable_nestloop TO on;
+SET enable_hashjoin TO on;
+
+-- Perform nestloop join between agg_pushdown_child1 and agg_pushdown_child2
+-- and aggregate the result.
+SET enable_nestloop TO on;
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for hash join.
+SET enable_nestloop TO off;
+SET enable_hashjoin TO on;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
+
+-- The same for merge join.
+SET enable_hashjoin TO off;
+SET enable_mergejoin TO on;
+SET enable_seqscan TO off;
+
+EXPLAIN (COSTS off)
+SELECT p.i, avg(c1.v + c2.v) FROM agg_pushdown_parent AS p JOIN
+agg_pushdown_child1 AS c1 ON c1.parent = p.i JOIN agg_pushdown_child2 AS c2 ON
+c2.parent = p.i WHERE c1.j = c2.k GROUP BY p.i;
--
2.21.3
Re: WIP: Aggregation push-down
On 7/3/20 6:07 AM, Laurenz Albe wrote:
On Thu, 2020-07-02 at 14:39 +0200, Daniel Gustafsson wrote:
This version now fails to apply to HEAD, with what looks like like a trivial
error in the expected test output. Can you please submit a rebased version so
we can see it run in the patch tester CI? I'm marking the entry as Waiting on
Author in the meantime.I have rebased the patch against current HEAD, it passes "make installcheck".
This needs another rebase so marked as Waiting on Author:
http://cfbot.cputube.org/patch_32_1247.log
As far as I can see reviewer concerns have been addressed as they have
come up, but as Tomas noted this is a bit of a niche feature as far as
who might commit it.
Tom, Tomas, would you care to share your remaining concerns, if any?
Regards,
--
-David
david@pgmasters.net
Re: WIP: Aggregation push-down
David Steele <david@pgmasters.net> wrote:
On 7/3/20 6:07 AM, Laurenz Albe wrote:
On Thu, 2020-07-02 at 14:39 +0200, Daniel Gustafsson wrote:
This version now fails to apply to HEAD, with what looks like like a trivial
error in the expected test output. Can you please submit a rebased version so
we can see it run in the patch tester CI? I'm marking the entry as Waiting on
Author in the meantime.I have rebased the patch against current HEAD, it passes "make installcheck".
This needs another rebase so marked as Waiting on Author:
http://cfbot.cputube.org/patch_32_1247.log
Besides the fact that I don't have time to work on it now, I'm not able to
convince myself to put more effort into it. If there's almost no progress for
several years, I don't believe another rebase(s) is anything but wasted
effort.
So I've withdrawn the patch, also to save CFMs from examining the history
again and again uselessly. The code is free so anyone can continue if he
thinks it makes sense. If it finally finds its way into the PG core and if
meaningful part of my code remains, I'd appreciate it if I was mentioned in
the release notes. Thanks.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
Re: WIP: Aggregation push-down
On 3/11/21 5:10 AM, Antonin Houska wrote:
David Steele <david@pgmasters.net> wrote:
On 7/3/20 6:07 AM, Laurenz Albe wrote:
On Thu, 2020-07-02 at 14:39 +0200, Daniel Gustafsson wrote:
This version now fails to apply to HEAD, with what looks like like a trivial
error in the expected test output. Can you please submit a rebased version so
we can see it run in the patch tester CI? I'm marking the entry as Waiting on
Author in the meantime.I have rebased the patch against current HEAD, it passes "make installcheck".
This needs another rebase so marked as Waiting on Author:
http://cfbot.cputube.org/patch_32_1247.logBesides the fact that I don't have time to work on it now, I'm not able to
convince myself to put more effort into it. If there's almost no progress for
several years, I don't believe another rebase(s) is anything but wasted
effort.So I've withdrawn the patch, also to save CFMs from examining the history
again and again uselessly. The code is free so anyone can continue if he
thinks it makes sense. If it finally finds its way into the PG core and if
meaningful part of my code remains, I'd appreciate it if I was mentioned in
the release notes. Thanks.
I'm sure you would receive credit if anyone picked this patch up.
Sorry it didn't work out, but it looks like withdrawing the patch was
the right way to go.
Regards,
--
-David
david@pgmasters.net
RE: WIP: Aggregation push-down
Hi everyone.
I develop postgresql's extension such as fdw in my work.
I'm interested in using postgresql for OLAP.
I think that this patch is realy useful when using OLAP queries.
Furthermore, I think it would be more useful if this patch works on a foreign table.
Actually, I changed this patch a little and confirmed that my idea is true.
The followings are things I found and differences of between my prototype and this patch.
1. Things I found
I execute a query which contain join of postgres_fdw's foreign table and a table and aggregation of the josin result.
In my setting, my prototype reduce this query's response by 93%.
2. Differences between my prototype and this patch
(1) Pushdown aggregation of foeign table if FDW pushdown partioal aggregation
(2) postgres_fdw pushdowns some partial aggregations
I attached my prototype source code and content of my experiment.
I want to resume development of this patch if there is some possibility of accept of this patch's function.
. I took a contact to Mr.Houska on resuming development of this patch.
As a result, Mr.Houska advised for me that I ask in pgsql-hackers whether any reviewers / committers are
interested to work on the patch.
Is anyone interested in my work?
Sincerely yours.
Yuuki Fujii
--
Yuuki Fujii
Information Technology R&D Center Mitsubishi Electric Corporation
Attachments:
v17-0004-pushdown-aggregation-foreign-table.patchapplication/octet-stream; name=v17-0004-pushdown-aggregation-foreign-table.patchDownload
diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c
index ad37a74221..ef4d8ab120 100644
--- a/contrib/postgres_fdw/deparse.c
+++ b/contrib/postgres_fdw/deparse.c
@@ -100,8 +100,21 @@ typedef struct deparse_expr_cxt
* a base relation. */
StringInfo buf; /* output buffer to append to */
List **params_list; /* exprs that will become remote Params */
+ RelAggInfo *agg_info;
+ List **avg_attrs;
+ List **sum_attrs;
+ AggSplit aggsplit;
} deparse_expr_cxt;
+typedef enum {
+ AGGFUNC_SUM,
+ AGGFUNC_AVG,
+ AGGFUNC_COUNT,
+ AGGFUNC_MAX,
+ AGGFUNC_MIN,
+ AGGFUNC_OTHER
+} AggFuncType;
+
#define REL_ALIAS_PREFIX "r"
/* Handy macro to add relation name qualification */
#define ADD_REL_QUALIFIER(buf, varno) \
@@ -184,6 +197,7 @@ static void appendAggOrderBy(List *orderList, List *targetList,
static void appendFunctionName(Oid funcid, deparse_expr_cxt *context);
static Node *deparseSortGroupClause(Index ref, List *tlist, bool force_colno,
deparse_expr_cxt *context);
+static AggFuncType getAggFunc(Aggref* aggref);
/*
* Helper functions
@@ -693,6 +707,7 @@ foreign_expr_walker(Node *node,
case T_Aggref:
{
Aggref *agg = (Aggref *) node;
+ AggFuncType aggfunc_type = AGGFUNC_OTHER;
ListCell *lc;
/* Not safe to pushdown when not in grouping context */
@@ -700,8 +715,15 @@ foreign_expr_walker(Node *node,
return false;
/* Only non-split aggregates are pushable. */
- if (agg->aggsplit != AGGSPLIT_SIMPLE)
+ if (agg->aggsplit == AGGSPLIT_INITIAL_SERIAL) {
+ fpinfo->aggsplit = AGGSPLIT_INITIAL_SERIAL;
+ aggfunc_type = getAggFunc(agg);
+ if (aggfunc_type == AGGFUNC_OTHER) {
+ return false;
+ }
+ } else if (agg->aggsplit != AGGSPLIT_SIMPLE) {
return false;
+ }
/* As usual, it must be shippable. */
if (!is_shippable(agg->aggfnoid, ProcedureRelationId, fpinfo))
@@ -723,6 +745,18 @@ foreign_expr_walker(Node *node,
n = (Node *) tle->expr;
}
+ if (fpinfo->aggsplit == AGGSPLIT_INITIAL_SERIAL) {
+ bool pushdown = false;
+ if (nodeTag(n) == T_Var) {
+ Var *var = (Var*)n;
+ if (var->vartype == INT8OID) {
+ pushdown = true;
+ }
+ }
+ if (pushdown == false) {
+ return false;
+ }
+ }
if (!foreign_expr_walker(n, glob_cxt, &inner_cxt))
return false;
@@ -995,7 +1029,8 @@ void
deparseSelectStmtForRel(StringInfo buf, PlannerInfo *root, RelOptInfo *rel,
List *tlist, List *remote_conds, List *pathkeys,
bool has_final_sort, bool has_limit, bool is_subquery,
- List **retrieved_attrs, List **params_list)
+ List **retrieved_attrs, List **params_list, List **avg_attrs,
+ List **sum_attrs)
{
deparse_expr_cxt context;
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) rel->fdw_private;
@@ -1013,6 +1048,10 @@ deparseSelectStmtForRel(StringInfo buf, PlannerInfo *root, RelOptInfo *rel,
context.foreignrel = rel;
context.scanrel = IS_UPPER_REL(rel) ? fpinfo->outerrel : rel;
context.params_list = params_list;
+ context.agg_info = fpinfo->agg_info;
+ context.avg_attrs = avg_attrs;
+ context.sum_attrs = sum_attrs;
+ context.aggsplit = fpinfo->aggsplit;
/* Construct SELECT clause */
deparseSelectSql(tlist, is_subquery, retrieved_attrs, &context);
@@ -1075,7 +1114,7 @@ deparseSelectStmtForRel(StringInfo buf, PlannerInfo *root, RelOptInfo *rel,
*/
static void
deparseSelectSql(List *tlist, bool is_subquery, List **retrieved_attrs,
- deparse_expr_cxt *context)
+ deparse_expr_cxt *context)
{
StringInfo buf = context->buf;
RelOptInfo *foreignrel = context->foreignrel;
@@ -1391,6 +1430,142 @@ get_jointype_name(JoinType jointype)
return NULL;
}
+/* Get function name */
+static StringInfo getFuncName(Oid funcid) {
+ StringInfo buf;
+ HeapTuple proctup;
+ Form_pg_proc procform;
+ const char* proname;
+
+ buf = makeStringInfo();
+ initStringInfo(buf);
+
+ proctup = SearchSysCache1(PROCOID, ObjectIdGetDatum(funcid));
+ if (!HeapTupleIsValid(proctup))
+ elog(ERROR, "cache lookup failed for function %u", funcid);
+ procform = (Form_pg_proc)GETSTRUCT(proctup);
+
+ /* Print schema name only if it's not pg_catalog */
+ if (procform->pronamespace != PG_CATALOG_NAMESPACE)
+ {
+ const char* schemaname;
+
+ schemaname = get_namespace_name(procform->pronamespace);
+ appendStringInfo(buf, "%s.", quote_identifier(schemaname));
+ }
+
+ /* Always print the function name */
+ proname = NameStr(procform->proname);
+ appendStringInfoString(buf, quote_identifier(proname));
+
+ ReleaseSysCache(proctup);
+
+ return buf;
+}
+
+/* Get name of aggregation function */
+static AggFuncType getAggFunc(Aggref *aggref) {
+ AggFuncType type = AGGFUNC_OTHER;
+ StringInfo buf = getFuncName(aggref->aggfnoid);
+ if (strcmp(buf->data, "avg") == 0) {
+ type = AGGFUNC_AVG;
+ } else if (strcmp(buf->data, "sum") == 0) {
+ type = AGGFUNC_SUM;
+ } else if (strcmp(buf->data, "count") == 0) {
+ type = AGGFUNC_COUNT;
+ } else if (strcmp(buf->data, "min") == 0) {
+ type = AGGFUNC_MIN;
+ } else if (strcmp(buf->data, "max") == 0) {
+ type = AGGFUNC_MAX;
+ } else {
+ type = AGGFUNC_OTHER;
+ }
+ return type;
+}
+
+/* Deparse arg of aggref */
+static void deparseAggrefArg(Aggref * node, deparse_expr_cxt *context){
+ StringInfo buf = context->buf;
+ bool use_variadic;
+
+ use_variadic = node->aggvariadic;
+
+ appendStringInfoChar(buf, '(');
+
+ /* Add DISTINCT */
+ appendStringInfoString(buf, (node->aggdistinct != NIL) ? "DISTINCT " : "");
+
+ if (AGGKIND_IS_ORDERED_SET(node->aggkind))
+ {
+ /* Add WITHIN GROUP (ORDER BY ..) */
+ ListCell* arg;
+ bool first = true;
+
+ Assert(!node->aggvariadic);
+ Assert(node->aggorder != NIL);
+
+ foreach(arg, node->aggdirectargs)
+ {
+ if (!first)
+ appendStringInfoString(buf, ", ");
+ first = false;
+
+ deparseExpr((Expr*)lfirst(arg), context);
+ }
+
+ appendStringInfoString(buf, ") WITHIN GROUP (ORDER BY ");
+ appendAggOrderBy(node->aggorder, node->args, context);
+ }
+ else
+ {
+ /* aggstar can be set only in zero-argument aggregates */
+ if (node->aggstar)
+ appendStringInfoChar(buf, '*');
+ else
+ {
+ ListCell* arg;
+ bool first = true;
+
+ /* Add all the arguments */
+ foreach(arg, node->args)
+ {
+ TargetEntry* tle = (TargetEntry*)lfirst(arg);
+ Node* n = (Node*)tle->expr;
+
+ if (tle->resjunk)
+ continue;
+
+ if (!first)
+ appendStringInfoString(buf, ", ");
+ first = false;
+
+ /* Add VARIADIC */
+ if (use_variadic && lnext(node->args, arg) == NULL)
+ appendStringInfoString(buf, "VARIADIC ");
+
+ deparseExpr((Expr*)n, context);
+ }
+ }
+
+ /* Add ORDER BY */
+ if (node->aggorder != NIL)
+ {
+ appendStringInfoString(buf, " ORDER BY ");
+ appendAggOrderBy(node->aggorder, node->args, context);
+ }
+ }
+
+ /* Add FILTER (WHERE ..) */
+ if (node->aggfilter != NULL)
+ {
+ appendStringInfoString(buf, ") FILTER (WHERE ");
+ deparseExpr((Expr*)node->aggfilter, context);
+ }
+
+ appendStringInfoChar(buf, ')');
+}
+
+
/*
* Deparse given targetlist and append it to context->buf.
*
@@ -1417,13 +1592,33 @@ deparseExplicitTargetList(List *tlist,
foreach(lc, tlist)
{
TargetEntry *tle = lfirst_node(TargetEntry, lc);
+ AggFuncType type;
if (i > 0)
appendStringInfoString(buf, ", ");
else if (is_returning)
appendStringInfoString(buf, " RETURNING ");
- deparseExpr((Expr *) tle->expr, context);
+ if ((nodeTag((Expr*)tle->expr) == T_Aggref)
+ && (context->aggsplit == AGGSPLIT_INITIAL_SERIAL)) {
+ type = getAggFunc((Aggref*)tle->expr);
+ if (type == AGGFUNC_AVG) {
+ *(context->avg_attrs) = lappend_int(*(context->avg_attrs), i + 1);
+ appendStringInfoString(buf, "count");
+ deparseAggrefArg((Aggref*)tle->expr, context);
+
+ appendStringInfoString(buf, ", ");
+ appendStringInfoString(buf, "sum");
+ deparseAggrefArg((Aggref*)tle->expr, context);
+ } else if (type == AGGFUNC_SUM) {
+ *(context->sum_attrs) = lappend_int(*(context->sum_attrs), i + 1);
+ deparseExpr((Expr*)tle->expr, context);
+ } else {
+ deparseExpr((Expr*)tle->expr, context);
+ }
+ } else {
+ deparseExpr((Expr*)tle->expr, context);
+ }
*retrieved_attrs = lappend_int(*retrieved_attrs, i + 1);
i++;
@@ -1646,6 +1841,8 @@ deparseRangeTblRef(StringInfo buf, PlannerInfo *root, RelOptInfo *foreignrel,
if (make_subquery)
{
List *retrieved_attrs;
+ List *avg_attrs = NIL;
+ List *sum_attrs = NIL;
int ncols;
/*
@@ -1661,7 +1858,8 @@ deparseRangeTblRef(StringInfo buf, PlannerInfo *root, RelOptInfo *foreignrel,
deparseSelectStmtForRel(buf, root, foreignrel, NIL,
fpinfo->remote_conds, NIL,
false, false, true,
- &retrieved_attrs, params_list);
+ &retrieved_attrs, params_list, &avg_attrs,
+ &sum_attrs);
appendStringInfoChar(buf, ')');
/* Append the relation alias. */
@@ -3148,9 +3346,17 @@ appendGroupByClause(List *tlist, deparse_expr_cxt *context)
Query *query = context->root->parse;
ListCell *lc;
bool first = true;
+ List *groupClause;
+
+ if (context->agg_info) {
+ groupClause = context->agg_info->group_clauses;
+ }
+ else {
+ groupClause = query->groupClause;
+ }
/* Nothing to be done, if there's no GROUP BY clause in the query. */
- if (!query->groupClause)
+ if (!groupClause)
return;
appendStringInfoString(buf, " GROUP BY ");
@@ -3161,7 +3367,7 @@ appendGroupByClause(List *tlist, deparse_expr_cxt *context)
*/
Assert(!query->groupingSets);
- foreach(lc, query->groupClause)
+ foreach(lc, groupClause)
{
SortGroupClause *grp = (SortGroupClause *) lfirst(lc);
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index a46834a377..f99a0e2b7f 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -44,6 +44,8 @@
#include "utils/rel.h"
#include "utils/sampling.h"
#include "utils/selfuncs.h"
+#include "utils/numeric.h"
+#include "libpq/pqformat.h"
PG_MODULE_MAGIC;
@@ -72,6 +74,10 @@ enum FdwScanPrivateIndex
/* Integer representing the desired fetch_size */
FdwScanPrivateFetchSize,
+ FdwScanPrivateAvgAttrs,
+ FdwScanPrivateSumAttrs,
+ FdwScanPrivateAggSplit,
+
/*
* String describing join i.e. names of relations being joined and types
* of join, added when the scan is join
@@ -159,6 +165,10 @@ typedef struct PgFdwScanState
MemoryContext temp_cxt; /* context for per-tuple temporary data */
int fetch_size; /* number of tuples per fetch */
+
+ List *avg_attrs;
+ List *sum_attrs;
+ AggSplit aggsplit;
} PgFdwScanState;
/*
@@ -481,7 +491,7 @@ static bool foreign_join_ok(PlannerInfo *root, RelOptInfo *joinrel,
JoinType jointype, RelOptInfo *outerrel, RelOptInfo *innerrel,
JoinPathExtraData *extra);
static bool foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel,
- Node *havingQual);
+ Node *havingQual, RelAggInfo *agg_info);
static List *get_useful_pathkeys_for_relation(PlannerInfo *root,
RelOptInfo *rel);
static List *get_useful_ecs_for_relation(PlannerInfo *root, RelOptInfo *rel);
@@ -1190,6 +1200,8 @@ postgresGetForeignPlan(PlannerInfo *root,
bool has_final_sort = false;
bool has_limit = false;
ListCell *lc;
+ List *avg_attrs = NIL;
+ List *sum_attrs = NIL;
/*
* Get FDW private data created by postgresGetForeignUpperPaths(), if any.
@@ -1351,7 +1363,8 @@ postgresGetForeignPlan(PlannerInfo *root,
deparseSelectStmtForRel(&sql, root, foreignrel, fdw_scan_tlist,
remote_exprs, best_path->path.pathkeys,
has_final_sort, has_limit, false,
- &retrieved_attrs, ¶ms_list);
+ &retrieved_attrs, ¶ms_list, &avg_attrs,
+ &sum_attrs);
/* Remember remote_exprs for possible use by postgresPlanDirectModify */
fpinfo->final_remote_exprs = remote_exprs;
@@ -1360,9 +1373,12 @@ postgresGetForeignPlan(PlannerInfo *root,
* Build the fdw_private list that will be available to the executor.
* Items in the list must match order in enum FdwScanPrivateIndex.
*/
- fdw_private = list_make3(makeString(sql.data),
+ fdw_private = list_make4(makeString(sql.data),
retrieved_attrs,
- makeInteger(fpinfo->fetch_size));
+ makeInteger(fpinfo->fetch_size),
+ avg_attrs);
+ fdw_private = lappend(fdw_private, sum_attrs);
+ fdw_private = lappend(fdw_private, makeInteger(fpinfo->aggsplit));
if (IS_JOIN_REL(foreignrel) || IS_UPPER_REL(foreignrel))
fdw_private = lappend(fdw_private,
makeString(fpinfo->relation_name));
@@ -1448,6 +1464,15 @@ postgresBeginForeignScan(ForeignScanState *node, int eflags)
fsstate->fetch_size = intVal(list_nth(fsplan->fdw_private,
FdwScanPrivateFetchSize));
+ fsstate->avg_attrs = (List*)(list_nth(fsplan->fdw_private,
+ FdwScanPrivateAvgAttrs));
+
+ fsstate->sum_attrs = (List*)(list_nth(fsplan->fdw_private,
+ FdwScanPrivateSumAttrs));
+
+ fsstate->aggsplit = intVal(list_nth(fsplan->fdw_private,
+ FdwScanPrivateAggSplit));
+
/* Create contexts for batches of tuples and per-tuple temp workspace. */
fsstate->batch_cxt = AllocSetContextCreate(estate->es_query_cxt,
"postgres_fdw tuple data",
@@ -2710,6 +2735,8 @@ estimate_path_cost_size(PlannerInfo *root,
/* Required only to be passed to deparseSelectStmtForRel */
List *retrieved_attrs;
+ List *avg_attrs;
+ List *sum_attrs;
/*
* param_join_conds might contain both clauses that are safe to send
@@ -2743,7 +2770,8 @@ estimate_path_cost_size(PlannerInfo *root,
remote_conds, pathkeys,
fpextra ? fpextra->has_final_sort : false,
fpextra ? fpextra->has_limit : false,
- false, &retrieved_attrs, NULL);
+ false, &retrieved_attrs, NULL, &avg_attrs,
+ &sum_attrs);
/* Get the remote estimate */
conn = GetConnection(fpinfo->user, false);
@@ -2920,6 +2948,7 @@ estimate_path_cost_size(PlannerInfo *root,
double input_rows;
int numGroupCols;
double numGroups = 1;
+ List *group_exprs;
/* The upper relation should have its outer relation set */
Assert(outerrel);
@@ -2957,9 +2986,16 @@ estimate_path_cost_size(PlannerInfo *root,
/* Get number of grouping columns and possible number of groups */
numGroupCols = list_length(root->parse->groupClause);
+ numGroups = 100;
+ if (fpinfo->agg_info) {
+ group_exprs = fpinfo->agg_info->group_exprs;
+ }
+ else {
+ group_exprs = get_sortgrouplist_exprs(root->parse->groupClause,
+ fpinfo->grouped_tlist);
+ }
numGroups = estimate_num_groups(root,
- get_sortgrouplist_exprs(root->parse->groupClause,
- fpinfo->grouped_tlist),
+ group_exprs,
input_rows, NULL);
/*
@@ -5568,7 +5604,7 @@ postgresGetForeignJoinPaths(PlannerInfo *root,
*/
static bool
foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel,
- Node *havingQual)
+ Node *havingQual, RelAggInfo *agg_info)
{
Query *query = root->parse;
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) grouped_rel->fdw_private;
@@ -5615,9 +5651,10 @@ foreign_grouping_ok(PlannerInfo *root, RelOptInfo *grouped_rel,
Expr *expr = (Expr *) lfirst(lc);
Index sgref = get_pathtarget_sortgroupref(grouping_target, i);
ListCell *l;
+ List *groupClause = (agg_info != NULL) ? agg_info->group_clauses : query->groupClause;
/* Check whether this expression is part of GROUP BY clause */
- if (sgref && get_sortgroupref_clause_noerr(sgref, query->groupClause))
+ if (sgref && get_sortgroupref_clause_noerr(sgref, groupClause))
{
TargetEntry *tle;
@@ -5888,6 +5925,7 @@ add_foreign_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
fpinfo->table = ifpinfo->table;
fpinfo->server = ifpinfo->server;
fpinfo->user = ifpinfo->user;
+ fpinfo->agg_info = extra->agg_info;
merge_fdw_options(fpinfo, ifpinfo, NULL);
/*
@@ -5896,7 +5934,8 @@ add_foreign_grouping_paths(PlannerInfo *root, RelOptInfo *input_rel,
* Use HAVING qual from extra. In case of child partition, it will have
* translated Vars.
*/
- if (!foreign_grouping_ok(root, grouped_rel, extra->havingQual))
+ if (!foreign_grouping_ok(root, grouped_rel, extra->havingQual,
+ extra->agg_info))
return;
/*
@@ -6329,6 +6368,7 @@ make_tuple_from_result_row(PGresult *res,
MemoryContext oldcontext;
ListCell *lc;
int j;
+ PgFdwScanState *festate = (fsstate == NULL) ? NULL : (PgFdwScanState*)fsstate->fdw_state;
Assert(row < PQntuples(res));
@@ -6370,13 +6410,43 @@ make_tuple_from_result_row(PGresult *res,
foreach(lc, retrieved_attrs)
{
int i = lfirst_int(lc);
- char *valstr;
-
- /* fetch next column's textual value */
- if (PQgetisnull(res, row, j))
- valstr = NULL;
- else
- valstr = PQgetvalue(res, row, j);
+ char *valstr = NULL;
+ char *countstr = NULL;
+ char *sumstr = NULL;
+ bool split_serial = false;
+
+ if (festate == NULL) {
+ split_serial = false;
+ } else if (festate->aggsplit == AGGSPLIT_INITIAL_SERIAL) {
+ split_serial = true;
+ } else {
+ split_serial = false;
+ }
+ if (split_serial == true) {
+ if (list_member_int(festate->avg_attrs, i)) {
+ /* fetch next column's textual value */
+ if (PQgetisnull(res, row, j)) {
+ countstr = NULL;
+ sumstr = NULL;
+ } else {
+ countstr = PQgetvalue(res, row, j);
+ j++;
+ sumstr = PQgetvalue(res, row, j);
+ }
+ } else {
+ /* fetch next column's textual value */
+ if (PQgetisnull(res, row, j))
+ valstr = NULL;
+ else
+ valstr = PQgetvalue(res, row, j);
+ }
+ } else {
+ /* fetch next column's textual value */
+ if (PQgetisnull(res, row, j))
+ valstr = NULL;
+ else
+ valstr = PQgetvalue(res, row, j);
+ }
/*
* convert value to internal representation
@@ -6388,12 +6458,63 @@ make_tuple_from_result_row(PGresult *res,
{
/* ordinary column */
Assert(i <= tupdesc->natts);
- nulls[i - 1] = (valstr == NULL);
- /* Apply the input function even to nulls, to support domains */
- values[i - 1] = InputFunctionCall(&attinmeta->attinfuncs[i - 1],
- valstr,
- attinmeta->attioparams[i - 1],
- attinmeta->atttypmods[i - 1]);
+
+ if (split_serial == false) {
+ nulls[i - 1] = (valstr == NULL);
+ /* Apply the input function even to nulls, to support domains */
+ values[i - 1] = InputFunctionCall(&attinmeta->attinfuncs[i - 1],
+ valstr,
+ attinmeta->attioparams[i - 1],
+ attinmeta->atttypmods[i - 1]);
+ } else {
+ if (list_member_int(festate->avg_attrs, i)) {
+ StringInfoData buf;
+ int64 count;
+ Numeric sumN;
+ bytea* sumX;
+ Datum temp;
+ bytea* result;
+
+ nulls[i - 1] = (sumstr == NULL);
+
+ count = DatumGetInt32(DirectFunctionCall1(int8in, PointerGetDatum(countstr)));
+
+ sumN = DatumGetNumeric(DirectFunctionCall3(numeric_in, PointerGetDatum(sumstr), 0,
+ Int32GetDatum(attinmeta->atttypmods[i - 1])));
+ temp = DirectFunctionCall1(numeric_send, NumericGetDatum(sumN));
+ sumX = DatumGetByteaPP(temp);
+ pq_begintypsend(&buf);
+ pq_sendint64(&buf, count);
+ pq_sendbytes(&buf, VARDATA_ANY(sumX), VARSIZE_ANY_EXHDR(sumX));
+ result = pq_endtypsend(&buf);
+ values[i - 1] = PointerGetDatum(result);
+ } else if (list_member_int(festate->sum_attrs, i)) {
+ StringInfoData buf;
+ Numeric sumN;
+ bytea* sumX;
+ Datum temp;
+ bytea* result;
+
+ nulls[i - 1] = (valstr == NULL);
+
+ sumN = DatumGetNumeric(DirectFunctionCall3(numeric_in, PointerGetDatum(valstr), 0,
+ Int32GetDatum(attinmeta->atttypmods[i - 1])));
+ temp = DirectFunctionCall1(numeric_send, NumericGetDatum(sumN));
+ sumX = DatumGetByteaPP(temp);
+ pq_begintypsend(&buf);
+ pq_sendint64(&buf, 1);
+ pq_sendbytes(&buf, VARDATA_ANY(sumX), VARSIZE_ANY_EXHDR(sumX));
+ result = pq_endtypsend(&buf);
+ values[i - 1] = PointerGetDatum(result);
+ } else {
+ nulls[i - 1] = (valstr == NULL);
+ /* Apply the input function even to nulls, to support domains */
+ values[i - 1] = InputFunctionCall(&attinmeta->attinfuncs[i - 1],
+ valstr,
+ attinmeta->attioparams[i - 1],
+ attinmeta->atttypmods[i - 1]);
+ }
+ }
}
else if (i == SelfItemPointerAttributeNumber)
{
diff --git a/contrib/postgres_fdw/postgres_fdw.h b/contrib/postgres_fdw/postgres_fdw.h
index eef410db39..40c564abef 100644
--- a/contrib/postgres_fdw/postgres_fdw.h
+++ b/contrib/postgres_fdw/postgres_fdw.h
@@ -122,6 +122,8 @@ typedef struct PgFdwRelationInfo
* representing the relation.
*/
int relation_index;
+ RelAggInfo *agg_info;
+ AggSplit aggsplit;
} PgFdwRelationInfo;
/* in postgres_fdw.c */
@@ -200,8 +202,9 @@ extern void deparseSelectStmtForRel(StringInfo buf, PlannerInfo *root,
RelOptInfo *foreignrel, List *tlist,
List *remote_conds, List *pathkeys,
bool has_final_sort, bool has_limit,
- bool is_subquery,
- List **retrieved_attrs, List **params_list);
+ bool is_subquery, List **retrieved_attrs,
+ List **params_list, List **avg_attrs,
+ List **sum_attrs);
extern const char *get_jointype_name(JoinType jointype);
/* in shippable.c */
diff --git a/src/backend/optimizer/path/allpaths.c b/src/backend/optimizer/path/allpaths.c
index e0b5418de4..0fd00ef2bd 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -49,7 +49,6 @@
#include "rewrite/rewriteManip.h"
#include "utils/lsyscache.h"
-
/* results of subquery_is_pushdown_safe */
typedef struct pushdown_safety_info
{
@@ -540,8 +539,37 @@ set_rel_pathlist(PlannerInfo *root, RelOptInfo *rel,
case RTE_RELATION:
if (rte->relkind == RELKIND_FOREIGN_TABLE)
{
+ RelOptInfo* rel_grouped;
+ RelAggInfo* agg_info;
+ GroupPathExtraData extra;
+
/* Foreign table */
set_foreign_pathlist(root, rel, rte);
+
+ /* Add paths to the grouped relation if one exists. */
+ rel_grouped = find_grouped_rel(root, rel->relids,
+ &agg_info);
+ if (rel_grouped) {
+ extra.target_parallel_safe = false;
+ extra.targetList = root->parse->targetList;
+ extra.havingQual = root->parse->havingQual;
+ extra.partial_costs_set = false;
+ extra.agg_info = agg_info;
+
+ if (enable_partitionwise_aggregate && !root->parse->groupingSets)
+ extra.patype = PARTITIONWISE_AGGREGATE_FULL;
+ else
+ extra.patype = PARTITIONWISE_AGGREGATE_NONE;
+
+ rel_grouped->fdwroutine->GetForeignUpperPaths(root, UPPERREL_GROUP_AGG,
+ rel, rel_grouped, (void*)&extra);
+ if (rel_grouped->pathlist != NIL) {
+ set_cheapest(rel_grouped);
+ }
+ else {
+ del_grouped_rel(root, rel_grouped, agg_info);
+ }
+ }
}
else if (rte->tablesample != NULL)
{
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 6fe12d08ce..0f6b787a91 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -3882,6 +3882,7 @@ create_grouping_paths(PlannerInfo *root,
extra.havingQual = parse->havingQual;
extra.targetList = parse->targetList;
extra.partial_costs_set = false;
+ extra.agg_info = NULL;
/*
* Determine whether partitionwise aggregation is in theory possible.
diff --git a/src/backend/optimizer/util/relnode.c b/src/backend/optimizer/util/relnode.c
index 4164f4b0e4..0854d426f9 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -412,9 +412,7 @@ build_simple_grouped_rel(PlannerInfo *root, int relid,
* TODO Consider relaxing some of these restrictions.
*/
rte = root->simple_rte_array[rel_plain->relid];
- if (rte->rtekind != RTE_RELATION ||
- rte->relkind == RELKIND_FOREIGN_TABLE ||
- rte->tablesample != NULL)
+ if (rte->rtekind != RTE_RELATION || rte->tablesample != NULL)
return NULL;
/*
@@ -443,6 +441,10 @@ build_simple_grouped_rel(PlannerInfo *root, int relid,
*/
rel_grouped = makeNode(RelOptInfo);
memcpy(rel_grouped, rel_plain, sizeof(RelOptInfo));
+ if (rte->relkind == RELKIND_FOREIGN_TABLE) {
+ rel_grouped->fdw_private = NULL;
+ rel_grouped->reloptkind = RELOPT_UPPER_REL;
+ }
/*
* Note on consider_startup: while the AGG_HASHED strategy needs the whole
@@ -658,6 +660,39 @@ add_rel_info(RelInfoList *list, void *data)
}
}
+/*
+ * del_rel_info
+ * Delete relation specific info to a list, and also add it to the auxiliary
+ * hashtable if there is one.
+ */
+static void
+del_rel_info(RelInfoList* list, void* data)
+{
+ Assert(IsA(data, RelOptInfo) || IsA(data, RelAggInfo));
+
+ /* GEQO requires us to append the new joinrel to the end of the list! */
+ list->items = list_delete(list->items, data);
+
+ /* store it into the auxiliary hashtable if there is one. */
+ if (list->hash)
+ {
+ Relids relids;
+ bool found;
+
+ if (IsA(data, RelOptInfo))
+ relids = ((RelOptInfo*)data)->relids;
+ else if (IsA(data, RelAggInfo))
+ relids = ((RelAggInfo*)data)->relids;
+
+ hash_search(list->hash,
+ &relids,
+ HASH_REMOVE,
+ &found);
+ Assert(!found);
+ }
+}
+
+
/*
* add_join_rel
* Add given join relation to the list of join relations in the given
@@ -682,6 +717,19 @@ add_grouped_rel(PlannerInfo *root, RelOptInfo *rel, RelAggInfo *agg_info)
add_rel_info(root->agg_info_list, agg_info);
}
+/*
+ * del_grouped_rel
+ * Del grouped base or join relation to the list of grouped relations in
+ * the given PlannerInfo. Also add the corresponding RelAggInfo to
+ * agg_info_list.
+ */
+void
+del_grouped_rel(PlannerInfo* root, RelOptInfo* rel, RelAggInfo* agg_info)
+{
+ del_rel_info(&root->upper_rels[UPPERREL_PARTIAL_GROUP_AGG], rel);
+ del_rel_info(root->agg_info_list, agg_info);
+}
+
/*
* find_grouped_rel
* Returns grouped relation entry (base or join relation) corresponding to
diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c
index 1773fa292e..4cf757c676 100644
--- a/src/backend/utils/adt/numeric.c
+++ b/src/backend/utils/adt/numeric.c
@@ -4874,7 +4874,6 @@ int8_avg_serialize(PG_FUNCTION_ARGS)
{
Datum temp;
NumericVar num;
-
init_var(&num);
#ifdef HAVE_INT128
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 98be8b7de2..1b35d94035 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -2593,6 +2593,7 @@ typedef struct
Node *havingQual;
List *targetList;
PartitionwiseAggregateType patype;
+ RelAggInfo* agg_info;
} GroupPathExtraData;
/*
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 47afa7ed1a..e902f11da0 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -301,6 +301,8 @@ extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
extern void add_grouped_rel(PlannerInfo *root, RelOptInfo *rel,
RelAggInfo *agg_info);
+extern void del_grouped_rel(PlannerInfo* root, RelOptInfo* rel,
+ RelAggInfo* agg_info);
extern RelOptInfo *find_grouped_rel(PlannerInfo *root, Relids relids,
RelAggInfo **agg_info_p);
extern RelOptInfo *build_join_rel(PlannerInfo *root,