Run-time pruning for ModifyTable
Deja vu from this time last year when despite everyone's best efforts
(mostly Alvaro) we missed getting run-time pruning in for MergeAppend
into the PG11 release. This year it was ModifyTable, which is now
possible thanks to Amit and Tom's modifications to the inheritance
planner.
I've attached what I have so far for this. I think it's mostly okay,
but my brain was overheating a bit at the inheritance_planner changes.
I'm not entirely certain that what I've got is correct there. My brain
struggled a bit with the code that Tom wrote to share the data
structures from the SELECT invocation of the grouping_planner() in
inheritance_planner() regarding subquery RTEs. I had to pull out some
more structures from the other PlannerInfo structure in order to get
the base quals from the target rel. I don't quite see a reason why
it's particularly wrong to tag those onto the final_rel, but I'll
prepare myself to be told that I'm wrong about that.
I'm not particularly happy about having to have written the
IS_DUMMY_MODIFYTABLE macro. I just didn't see a more simple way to
determine if the ModifyTable just contains a single dummy Append path.
I also had to change the ModifyTable resultRelInfo pointer to an array
of pointers. This seems to be required since we need to somehow ignore
ResultRelInfos which were pruned. I didn't do any performance testing
for the added level of indirection, I just imagined that it's
unmeasurable.
I'll include this in for July 'fest.
--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
runtime_pruning_for_modifytable.patchapplication/octet-stream; name=runtime_pruning_for_modifytable.patchDownload
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 1759b9e1b6..5193e4a425 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -1951,7 +1951,7 @@ postgresBeginForeignInsert(ModifyTableState *mtstate,
if (plan && plan->operation == CMD_UPDATE &&
(resultRelInfo->ri_usesFdwDirectModify ||
resultRelInfo->ri_FdwState) &&
- resultRelInfo > mtstate->resultRelInfo + mtstate->mt_whichplan)
+ resultRelInfo > mtstate->resultRelInfos[mtstate->mt_whichplan])
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot route tuples into foreign table to be updated \"%s\"",
@@ -2005,7 +2005,7 @@ postgresBeginForeignInsert(ModifyTableState *mtstate,
*/
if (plan && plan->operation == CMD_UPDATE &&
resultRelation == plan->rootRelation)
- resultRelation = mtstate->resultRelInfo[0].ri_RangeTableIndex;
+ resultRelation = mtstate->resultRelInfos[0]->ri_RangeTableIndex;
}
/* Construct the SQL command string. */
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f1161f0fee..7cd927f689 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2853,7 +2853,7 @@ CopyFrom(CopyState cstate)
mtstate->ps.plan = NULL;
mtstate->ps.state = estate;
mtstate->operation = CMD_INSERT;
- mtstate->resultRelInfo = estate->es_result_relations;
+ mtstate->resultRelInfos = &estate->es_result_relations;
if (resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignInsert != NULL)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 92969636b7..a47ec903fa 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -3190,14 +3190,14 @@ show_modifytable_info(ModifyTableState *mtstate, List *ancestors,
/* Should we explicitly label target relations? */
labeltargets = (mtstate->mt_nplans > 1 ||
(mtstate->mt_nplans == 1 &&
- mtstate->resultRelInfo->ri_RangeTableIndex != node->nominalRelation));
+ mtstate->resultRelInfos[0]->ri_RangeTableIndex != node->nominalRelation));
if (labeltargets)
ExplainOpenGroup("Target Tables", "Target Tables", false, es);
for (j = 0; j < mtstate->mt_nplans; j++)
{
- ResultRelInfo *resultRelInfo = mtstate->resultRelInfo + j;
+ ResultRelInfo *resultRelInfo = mtstate->resultRelInfos[j];
FdwRoutine *fdwroutine = resultRelInfo->ri_FdwRoutine;
if (labeltargets)
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index 6f2b4d62b4..b91d286836 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -471,7 +471,7 @@ ExecHashSubPlanResultRelsByOid(ModifyTableState *mtstate,
/* Hash all subplans by their Oid */
for (i = 0; i < mtstate->mt_nplans; i++)
{
- ResultRelInfo *rri = &mtstate->resultRelInfo[i];
+ ResultRelInfo *rri = mtstate->resultRelInfos[i];
bool found;
Oid partoid = RelationGetRelid(rri->ri_RelationDesc);
SubplanResultRelHashElem *elem;
@@ -508,7 +508,7 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
ModifyTable *node = (ModifyTable *) mtstate->ps.plan;
Relation rootrel = rootResultRelInfo->ri_RelationDesc,
partrel;
- Relation firstResultRel = mtstate->resultRelInfo[0].ri_RelationDesc;
+ Relation firstResultRel = mtstate->resultRelInfos[0]->ri_RelationDesc;
ResultRelInfo *leaf_part_rri;
MemoryContext oldcxt;
AttrNumber *part_attnos = NULL;
@@ -556,7 +556,7 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
List *wcoList;
List *wcoExprs = NIL;
ListCell *ll;
- int firstVarno = mtstate->resultRelInfo[0].ri_RangeTableIndex;
+ int firstVarno = mtstate->resultRelInfos[0]->ri_RangeTableIndex;
/*
* In the case of INSERT on a partitioned table, there is only one
@@ -622,7 +622,7 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
TupleTableSlot *slot;
ExprContext *econtext;
List *returningList;
- int firstVarno = mtstate->resultRelInfo[0].ri_RangeTableIndex;
+ int firstVarno = mtstate->resultRelInfos[0]->ri_RangeTableIndex;
/* See the comment above for WCO lists. */
Assert((node->operation == CMD_INSERT &&
@@ -683,7 +683,7 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
*/
if (node && node->onConflictAction != ONCONFLICT_NONE)
{
- int firstVarno = mtstate->resultRelInfo[0].ri_RangeTableIndex;
+ int firstVarno = mtstate->resultRelInfos[0]->ri_RangeTableIndex;
TupleDesc partrelDesc = RelationGetDescr(partrel);
ExprContext *econtext = mtstate->ps.ps_ExprContext;
ListCell *lc;
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 24ed3718ef..650a3a206a 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -57,6 +57,9 @@
#include "utils/memutils.h"
#include "utils/rel.h"
+ /* Special values for mt_whichplan */
+#define WHICHPLAN_CHOOSE_PARTITIONS -1
+#define WHICHPLAN_NO_MATCHING_PARTITIONS -2
static bool ExecOnConflictUpdate(ModifyTableState *mtstate,
ResultRelInfo *resultRelInfo,
@@ -1262,7 +1265,7 @@ lreplace:;
* retrieve the one for this resultRel, we need to know the
* position of the resultRel in mtstate->resultRelInfo[].
*/
- map_index = resultRelInfo - mtstate->resultRelInfo;
+ map_index = mtstate->mt_whichplan;
Assert(map_index >= 0 && map_index < mtstate->mt_nplans);
tupconv_map = tupconv_map_for_subplan(mtstate, map_index);
if (tupconv_map != NULL)
@@ -1710,12 +1713,12 @@ static void
fireBSTriggers(ModifyTableState *node)
{
ModifyTable *plan = (ModifyTable *) node->ps.plan;
- ResultRelInfo *resultRelInfo = node->resultRelInfo;
+ ResultRelInfo *resultRelInfo = node->resultRelInfos[0];
/*
* If the node modifies a partitioned table, we must fire its triggers.
- * Note that in that case, node->resultRelInfo points to the first leaf
- * partition, not the root table.
+ * Note that in that case, node->resultRelInfos[0] points to the first
+ * leaf partition, not the root table.
*/
if (node->rootResultRelInfo != NULL)
resultRelInfo = node->rootResultRelInfo;
@@ -1753,13 +1756,14 @@ static ResultRelInfo *
getTargetResultRelInfo(ModifyTableState *node)
{
/*
- * Note that if the node modifies a partitioned table, node->resultRelInfo
- * points to the first leaf partition, not the root table.
+ * Note that if the node modifies a partitioned table,
+ * node->resultRelInfos[0] points to the first leaf partition, not the
+ * root table.
*/
if (node->rootResultRelInfo != NULL)
return node->rootResultRelInfo;
else
- return node->resultRelInfo;
+ return node->resultRelInfos[0];
}
/*
@@ -1938,7 +1942,7 @@ static void
ExecSetupChildParentMapForSubplan(ModifyTableState *mtstate)
{
ResultRelInfo *targetRelInfo = getTargetResultRelInfo(mtstate);
- ResultRelInfo *resultRelInfos = mtstate->resultRelInfo;
+ ResultRelInfo **resultRelInfos = mtstate->resultRelInfos;
TupleDesc outdesc;
int numResultRelInfos = mtstate->mt_nplans;
int i;
@@ -1958,7 +1962,7 @@ ExecSetupChildParentMapForSubplan(ModifyTableState *mtstate)
for (i = 0; i < numResultRelInfos; ++i)
{
mtstate->mt_per_subplan_tupconv_maps[i] =
- convert_tuples_by_name(RelationGetDescr(resultRelInfos[i].ri_RelationDesc),
+ convert_tuples_by_name(RelationGetDescr(resultRelInfos[i]->ri_RelationDesc),
outdesc,
gettext_noop("could not convert row type"));
}
@@ -2035,8 +2039,47 @@ ExecModifyTable(PlanState *pstate)
node->fireBSTriggers = false;
}
+ if (node->mt_whichplan < 0)
+ {
+ /* Handle choosing the valid partitions */
+ if (node->mt_whichplan == WHICHPLAN_CHOOSE_PARTITIONS)
+ {
+ PartitionPruneState *prunestate = node->mt_prune_state;
+
+ /* There should always be at least one */
+ Assert(node->mt_nplans > 0);
+
+ /*
+ * When partition pruning is enabled and exec params match the
+ * partition key then determine the minimum set of matching
+ * subnodes. Otherwise we match to all subnodes.
+ */
+ if (prunestate != NULL && prunestate->do_exec_prune)
+ {
+ node->mt_valid_subplans = ExecFindMatchingSubPlans(prunestate);
+ node->mt_whichplan = bms_next_member(node->mt_valid_subplans, -1);
+
+ /* If no subplan matches these params then we're done */
+ if (node->mt_whichplan < 0)
+ goto done;
+ }
+ else
+ {
+ node->mt_valid_subplans = bms_add_range(NULL, 0,
+ node->mt_nplans - 1);
+ node->mt_whichplan = 0;
+ }
+ }
+
+ /* partition pruning determined that no partitions match */
+ else if (node->mt_whichplan == WHICHPLAN_NO_MATCHING_PARTITIONS)
+ goto done;
+ else
+ elog(ERROR, "invalid subplan index: %d", node->mt_whichplan);
+ }
+
/* Preload local variables */
- resultRelInfo = node->resultRelInfo + node->mt_whichplan;
+ resultRelInfo = node->resultRelInfos[node->mt_whichplan];
subplanstate = node->mt_plans[node->mt_whichplan];
junkfilter = resultRelInfo->ri_junkFilter;
@@ -2078,10 +2121,12 @@ ExecModifyTable(PlanState *pstate)
if (TupIsNull(planSlot))
{
/* advance to next subplan if any */
- node->mt_whichplan++;
- if (node->mt_whichplan < node->mt_nplans)
+ node->mt_whichplan = bms_next_member(node->mt_valid_subplans,
+ node->mt_whichplan);
+
+ if (node->mt_whichplan >= 0)
{
- resultRelInfo++;
+ resultRelInfo = node->resultRelInfos[node->mt_whichplan];
subplanstate = node->mt_plans[node->mt_whichplan];
junkfilter = resultRelInfo->ri_junkFilter;
estate->es_result_relation_info = resultRelInfo;
@@ -2251,6 +2296,8 @@ ExecModifyTable(PlanState *pstate)
/* Restore es_result_relation_info before exiting */
estate->es_result_relation_info = saved_resultRelInfo;
+done:
+
/*
* We're done, but fire AFTER STATEMENT triggers before exiting.
*/
@@ -2273,9 +2320,11 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
int nplans = list_length(node->plans);
ResultRelInfo *saved_resultRelInfo;
ResultRelInfo *resultRelInfo;
+ Bitmapset *validsubplans;
Plan *subplan;
ListCell *l;
- int i;
+ int i,
+ j;
Relation rel;
bool update_tuple_routing_needed = node->partColsUpdated;
@@ -2293,9 +2342,75 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
mtstate->operation = operation;
mtstate->canSetTag = node->canSetTag;
mtstate->mt_done = false;
+ mtstate->mt_whichplan = WHICHPLAN_CHOOSE_PARTITIONS;
+
+ /* If run-time partition pruning is enabled, then set that up now */
+ if (node->part_prune_info != NULL)
+ {
+ PartitionPruneState *prunestate;
+
+ ExecAssignExprContext(estate, &mtstate->ps);
+
+ prunestate = ExecCreatePartitionPruneState(&mtstate->ps,
+ node->part_prune_info);
+ mtstate->mt_prune_state = prunestate;
+
+ /* Perform an initial partition prune, if required. */
+ if (prunestate->do_initial_prune)
+ {
+ /* Determine which subplans match the external params */
+ validsubplans = ExecFindInitialMatchingSubPlans(prunestate,
+ list_length(node->plans));
+
+ /*
+ * The case where no subplans survive pruning must be handled
+ * specially. The problem here is that code in explain.c requires
+ * an Append to have at least one subplan in order for it to
+ * properly determine the Vars in that subplan's targetlist. We
+ * sidestep this issue by just initializing the first subplan and
+ * setting as_whichplan to NO_MATCHING_SUBPLANS to indicate that
+ * we don't really need to scan any subnodes.
+ */
+ if (bms_is_empty(validsubplans))
+ {
+ mtstate->mt_whichplan = WHICHPLAN_NO_MATCHING_PARTITIONS;
+
+ /* Mark the first as valid so that it's initialized below */
+ validsubplans = bms_make_singleton(0);
+ }
+
+ nplans = bms_num_members(validsubplans);
+ }
+ else
+ {
+ /* We'll need to initialize all subplans */
+ nplans = list_length(node->plans);
+ validsubplans = bms_add_range(NULL, 0, nplans - 1);
+ }
+
+ /*
+ * If no runtime pruning is required, we can fill mt_valid_subplans
+ * immediately, preventing later calls to ExecFindMatchingSubPlans.
+ */
+ if (!prunestate->do_exec_prune)
+ mtstate->mt_valid_subplans = bms_add_range(NULL, 0, nplans - 1);
+ }
+ else
+ {
+ nplans = list_length(node->plans);
+
+ /*
+ * When run-time partition pruning is not enabled we can just mark all
+ * plans as valid, they must also all be initialized.
+ */
+ validsubplans = bms_add_range(NULL, 0, nplans - 1);
+ mtstate->mt_prune_state = NULL;
+ }
+
mtstate->mt_plans = (PlanState **) palloc0(sizeof(PlanState *) * nplans);
- mtstate->resultRelInfo = estate->es_result_relations + node->resultRelIndex;
+ mtstate->resultRelInfos = (ResultRelInfo **)
+ palloc(sizeof(ResultRelInfo *) * nplans);
mtstate->mt_scans = (TupleTableSlot **) palloc0(sizeof(TupleTableSlot *) * nplans);
/* If modifying a partitioned table, initialize the root table info */
@@ -2322,11 +2437,18 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
*/
saved_resultRelInfo = estate->es_result_relation_info;
- resultRelInfo = mtstate->resultRelInfo;
- i = 0;
+ j = i = 0;
foreach(l, node->plans)
{
+ if (!bms_is_member(i, validsubplans))
+ {
+ i++;
+ continue;
+ }
+
subplan = (Plan *) lfirst(l);
+ resultRelInfo = estate->es_result_relations + node->resultRelIndex + i;
+ mtstate->resultRelInfos[j] = resultRelInfo;
/* Initialize the usesFdwDirectModify flag */
resultRelInfo->ri_usesFdwDirectModify = bms_is_member(i,
@@ -2364,9 +2486,9 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
/* Now init the plan for this result rel */
estate->es_result_relation_info = resultRelInfo;
- mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
- mtstate->mt_scans[i] =
- ExecInitExtraTupleSlot(mtstate->ps.state, ExecGetResultType(mtstate->mt_plans[i]),
+ mtstate->mt_plans[j] = ExecInitNode(subplan, estate, eflags);
+ mtstate->mt_scans[j] =
+ ExecInitExtraTupleSlot(mtstate->ps.state, ExecGetResultType(mtstate->mt_plans[j]),
table_slot_callbacks(resultRelInfo->ri_RelationDesc));
/* Also let FDWs init themselves for foreign-table result rels */
@@ -2382,9 +2504,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
i,
eflags);
}
-
- resultRelInfo++;
i++;
+ j++;
}
estate->es_result_relation_info = saved_resultRelInfo;
@@ -2433,14 +2554,21 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
/*
* Initialize any WITH CHECK OPTION constraints if needed.
*/
- resultRelInfo = mtstate->resultRelInfo;
- i = 0;
+ j = i = 0;
foreach(l, node->withCheckOptionLists)
{
- List *wcoList = (List *) lfirst(l);
+ List *wcoList;
List *wcoExprs = NIL;
ListCell *ll;
+ if (!bms_is_member(i, validsubplans))
+ {
+ i++;
+ continue;
+ }
+
+ wcoList = (List *) lfirst(l);
+
foreach(ll, wcoList)
{
WithCheckOption *wco = (WithCheckOption *) lfirst(ll);
@@ -2450,9 +2578,10 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
wcoExprs = lappend(wcoExprs, wcoExpr);
}
+ resultRelInfo = mtstate->resultRelInfos[j];
resultRelInfo->ri_WithCheckOptions = wcoList;
resultRelInfo->ri_WithCheckOptionExprs = wcoExprs;
- resultRelInfo++;
+ j++;
i++;
}
@@ -2482,16 +2611,25 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
/*
* Build a projection for each result rel.
*/
- resultRelInfo = mtstate->resultRelInfo;
+ j = i = 0;
foreach(l, node->returningLists)
{
- List *rlist = (List *) lfirst(l);
+ List *rlist;
+
+ if (!bms_is_member(i, validsubplans))
+ {
+ i++;
+ continue;
+ }
+ rlist = (List *) lfirst(l);
+
+ resultRelInfo = mtstate->resultRelInfos[j];
resultRelInfo->ri_returningList = rlist;
resultRelInfo->ri_projectReturning =
ExecBuildProjectionInfo(rlist, econtext, slot, &mtstate->ps,
resultRelInfo->ri_RelationDesc->rd_att);
- resultRelInfo++;
+ j++;
}
}
else
@@ -2502,12 +2640,10 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
*/
mtstate->ps.plan->targetlist = NIL;
ExecInitResultTypeTL(&mtstate->ps);
-
- mtstate->ps.ps_ExprContext = NULL;
}
/* Set the list of arbiter indexes if needed for ON CONFLICT */
- resultRelInfo = mtstate->resultRelInfo;
+ resultRelInfo = mtstate->resultRelInfos[0];
if (node->onConflictAction != ONCONFLICT_NONE)
resultRelInfo->ri_onConflictArbiterIndexes = node->arbiterIndexes;
@@ -2596,7 +2732,6 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
}
/* select first subplan */
- mtstate->mt_whichplan = 0;
subplan = (Plan *) linitial(node->plans);
EvalPlanQualSetPlan(&mtstate->mt_epqstate, subplan,
mtstate->mt_arowmarks[0]);
@@ -2645,12 +2780,12 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
if (junk_filter_needed)
{
- resultRelInfo = mtstate->resultRelInfo;
for (i = 0; i < nplans; i++)
{
JunkFilter *j;
TupleTableSlot *junkresslot;
+ resultRelInfo = mtstate->resultRelInfos[i];
subplan = mtstate->mt_plans[i]->plan;
if (operation == CMD_INSERT || operation == CMD_UPDATE)
ExecCheckPlanOutput(resultRelInfo->ri_RelationDesc,
@@ -2693,13 +2828,12 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
}
resultRelInfo->ri_junkFilter = j;
- resultRelInfo++;
}
}
else
{
if (operation == CMD_INSERT)
- ExecCheckPlanOutput(mtstate->resultRelInfo->ri_RelationDesc,
+ ExecCheckPlanOutput(mtstate->resultRelInfos[0]->ri_RelationDesc,
subplan->targetlist);
}
}
@@ -2738,7 +2872,7 @@ ExecEndModifyTable(ModifyTableState *node)
*/
for (i = 0; i < node->mt_nplans; i++)
{
- ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
+ ResultRelInfo *resultRelInfo = node->resultRelInfos[i];
if (!resultRelInfo->ri_usesFdwDirectModify &&
resultRelInfo->ri_FdwRoutine != NULL &&
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 78deade89b..be9fb8a69b 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -215,6 +215,7 @@ _copyModifyTable(const ModifyTable *from)
COPY_BITMAPSET_FIELD(fdwDirectModifyPlans);
COPY_NODE_FIELD(rowMarks);
COPY_SCALAR_FIELD(epqParam);
+ COPY_NODE_FIELD(part_prune_info);
COPY_SCALAR_FIELD(onConflictAction);
COPY_NODE_FIELD(arbiterIndexes);
COPY_NODE_FIELD(onConflictSet);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8400dd319e..58458320f7 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -416,6 +416,7 @@ _outModifyTable(StringInfo str, const ModifyTable *node)
WRITE_BITMAPSET_FIELD(fdwDirectModifyPlans);
WRITE_NODE_FIELD(rowMarks);
WRITE_INT_FIELD(epqParam);
+ WRITE_NODE_FIELD(part_prune_info);
WRITE_ENUM_FIELD(onConflictAction, OnConflictAction);
WRITE_NODE_FIELD(arbiterIndexes);
WRITE_NODE_FIELD(onConflictSet);
@@ -2085,6 +2086,7 @@ _outModifyTablePath(StringInfo str, const ModifyTablePath *node)
WRITE_UINT_FIELD(rootRelation);
WRITE_BOOL_FIELD(partColsUpdated);
WRITE_NODE_FIELD(resultRelations);
+ WRITE_NODE_FIELD(partitioned_rels);
WRITE_NODE_FIELD(subpaths);
WRITE_NODE_FIELD(subroots);
WRITE_NODE_FIELD(withCheckOptionLists);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 6c2626ee62..6c753bf4dc 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -1616,6 +1616,7 @@ _readModifyTable(void)
READ_BITMAPSET_FIELD(fdwDirectModifyPlans);
READ_NODE_FIELD(rowMarks);
READ_INT_FIELD(epqParam);
+ READ_NODE_FIELD(part_prune_info);
READ_ENUM_FIELD(onConflictAction, OnConflictAction);
READ_NODE_FIELD(arbiterIndexes);
READ_NODE_FIELD(onConflictSet);
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 608d5adfed..a23594018f 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -286,7 +286,8 @@ static ModifyTable *make_modifytable(PlannerInfo *root,
bool partColsUpdated,
List *resultRelations, List *subplans, List *subroots,
List *withCheckOptionLists, List *returningLists,
- List *rowMarks, OnConflictExpr *onconflict, int epqParam);
+ List *rowMarks, OnConflictExpr *onconflict, int epqParam,
+ PartitionPruneInfo *partpruneinfos);
static GatherMerge *create_gather_merge_plan(PlannerInfo *root,
GatherMergePath *best_path);
@@ -1234,6 +1235,7 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
make_partition_pruneinfo(root, rel,
best_path->subpaths,
best_path->partitioned_rels,
+ NIL,
prunequal);
}
@@ -1399,6 +1401,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
partpruneinfo = make_partition_pruneinfo(root, rel,
best_path->subpaths,
best_path->partitioned_rels,
+ NIL,
prunequal);
}
@@ -2579,6 +2582,7 @@ create_modifytable_plan(PlannerInfo *root, ModifyTablePath *best_path)
List *subplans = NIL;
ListCell *subpaths,
*subroots;
+ PartitionPruneInfo *partpruneinfos = NULL;
/* Build the plan for each input path */
forboth(subpaths, best_path->subpaths,
@@ -2607,6 +2611,30 @@ create_modifytable_plan(PlannerInfo *root, ModifyTablePath *best_path)
subplans = lappend(subplans, subplan);
}
+ if (enable_partition_pruning &&
+ best_path->partitioned_rels != NIL &&
+ !IS_DUMMY_MODIFYTABLE(best_path))
+ {
+ RelOptInfo *rel = best_path->path.parent;
+ List *prunequal = NIL;
+
+ prunequal = extract_actual_clauses(rel->baserestrictinfo, false);
+
+ /*
+ * If any quals exist, then these may be useful to allow us to perform
+ * further partition pruning during execution. We'll generate a
+ * PartitionPruneInfo for each partitioned rel to store these quals
+ * and allow translation of partition indexes into subpath indexes.
+ */
+ if (prunequal != NIL)
+ partpruneinfos = make_partition_pruneinfo(root,
+ best_path->path.parent,
+ best_path->subpaths,
+ best_path->partitioned_rels,
+ best_path->resultRelations,
+ prunequal);
+ }
+
plan = make_modifytable(root,
best_path->operation,
best_path->canSetTag,
@@ -2620,7 +2648,8 @@ create_modifytable_plan(PlannerInfo *root, ModifyTablePath *best_path)
best_path->returningLists,
best_path->rowMarks,
best_path->onconflict,
- best_path->epqParam);
+ best_path->epqParam,
+ partpruneinfos);
copy_generic_path_info(&plan->plan, &best_path->path);
@@ -6579,7 +6608,8 @@ make_modifytable(PlannerInfo *root,
bool partColsUpdated,
List *resultRelations, List *subplans, List *subroots,
List *withCheckOptionLists, List *returningLists,
- List *rowMarks, OnConflictExpr *onconflict, int epqParam)
+ List *rowMarks, OnConflictExpr *onconflict, int epqParam,
+ PartitionPruneInfo *partpruneinfos)
{
ModifyTable *node = makeNode(ModifyTable);
List *fdw_private_list;
@@ -6640,6 +6670,7 @@ make_modifytable(PlannerInfo *root,
node->returningLists = returningLists;
node->rowMarks = rowMarks;
node->epqParam = epqParam;
+ node->part_prune_info = partpruneinfos;
/*
* For each result relation that is a foreign table, allow the FDW to
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index cb897cc7f4..6d0a14a81b 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -1230,6 +1230,9 @@ inheritance_planner(PlannerInfo *root)
RangeTblEntry *parent_rte;
Bitmapset *parent_relids;
Query **parent_parses;
+ PlannerInfo *partition_root = NULL;
+ List *partitioned_rels = NIL;
+ bool dummy_update = false;
/* Should only get here for UPDATE or DELETE */
Assert(parse->commandType == CMD_UPDATE ||
@@ -1341,6 +1344,13 @@ inheritance_planner(PlannerInfo *root)
* expand_partitioned_rtentry for the UPDATE target.)
*/
root->partColsUpdated = subroot->partColsUpdated;
+
+ /*
+ * Save this for later so that we can enable run-time pruning on
+ * the partitioned table(s).
+ */
+ if (parent_rte->relkind == RELKIND_PARTITIONED_TABLE)
+ partition_root = subroot;
}
/*----------
@@ -1735,6 +1745,9 @@ inheritance_planner(PlannerInfo *root)
returningLists = list_make1(parse->returningList);
/* Disable tuple routing, too, just to be safe */
root->partColsUpdated = false;
+
+ /* Mark that we're performing a dummy update */
+ dummy_update = true;
}
else
{
@@ -1773,6 +1786,69 @@ inheritance_planner(PlannerInfo *root)
else
rowMarks = root->rowMarks;
+
+ /*
+ * When performing UPDATE/DELETE on a partitioned table, if the query has
+ * a WHERE clause which supports it, we may be able to perform run-time
+ * partition pruning. The following code sets things up to allow this to
+ * be possible.
+ */
+ if (partition_root && !dummy_update)
+ {
+ RelOptInfo *parent_rel;
+ int i;
+
+ /*
+ * Fetch the target partitioned table from the SELECT version of
+ * the query which we performed above. This may have the base quals
+ * which could allow the run-time pruning to work.
+ */
+ parent_rel = partition_root->simple_rel_array[top_parentRTindex];
+
+ final_rel->baserestrictinfo = parent_rel->baserestrictinfo;
+
+ /* build a list of partitioned rels */
+ i = -1;
+ while ((i = bms_next_member(parent_relids, i)) > 0)
+ partitioned_rels = lappend_int(partitioned_rels, i);
+
+
+ /*
+ * In order to build the run-time pruning data we'll need append rels
+ * any sub-partitioned tables. If there are some of those and the
+ * append_rel_array is not already allocated, then do that now.
+ */
+ if (list_length(partitioned_rels) > 1 &&
+ root->append_rel_array == NULL)
+ root->append_rel_array = palloc0(sizeof(AppendRelInfo *) *
+ root->simple_rel_array_size);
+
+ /*
+ * There can only be a single partition hierarchy, so it's fine to
+ * just make a single element list of the partitioned_rels.
+ */
+ partitioned_rels = list_make1(partitioned_rels);
+
+ i = -1;
+ while ((i = bms_next_member(parent_relids, i)) >= 0)
+ {
+ Assert(root->simple_rel_array[i] == NULL);
+
+ root->simple_rel_array[i] = partition_root->simple_rel_array[i];
+
+ /*
+ * The root partition won't have an append rel entry, so we can
+ * skip that. We'll need to take the partition_root's version for
+ * any sub-partitioned table's
+ */
+ if (i != top_parentRTindex)
+ {
+ Assert(root->append_rel_array[i] == NULL);
+ root->append_rel_array[i] = partition_root->append_rel_array[i];
+ }
+ }
+ }
+
/* Create Path representing a ModifyTable to do the UPDATE/DELETE work */
add_path(final_rel, (Path *)
create_modifytable_path(root, final_rel,
@@ -1782,6 +1858,7 @@ inheritance_planner(PlannerInfo *root)
rootRelation,
root->partColsUpdated,
resultRelations,
+ partitioned_rels,
subpaths,
subroots,
withCheckOptionLists,
@@ -2365,6 +2442,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
rootRelation,
false,
list_make1_int(parse->resultRelation),
+ NIL,
list_make1(path),
list_make1(root),
withCheckOptionLists,
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index d884d2bb00..0e929a183c 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3456,6 +3456,9 @@ create_lockrows_path(PlannerInfo *root, RelOptInfo *rel,
* 'partColsUpdated' is true if any partitioning columns are being updated,
* either from the target relation or a descendent partitioned table.
* 'resultRelations' is an integer list of actual RT indexes of target rel(s)
+ * 'partitioned_rels' is an integer list of RT indexes of non-leaf tables in
+ * the partition tree, if this is an UPDATE/DELETE to a partitioned table.
+ * Otherwise NIL.
* 'subpaths' is a list of Path(s) producing source data (one per rel)
* 'subroots' is a list of PlannerInfo structs (one per rel)
* 'withCheckOptionLists' is a list of WCO lists (one per rel)
@@ -3469,8 +3472,8 @@ create_modifytable_path(PlannerInfo *root, RelOptInfo *rel,
CmdType operation, bool canSetTag,
Index nominalRelation, Index rootRelation,
bool partColsUpdated,
- List *resultRelations, List *subpaths,
- List *subroots,
+ List *resultRelations, List *partitioned_rels,
+ List *subpaths, List *subroots,
List *withCheckOptionLists, List *returningLists,
List *rowMarks, OnConflictExpr *onconflict,
int epqParam)
@@ -3538,6 +3541,7 @@ create_modifytable_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->rootRelation = rootRelation;
pathnode->partColsUpdated = partColsUpdated;
pathnode->resultRelations = resultRelations;
+ pathnode->partitioned_rels = list_copy(partitioned_rels);
pathnode->subpaths = subpaths;
pathnode->subroots = subroots;
pathnode->withCheckOptionLists = withCheckOptionLists;
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index 5982af4de1..42c2725b25 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -210,7 +210,10 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
* 'parentrel' is the RelOptInfo for an appendrel, and 'subpaths' is the list
* of scan paths for its child rels.
*
- * 'partitioned_rels' is a List containing Lists of relids of partitioned
+ * If 'resultRelations' is non-NIL, then this List of relids is used to build
+ * the mapping structures. Otherwise the 'subpaths' List is used.
+ *
+ * 'partitioned_rels' is a List containing Lists of relids of partitioned
* tables (a/k/a non-leaf partitions) that are parents of some of the child
* rels. Here we attempt to populate the PartitionPruneInfo by adding a
* 'prune_infos' item for each sublist in the 'partitioned_rels' list.
@@ -225,6 +228,7 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
PartitionPruneInfo *
make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
List *subpaths, List *partitioned_rels,
+ List *resultRelations,
List *prunequal)
{
PartitionPruneInfo *pruneinfo;
@@ -242,23 +246,36 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
relid_subplan_map = palloc0(sizeof(int) * root->simple_rel_array_size);
/*
- * relid_subplan_map maps relid of a leaf partition to the index in
- * 'subpaths' of the scan plan for that partition.
+ * If 'resultRelations' are present then map these, otherwise we map the
+ * 'subpaths' List.
*/
- i = 1;
- foreach(lc, subpaths)
+ if (resultRelations != NIL)
{
- Path *path = (Path *) lfirst(lc);
- RelOptInfo *pathrel = path->parent;
-
- Assert(IS_SIMPLE_REL(pathrel));
- Assert(pathrel->relid < root->simple_rel_array_size);
- /* No duplicates please */
- Assert(relid_subplan_map[pathrel->relid] == 0);
+ i = 1;
+ foreach(lc, resultRelations)
+ {
+ int resultrel = lfirst_int(lc);
- relid_subplan_map[pathrel->relid] = i++;
+ Assert(resultrel < root->simple_rel_array_size);
+ relid_subplan_map[resultrel] = i++;
+ }
}
+ else
+ {
+ i = 1;
+ foreach(lc, subpaths)
+ {
+ Path *path = (Path *)lfirst(lc);
+ RelOptInfo *pathrel = path->parent;
+ Assert(IS_SIMPLE_REL(pathrel));
+ Assert(pathrel->relid < root->simple_rel_array_size);
+ /* No duplicates please */
+ Assert(relid_subplan_map[pathrel->relid] == 0);
+
+ relid_subplan_map[pathrel->relid] = i++;
+ }
+ }
/* We now build a PartitionedRelPruneInfo for each partitioned rel. */
prunerelinfos = NIL;
foreach(lc, partitioned_rels)
@@ -400,9 +417,12 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
* an adjust_appendrel_attrs step. But it might not be, and then
* we have to translate. We update the prunequal parameter here,
* because in later iterations of the loop for child partitions,
- * we want to translate from parent to child variables.
+ * we want to translate from parent to child variables. We don't
+ * need to do this when planning a non-SELECT as we're only
+ * working with a single partition hierarchy in that case.
*/
- if (!bms_equal(parentrel->relids, subpart->relids))
+ if (root->parse->commandType == CMD_SELECT &&
+ !bms_equal(parentrel->relids, subpart->relids))
{
int nappinfos;
AppendRelInfo **appinfos = find_appinfos_by_relids(root,
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 99b9fa414f..de4f3689ee 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1115,7 +1115,7 @@ typedef struct ModifyTableState
int mt_whichplan; /* which one is being executed (0..n-1) */
TupleTableSlot **mt_scans; /* input tuple corresponding to underlying
* plans */
- ResultRelInfo *resultRelInfo; /* per-subplan target relations */
+ ResultRelInfo **resultRelInfos; /* per-subplan target relations */
ResultRelInfo *rootResultRelInfo; /* root target relation (partitioned
* table root) */
List **mt_arowmarks; /* per-subplan ExecAuxRowMark lists */
@@ -1140,6 +1140,14 @@ typedef struct ModifyTableState
/* Per plan map for tuple conversion from child to root */
TupleConversionMap **mt_per_subplan_tupconv_maps;
+
+ /*
+ * Details required to allow partitions to be eliminated from the scan, or
+ * NULL if not possible.
+ */
+ struct PartitionPruneState *mt_prune_state;
+ Bitmapset *mt_valid_subplans; /* for runtime pruning, valid mt_plans
+ * indexes to scan. */
} ModifyTableState;
/* ----------------
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 441e64eca9..e877ad35bc 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -1376,6 +1376,10 @@ typedef struct AppendPath
#define IS_DUMMY_APPEND(p) \
(IsA((p), AppendPath) && ((AppendPath *) (p))->subpaths == NIL)
+#define IS_DUMMY_MODIFYTABLE(p) \
+ (list_length((p)->subpaths) == 1 && \
+ IS_DUMMY_APPEND(linitial((p)->subpaths)))
+
/*
* A relation that's been proven empty will have one path that is dummy
* (but might have projection paths on top). For historical reasons,
@@ -1771,6 +1775,8 @@ typedef struct ModifyTablePath
Index rootRelation; /* Root RT index, if target is partitioned */
bool partColsUpdated; /* some part key in hierarchy updated */
List *resultRelations; /* integer list of RT indexes */
+ /* RT indexes of non-leaf tables in a partition tree */
+ List *partitioned_rels;
List *subpaths; /* Path(s) producing source data */
List *subroots; /* per-target-table PlannerInfos */
List *withCheckOptionLists; /* per-target-table WCO lists */
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 70f8b8e22b..aa43f12e8a 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -231,6 +231,8 @@ typedef struct ModifyTable
Bitmapset *fdwDirectModifyPlans; /* indices of FDW DM plans */
List *rowMarks; /* PlanRowMarks (non-locking only) */
int epqParam; /* ID of Param for EvalPlanQual re-eval */
+ /* Mapping details for run-time subplan pruning, one per partitioned_rels */
+ struct PartitionPruneInfo *part_prune_info;
OnConflictAction onConflictAction; /* ON CONFLICT action */
List *arbiterIndexes; /* List of ON CONFLICT arbiter index OIDs */
List *onConflictSet; /* SET for INSERT ON CONFLICT DO UPDATE */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index e70d6a3f18..f0e06bd27d 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -254,10 +254,12 @@ extern LockRowsPath *create_lockrows_path(PlannerInfo *root, RelOptInfo *rel,
extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
RelOptInfo *rel,
CmdType operation, bool canSetTag,
- Index nominalRelation, Index rootRelation,
+ Index nominalRelation,
+ Index rootRelation,
bool partColsUpdated,
- List *resultRelations, List *subpaths,
- List *subroots,
+ List *resultRelations,
+ List *partitioned_rels,
+ List *subpaths, List *subroots,
List *withCheckOptionLists, List *returningLists,
List *rowMarks, OnConflictExpr *onconflict,
int epqParam);
diff --git a/src/include/partitioning/partprune.h b/src/include/partitioning/partprune.h
index 81318c785a..8412efe359 100644
--- a/src/include/partitioning/partprune.h
+++ b/src/include/partitioning/partprune.h
@@ -72,6 +72,7 @@ extern PartitionPruneInfo *make_partition_pruneinfo(struct PlannerInfo *root,
struct RelOptInfo *parentrel,
List *subpaths,
List *partitioned_rels,
+ List *resultRelations,
List *prunequal);
extern Bitmapset *prune_append_rel_partitions(struct RelOptInfo *rel);
extern Bitmapset *get_matching_partitions(PartitionPruneContext *context,
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 841bd8bc67..eed84464b4 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -3358,6 +3358,111 @@ explain (analyze, costs off, summary off, timing off) select * from ma_test wher
reset enable_seqscan;
reset enable_sort;
+--
+-- Test run-time pruning of ModifyTable subnodes
+--
+-- Ensure only ma_test_p3 is scanned.
+explain (analyze, costs off, summary off, timing off) delete from ma_test where a = (select 29);
+ QUERY PLAN
+------------------------------------------------------
+ Delete on ma_test (actual rows=0 loops=1)
+ Delete on ma_test_p1
+ Delete on ma_test_p2
+ Delete on ma_test_p3
+ InitPlan 1 (returns $0)
+ -> Result (actual rows=1 loops=1)
+ -> Seq Scan on ma_test_p1 (never executed)
+ Filter: (a = $0)
+ -> Seq Scan on ma_test_p2 (never executed)
+ Filter: (a = $0)
+ -> Seq Scan on ma_test_p3 (actual rows=1 loops=1)
+ Filter: (a = $0)
+ Rows Removed by Filter: 9
+(13 rows)
+
+-- Ensure no partitions are scanned.
+explain (analyze, costs off, summary off, timing off) delete from ma_test where a = (select 30);
+ QUERY PLAN
+-----------------------------------------------
+ Delete on ma_test (actual rows=0 loops=1)
+ Delete on ma_test_p1
+ Delete on ma_test_p2
+ Delete on ma_test_p3
+ InitPlan 1 (returns $0)
+ -> Result (actual rows=1 loops=1)
+ -> Seq Scan on ma_test_p1 (never executed)
+ Filter: (a = $0)
+ -> Seq Scan on ma_test_p2 (never executed)
+ Filter: (a = $0)
+ -> Seq Scan on ma_test_p3 (never executed)
+ Filter: (a = $0)
+(12 rows)
+
+-- Ensure partition pruning works with an update of the partition key.
+explain (analyze, costs off, summary off, timing off) update ma_test set a = 29 where a = (select 1);
+ QUERY PLAN
+------------------------------------------------------
+ Update on ma_test (actual rows=0 loops=1)
+ Update on ma_test_p1
+ Update on ma_test_p2
+ Update on ma_test_p3
+ InitPlan 1 (returns $0)
+ -> Result (actual rows=1 loops=1)
+ -> Seq Scan on ma_test_p1 (actual rows=1 loops=1)
+ Filter: (a = $0)
+ Rows Removed by Filter: 9
+ -> Seq Scan on ma_test_p2 (never executed)
+ Filter: (a = $0)
+ -> Seq Scan on ma_test_p3 (never executed)
+ Filter: (a = $0)
+(13 rows)
+
+-- Verify the above command
+select tableoid::regclass,a from ma_test where a = 29;
+ tableoid | a
+------------+----
+ ma_test_p3 | 29
+(1 row)
+
+truncate ma_test;
+prepare mt_q1 (int) as
+delete from ma_test where a > $1;
+set plan_cache_mode = force_generic_plan;
+explain (analyze, costs off, summary off, timing off) execute mt_q1(15);
+ QUERY PLAN
+------------------------------------------------------
+ Delete on ma_test (actual rows=0 loops=1)
+ Delete on ma_test_p2
+ Delete on ma_test_p3
+ Subplans Removed: 1
+ -> Seq Scan on ma_test_p2 (actual rows=0 loops=1)
+ Filter: (a > $1)
+ -> Seq Scan on ma_test_p3 (actual rows=0 loops=1)
+ Filter: (a > $1)
+(8 rows)
+
+explain (analyze, costs off, summary off, timing off) execute mt_q1(25);
+ QUERY PLAN
+------------------------------------------------------
+ Delete on ma_test (actual rows=0 loops=1)
+ Delete on ma_test_p3
+ Subplans Removed: 2
+ -> Seq Scan on ma_test_p3 (actual rows=0 loops=1)
+ Filter: (a > $1)
+(5 rows)
+
+-- Ensure ModifyTable behaves correctly when no subplans match exec params
+explain (analyze, costs off, summary off, timing off) execute mt_q1(35);
+ QUERY PLAN
+-----------------------------------------------
+ Delete on ma_test (actual rows=0 loops=1)
+ Delete on ma_test_p1
+ Subplans Removed: 2
+ -> Seq Scan on ma_test_p1 (never executed)
+ Filter: (a > $1)
+(5 rows)
+
+reset plan_cache_mode;
drop table ma_test;
reset enable_indexonlyscan;
--
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index 071e28dce8..465721fe8b 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -884,6 +884,36 @@ explain (analyze, costs off, summary off, timing off) select * from ma_test wher
reset enable_seqscan;
reset enable_sort;
+--
+-- Test run-time pruning of ModifyTable subnodes
+--
+
+-- Ensure only ma_test_p3 is scanned.
+explain (analyze, costs off, summary off, timing off) delete from ma_test where a = (select 29);
+
+-- Ensure no partitions are scanned.
+explain (analyze, costs off, summary off, timing off) delete from ma_test where a = (select 30);
+
+-- Ensure partition pruning works with an update of the partition key.
+explain (analyze, costs off, summary off, timing off) update ma_test set a = 29 where a = (select 1);
+
+-- Verify the above command
+select tableoid::regclass,a from ma_test where a = 29;
+
+truncate ma_test;
+
+prepare mt_q1 (int) as
+delete from ma_test where a > $1;
+
+set plan_cache_mode = force_generic_plan;
+
+explain (analyze, costs off, summary off, timing off) execute mt_q1(15);
+explain (analyze, costs off, summary off, timing off) execute mt_q1(25);
+-- Ensure ModifyTable behaves correctly when no subplans match exec params
+explain (analyze, costs off, summary off, timing off) execute mt_q1(35);
+
+reset plan_cache_mode;
+
drop table ma_test;
reset enable_indexonlyscan;
Hi David,
On Thu, Jun 27, 2019 at 2:28 PM David Rowley
<david.rowley@2ndquadrant.com> wrote:
Deja vu from this time last year when despite everyone's best efforts
(mostly Alvaro) we missed getting run-time pruning in for MergeAppend
into the PG11 release. This year it was ModifyTable, which is now
possible thanks to Amit and Tom's modifications to the inheritance
planner.I've attached what I have so far for this.
Thanks for working on this. IIUC, the feature is to skip modifying a
given result relation if run-time pruning dictates that none of its
existing rows will match some dynamically computable quals.
I think it's mostly okay,
but my brain was overheating a bit at the inheritance_planner changes.
I think we need to consider the fact that there is a proposal [1]/messages/by-id/357.1550612935@sss.pgh.pa.us to
get rid of inheritance_planner() as the way of planning UPDATE/DELETEs
on inheritance trees. If we go that route, then a given partitioned
target table will be expanded at the bottom and so, there's no need
for ModifyTable to have its own run-time pruning info, because
Append/MergeAppend will have it. Maybe, we will need some code in
ExecInitModifyTable() and ExecModifyTable() to handle the case where
run-time pruning, during plan tree initialization and plan tree
execution respectively, may have rendered modifying a given result
relation unnecessary.
A cursory look at the patch suggests that most of its changes will be
for nothing if [1]/messages/by-id/357.1550612935@sss.pgh.pa.us materializes. What do you think about that?
Thanks,
Amit
On Wed, 3 Jul 2019 at 17:27, Amit Langote <amitlangote09@gmail.com> wrote:
A cursory look at the patch suggests that most of its changes will be
for nothing if [1] materializes. What do you think about that?
Yeah, I had this in mind when writing the patch, but kept going
anyway. I think it's only really a small patch of this patch that
would get wiped out with that change. Just the planner.c stuff.
Everything else is still required, as far as I understand.
--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On Wed, Jul 3, 2019 at 4:34 PM David Rowley
<david.rowley@2ndquadrant.com> wrote:
On Wed, 3 Jul 2019 at 17:27, Amit Langote <amitlangote09@gmail.com> wrote:
A cursory look at the patch suggests that most of its changes will be
for nothing if [1] materializes. What do you think about that?Yeah, I had this in mind when writing the patch, but kept going
anyway. I think it's only really a small patch of this patch that
would get wiped out with that change. Just the planner.c stuff.
Everything else is still required, as far as I understand.
If I understand the details of [1] correctly, ModifyTable will no
longer have N subplans for N result relations as there are today. So,
it doesn't make sense for ModifyTable to contain
PartitionedRelPruneInfos and for ExecInitModifyTable/ExecModifyTable
to have to perform initial and execution-time pruning, respectively.
As I said, bottom expansion of target inheritance will mean pruning
(both plan-time and run-time) will occur at the bottom too, so the
run-time pruning capabilities of nodes that already have it will be
used for UPDATE and DELETE too.
Thanks,
Amit
Hi, Amit
If I understand the details of [1] correctly, ModifyTable will no longer
have N subplans for N result relations as there are today. So, it doesn't
make sense for ModifyTable to contain PartitionedRelPruneInfos and for
ExecInitModifyTable/ExecModifyTable
to have to perform initial and execution-time pruning, respectively.
Does this mean that the generic plan will not have N subplans for N result relations?
I thought [1] would make creating generic plans faster, but is this correct?
regards,
kato sho
Show quoted text
-----Original Message-----
From: Amit Langote [mailto:amitlangote09@gmail.com]
Sent: Wednesday, July 3, 2019 5:41 PM
To: David Rowley <david.rowley@2ndquadrant.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: Run-time pruning for ModifyTableOn Wed, Jul 3, 2019 at 4:34 PM David Rowley <david.rowley@2ndquadrant.com>
wrote:On Wed, 3 Jul 2019 at 17:27, Amit Langote <amitlangote09@gmail.com>
wrote:
A cursory look at the patch suggests that most of its changes will
be for nothing if [1] materializes. What do you think about that?Yeah, I had this in mind when writing the patch, but kept going
anyway. I think it's only really a small patch of this patch that
would get wiped out with that change. Just the planner.c stuff.
Everything else is still required, as far as I understand.If I understand the details of [1] correctly, ModifyTable will no longer
have N subplans for N result relations as there are today. So, it doesn't
make sense for ModifyTable to contain PartitionedRelPruneInfos and for
ExecInitModifyTable/ExecModifyTable
to have to perform initial and execution-time pruning, respectively.
As I said, bottom expansion of target inheritance will mean pruning (both
plan-time and run-time) will occur at the bottom too, so the run-time
pruning capabilities of nodes that already have it will be used for UPDATE
and DELETE too.Thanks,
Amit
Kato-san,
On Thu, Jul 4, 2019 at 1:40 PM Kato, Sho <kato-sho@jp.fujitsu.com> wrote:
If I understand the details of [1] correctly, ModifyTable will no longer
have N subplans for N result relations as there are today. So, it doesn't
make sense for ModifyTable to contain PartitionedRelPruneInfos and for
ExecInitModifyTable/ExecModifyTable
to have to perform initial and execution-time pruning, respectively.Does this mean that the generic plan will not have N subplans for N result relations?
I thought [1] would make creating generic plans faster, but is this correct?
Yeah, making a generic plan for UPDATE of inheritance tables will
certainly become faster, because we will no longer plan the same query
N times for N child tables. There will still be N result relations
but only one sub-plan to fetch the rows from. Also, planning will
still cost O(N), but with a much smaller constant factor.
By the way, let's keep any further discussion on this particular topic
in the other thread.
Thanks,
Amit
On Monday, July 8, 2019 11:34 AM, Amit Langote wrote:
By the way, let's keep any further discussion on this particular topic
in the other thread.
Thanks for the details. I got it.
Regards,
Kato sho
Show quoted text
-----Original Message-----
From: Amit Langote [mailto:amitlangote09@gmail.com]
Sent: Monday, July 8, 2019 11:34 AM
To: Kato, Sho/加藤 翔 <kato-sho@jp.fujitsu.com>
Cc: David Rowley <david.rowley@2ndquadrant.com>; PostgreSQL Hackers
<pgsql-hackers@lists.postgresql.org>
Subject: Re: Run-time pruning for ModifyTableKato-san,
On Thu, Jul 4, 2019 at 1:40 PM Kato, Sho <kato-sho@jp.fujitsu.com> wrote:
If I understand the details of [1] correctly, ModifyTable will no
longer have N subplans for N result relations as there are today.
So, it doesn't make sense for ModifyTable to contain
PartitionedRelPruneInfos and forExecInitModifyTable/ExecModifyTable
to have to perform initial and execution-time pruning, respectively.
Does this mean that the generic plan will not have N subplans for N
result relations?
I thought [1] would make creating generic plans faster, but is this
correct?
Yeah, making a generic plan for UPDATE of inheritance tables will
certainly become faster, because we will no longer plan the same query
N times for N child tables. There will still be N result relations but
only one sub-plan to fetch the rows from. Also, planning will still cost
O(N), but with a much smaller constant factor.By the way, let's keep any further discussion on this particular topic
in the other thread.Thanks,
Amit
Here's a rebased version of this patch (it had a trivial conflict).
No further changes.
--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
runtime_pruning_for_modifytable.patchtext/x-diff; charset=us-asciiDownload
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 82d8140ba2..677b9cdd58 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -1951,7 +1951,7 @@ postgresBeginForeignInsert(ModifyTableState *mtstate,
if (plan && plan->operation == CMD_UPDATE &&
(resultRelInfo->ri_usesFdwDirectModify ||
resultRelInfo->ri_FdwState) &&
- resultRelInfo > mtstate->resultRelInfo + mtstate->mt_whichplan)
+ resultRelInfo > mtstate->resultRelInfos[mtstate->mt_whichplan])
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot route tuples into foreign table to be updated \"%s\"",
@@ -2005,7 +2005,7 @@ postgresBeginForeignInsert(ModifyTableState *mtstate,
*/
if (plan && plan->operation == CMD_UPDATE &&
resultRelation == plan->rootRelation)
- resultRelation = mtstate->resultRelInfo[0].ri_RangeTableIndex;
+ resultRelation = mtstate->resultRelInfos[0]->ri_RangeTableIndex;
}
/* Construct the SQL command string. */
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index 3aeef30b28..2e9954ddda 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2856,7 +2856,7 @@ CopyFrom(CopyState cstate)
mtstate->ps.plan = NULL;
mtstate->ps.state = estate;
mtstate->operation = CMD_INSERT;
- mtstate->resultRelInfo = estate->es_result_relations;
+ mtstate->resultRelInfos = &estate->es_result_relations;
if (resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignInsert != NULL)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 62fb3434a3..682bf615db 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -3190,14 +3190,14 @@ show_modifytable_info(ModifyTableState *mtstate, List *ancestors,
/* Should we explicitly label target relations? */
labeltargets = (mtstate->mt_nplans > 1 ||
(mtstate->mt_nplans == 1 &&
- mtstate->resultRelInfo->ri_RangeTableIndex != node->nominalRelation));
+ mtstate->resultRelInfos[0]->ri_RangeTableIndex != node->nominalRelation));
if (labeltargets)
ExplainOpenGroup("Target Tables", "Target Tables", false, es);
for (j = 0; j < mtstate->mt_nplans; j++)
{
- ResultRelInfo *resultRelInfo = mtstate->resultRelInfo + j;
+ ResultRelInfo *resultRelInfo = mtstate->resultRelInfos[j];
FdwRoutine *fdwroutine = resultRelInfo->ri_FdwRoutine;
if (labeltargets)
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index d23f292cb0..8d1d961809 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -471,7 +471,7 @@ ExecHashSubPlanResultRelsByOid(ModifyTableState *mtstate,
/* Hash all subplans by their Oid */
for (i = 0; i < mtstate->mt_nplans; i++)
{
- ResultRelInfo *rri = &mtstate->resultRelInfo[i];
+ ResultRelInfo *rri = mtstate->resultRelInfos[i];
bool found;
Oid partoid = RelationGetRelid(rri->ri_RelationDesc);
SubplanResultRelHashElem *elem;
@@ -508,7 +508,7 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
ModifyTable *node = (ModifyTable *) mtstate->ps.plan;
Relation rootrel = rootResultRelInfo->ri_RelationDesc,
partrel;
- Relation firstResultRel = mtstate->resultRelInfo[0].ri_RelationDesc;
+ Relation firstResultRel = mtstate->resultRelInfos[0]->ri_RelationDesc;
ResultRelInfo *leaf_part_rri;
MemoryContext oldcxt;
AttrNumber *part_attnos = NULL;
@@ -556,7 +556,7 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
List *wcoList;
List *wcoExprs = NIL;
ListCell *ll;
- int firstVarno = mtstate->resultRelInfo[0].ri_RangeTableIndex;
+ int firstVarno = mtstate->resultRelInfos[0]->ri_RangeTableIndex;
/*
* In the case of INSERT on a partitioned table, there is only one
@@ -621,7 +621,7 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
TupleTableSlot *slot;
ExprContext *econtext;
List *returningList;
- int firstVarno = mtstate->resultRelInfo[0].ri_RangeTableIndex;
+ int firstVarno = mtstate->resultRelInfos[0]->ri_RangeTableIndex;
/* See the comment above for WCO lists. */
Assert((node->operation == CMD_INSERT &&
@@ -681,7 +681,7 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
*/
if (node && node->onConflictAction != ONCONFLICT_NONE)
{
- int firstVarno = mtstate->resultRelInfo[0].ri_RangeTableIndex;
+ int firstVarno = mtstate->resultRelInfos[0]->ri_RangeTableIndex;
TupleDesc partrelDesc = RelationGetDescr(partrel);
ExprContext *econtext = mtstate->ps.ps_ExprContext;
ListCell *lc;
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index c9d024ead5..f35291cb12 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -57,6 +57,9 @@
#include "utils/memutils.h"
#include "utils/rel.h"
+ /* Special values for mt_whichplan */
+#define WHICHPLAN_CHOOSE_PARTITIONS -1
+#define WHICHPLAN_NO_MATCHING_PARTITIONS -2
static bool ExecOnConflictUpdate(ModifyTableState *mtstate,
ResultRelInfo *resultRelInfo,
@@ -1261,7 +1264,7 @@ lreplace:;
* retrieve the one for this resultRel, we need to know the
* position of the resultRel in mtstate->resultRelInfo[].
*/
- map_index = resultRelInfo - mtstate->resultRelInfo;
+ map_index = mtstate->mt_whichplan;
Assert(map_index >= 0 && map_index < mtstate->mt_nplans);
tupconv_map = tupconv_map_for_subplan(mtstate, map_index);
if (tupconv_map != NULL)
@@ -1707,12 +1710,12 @@ static void
fireBSTriggers(ModifyTableState *node)
{
ModifyTable *plan = (ModifyTable *) node->ps.plan;
- ResultRelInfo *resultRelInfo = node->resultRelInfo;
+ ResultRelInfo *resultRelInfo = node->resultRelInfos[0];
/*
* If the node modifies a partitioned table, we must fire its triggers.
- * Note that in that case, node->resultRelInfo points to the first leaf
- * partition, not the root table.
+ * Note that in that case, node->resultRelInfos[0] points to the first
+ * leaf partition, not the root table.
*/
if (node->rootResultRelInfo != NULL)
resultRelInfo = node->rootResultRelInfo;
@@ -1750,13 +1753,14 @@ static ResultRelInfo *
getTargetResultRelInfo(ModifyTableState *node)
{
/*
- * Note that if the node modifies a partitioned table, node->resultRelInfo
- * points to the first leaf partition, not the root table.
+ * Note that if the node modifies a partitioned table,
+ * node->resultRelInfos[0] points to the first leaf partition, not the
+ * root table.
*/
if (node->rootResultRelInfo != NULL)
return node->rootResultRelInfo;
else
- return node->resultRelInfo;
+ return node->resultRelInfos[0];
}
/*
@@ -1935,7 +1939,7 @@ static void
ExecSetupChildParentMapForSubplan(ModifyTableState *mtstate)
{
ResultRelInfo *targetRelInfo = getTargetResultRelInfo(mtstate);
- ResultRelInfo *resultRelInfos = mtstate->resultRelInfo;
+ ResultRelInfo **resultRelInfos = mtstate->resultRelInfos;
TupleDesc outdesc;
int numResultRelInfos = mtstate->mt_nplans;
int i;
@@ -1955,7 +1959,7 @@ ExecSetupChildParentMapForSubplan(ModifyTableState *mtstate)
for (i = 0; i < numResultRelInfos; ++i)
{
mtstate->mt_per_subplan_tupconv_maps[i] =
- convert_tuples_by_name(RelationGetDescr(resultRelInfos[i].ri_RelationDesc),
+ convert_tuples_by_name(RelationGetDescr(resultRelInfos[i]->ri_RelationDesc),
outdesc);
}
}
@@ -2031,8 +2035,47 @@ ExecModifyTable(PlanState *pstate)
node->fireBSTriggers = false;
}
+ if (node->mt_whichplan < 0)
+ {
+ /* Handle choosing the valid partitions */
+ if (node->mt_whichplan == WHICHPLAN_CHOOSE_PARTITIONS)
+ {
+ PartitionPruneState *prunestate = node->mt_prune_state;
+
+ /* There should always be at least one */
+ Assert(node->mt_nplans > 0);
+
+ /*
+ * When partition pruning is enabled and exec params match the
+ * partition key then determine the minimum set of matching
+ * subnodes. Otherwise we match to all subnodes.
+ */
+ if (prunestate != NULL && prunestate->do_exec_prune)
+ {
+ node->mt_valid_subplans = ExecFindMatchingSubPlans(prunestate);
+ node->mt_whichplan = bms_next_member(node->mt_valid_subplans, -1);
+
+ /* If no subplan matches these params then we're done */
+ if (node->mt_whichplan < 0)
+ goto done;
+ }
+ else
+ {
+ node->mt_valid_subplans = bms_add_range(NULL, 0,
+ node->mt_nplans - 1);
+ node->mt_whichplan = 0;
+ }
+ }
+
+ /* partition pruning determined that no partitions match */
+ else if (node->mt_whichplan == WHICHPLAN_NO_MATCHING_PARTITIONS)
+ goto done;
+ else
+ elog(ERROR, "invalid subplan index: %d", node->mt_whichplan);
+ }
+
/* Preload local variables */
- resultRelInfo = node->resultRelInfo + node->mt_whichplan;
+ resultRelInfo = node->resultRelInfos[node->mt_whichplan];
subplanstate = node->mt_plans[node->mt_whichplan];
junkfilter = resultRelInfo->ri_junkFilter;
@@ -2074,10 +2117,12 @@ ExecModifyTable(PlanState *pstate)
if (TupIsNull(planSlot))
{
/* advance to next subplan if any */
- node->mt_whichplan++;
- if (node->mt_whichplan < node->mt_nplans)
+ node->mt_whichplan = bms_next_member(node->mt_valid_subplans,
+ node->mt_whichplan);
+
+ if (node->mt_whichplan >= 0)
{
- resultRelInfo++;
+ resultRelInfo = node->resultRelInfos[node->mt_whichplan];
subplanstate = node->mt_plans[node->mt_whichplan];
junkfilter = resultRelInfo->ri_junkFilter;
estate->es_result_relation_info = resultRelInfo;
@@ -2247,6 +2292,8 @@ ExecModifyTable(PlanState *pstate)
/* Restore es_result_relation_info before exiting */
estate->es_result_relation_info = saved_resultRelInfo;
+done:
+
/*
* We're done, but fire AFTER STATEMENT triggers before exiting.
*/
@@ -2269,9 +2316,11 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
int nplans = list_length(node->plans);
ResultRelInfo *saved_resultRelInfo;
ResultRelInfo *resultRelInfo;
+ Bitmapset *validsubplans;
Plan *subplan;
ListCell *l;
- int i;
+ int i,
+ j;
Relation rel;
bool update_tuple_routing_needed = node->partColsUpdated;
@@ -2289,9 +2338,75 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
mtstate->operation = operation;
mtstate->canSetTag = node->canSetTag;
mtstate->mt_done = false;
+ mtstate->mt_whichplan = WHICHPLAN_CHOOSE_PARTITIONS;
+
+ /* If run-time partition pruning is enabled, then set that up now */
+ if (node->part_prune_info != NULL)
+ {
+ PartitionPruneState *prunestate;
+
+ ExecAssignExprContext(estate, &mtstate->ps);
+
+ prunestate = ExecCreatePartitionPruneState(&mtstate->ps,
+ node->part_prune_info);
+ mtstate->mt_prune_state = prunestate;
+
+ /* Perform an initial partition prune, if required. */
+ if (prunestate->do_initial_prune)
+ {
+ /* Determine which subplans match the external params */
+ validsubplans = ExecFindInitialMatchingSubPlans(prunestate,
+ list_length(node->plans));
+
+ /*
+ * The case where no subplans survive pruning must be handled
+ * specially. The problem here is that code in explain.c requires
+ * an Append to have at least one subplan in order for it to
+ * properly determine the Vars in that subplan's targetlist. We
+ * sidestep this issue by just initializing the first subplan and
+ * setting as_whichplan to NO_MATCHING_SUBPLANS to indicate that
+ * we don't really need to scan any subnodes.
+ */
+ if (bms_is_empty(validsubplans))
+ {
+ mtstate->mt_whichplan = WHICHPLAN_NO_MATCHING_PARTITIONS;
+
+ /* Mark the first as valid so that it's initialized below */
+ validsubplans = bms_make_singleton(0);
+ }
+
+ nplans = bms_num_members(validsubplans);
+ }
+ else
+ {
+ /* We'll need to initialize all subplans */
+ nplans = list_length(node->plans);
+ validsubplans = bms_add_range(NULL, 0, nplans - 1);
+ }
+
+ /*
+ * If no runtime pruning is required, we can fill mt_valid_subplans
+ * immediately, preventing later calls to ExecFindMatchingSubPlans.
+ */
+ if (!prunestate->do_exec_prune)
+ mtstate->mt_valid_subplans = bms_add_range(NULL, 0, nplans - 1);
+ }
+ else
+ {
+ nplans = list_length(node->plans);
+
+ /*
+ * When run-time partition pruning is not enabled we can just mark all
+ * plans as valid, they must also all be initialized.
+ */
+ validsubplans = bms_add_range(NULL, 0, nplans - 1);
+ mtstate->mt_prune_state = NULL;
+ }
+
mtstate->mt_plans = (PlanState **) palloc0(sizeof(PlanState *) * nplans);
- mtstate->resultRelInfo = estate->es_result_relations + node->resultRelIndex;
+ mtstate->resultRelInfos = (ResultRelInfo **)
+ palloc(sizeof(ResultRelInfo *) * nplans);
mtstate->mt_scans = (TupleTableSlot **) palloc0(sizeof(TupleTableSlot *) * nplans);
/* If modifying a partitioned table, initialize the root table info */
@@ -2318,11 +2433,18 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
*/
saved_resultRelInfo = estate->es_result_relation_info;
- resultRelInfo = mtstate->resultRelInfo;
- i = 0;
+ j = i = 0;
foreach(l, node->plans)
{
+ if (!bms_is_member(i, validsubplans))
+ {
+ i++;
+ continue;
+ }
+
subplan = (Plan *) lfirst(l);
+ resultRelInfo = estate->es_result_relations + node->resultRelIndex + i;
+ mtstate->resultRelInfos[j] = resultRelInfo;
/* Initialize the usesFdwDirectModify flag */
resultRelInfo->ri_usesFdwDirectModify = bms_is_member(i,
@@ -2360,9 +2482,9 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
/* Now init the plan for this result rel */
estate->es_result_relation_info = resultRelInfo;
- mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
- mtstate->mt_scans[i] =
- ExecInitExtraTupleSlot(mtstate->ps.state, ExecGetResultType(mtstate->mt_plans[i]),
+ mtstate->mt_plans[j] = ExecInitNode(subplan, estate, eflags);
+ mtstate->mt_scans[j] =
+ ExecInitExtraTupleSlot(mtstate->ps.state, ExecGetResultType(mtstate->mt_plans[j]),
table_slot_callbacks(resultRelInfo->ri_RelationDesc));
/* Also let FDWs init themselves for foreign-table result rels */
@@ -2378,9 +2500,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
i,
eflags);
}
-
- resultRelInfo++;
i++;
+ j++;
}
estate->es_result_relation_info = saved_resultRelInfo;
@@ -2429,14 +2550,21 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
/*
* Initialize any WITH CHECK OPTION constraints if needed.
*/
- resultRelInfo = mtstate->resultRelInfo;
- i = 0;
+ j = i = 0;
foreach(l, node->withCheckOptionLists)
{
- List *wcoList = (List *) lfirst(l);
+ List *wcoList;
List *wcoExprs = NIL;
ListCell *ll;
+ if (!bms_is_member(i, validsubplans))
+ {
+ i++;
+ continue;
+ }
+
+ wcoList = (List *) lfirst(l);
+
foreach(ll, wcoList)
{
WithCheckOption *wco = (WithCheckOption *) lfirst(ll);
@@ -2446,9 +2574,10 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
wcoExprs = lappend(wcoExprs, wcoExpr);
}
+ resultRelInfo = mtstate->resultRelInfos[j];
resultRelInfo->ri_WithCheckOptions = wcoList;
resultRelInfo->ri_WithCheckOptionExprs = wcoExprs;
- resultRelInfo++;
+ j++;
i++;
}
@@ -2478,16 +2607,25 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
/*
* Build a projection for each result rel.
*/
- resultRelInfo = mtstate->resultRelInfo;
+ j = i = 0;
foreach(l, node->returningLists)
{
- List *rlist = (List *) lfirst(l);
+ List *rlist;
+ if (!bms_is_member(i, validsubplans))
+ {
+ i++;
+ continue;
+ }
+
+ rlist = (List *) lfirst(l);
+
+ resultRelInfo = mtstate->resultRelInfos[j];
resultRelInfo->ri_returningList = rlist;
resultRelInfo->ri_projectReturning =
ExecBuildProjectionInfo(rlist, econtext, slot, &mtstate->ps,
resultRelInfo->ri_RelationDesc->rd_att);
- resultRelInfo++;
+ j++;
}
}
else
@@ -2498,12 +2636,10 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
*/
mtstate->ps.plan->targetlist = NIL;
ExecInitResultTypeTL(&mtstate->ps);
-
- mtstate->ps.ps_ExprContext = NULL;
}
/* Set the list of arbiter indexes if needed for ON CONFLICT */
- resultRelInfo = mtstate->resultRelInfo;
+ resultRelInfo = mtstate->resultRelInfos[0];
if (node->onConflictAction != ONCONFLICT_NONE)
resultRelInfo->ri_onConflictArbiterIndexes = node->arbiterIndexes;
@@ -2597,7 +2733,6 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
}
/* select first subplan */
- mtstate->mt_whichplan = 0;
subplan = (Plan *) linitial(node->plans);
EvalPlanQualSetPlan(&mtstate->mt_epqstate, subplan,
mtstate->mt_arowmarks[0]);
@@ -2646,12 +2781,12 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
if (junk_filter_needed)
{
- resultRelInfo = mtstate->resultRelInfo;
for (i = 0; i < nplans; i++)
{
JunkFilter *j;
TupleTableSlot *junkresslot;
+ resultRelInfo = mtstate->resultRelInfos[i];
subplan = mtstate->mt_plans[i]->plan;
if (operation == CMD_INSERT || operation == CMD_UPDATE)
ExecCheckPlanOutput(resultRelInfo->ri_RelationDesc,
@@ -2694,13 +2829,12 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
}
resultRelInfo->ri_junkFilter = j;
- resultRelInfo++;
}
}
else
{
if (operation == CMD_INSERT)
- ExecCheckPlanOutput(mtstate->resultRelInfo->ri_RelationDesc,
+ ExecCheckPlanOutput(mtstate->resultRelInfos[0]->ri_RelationDesc,
subplan->targetlist);
}
}
@@ -2739,7 +2873,7 @@ ExecEndModifyTable(ModifyTableState *node)
*/
for (i = 0; i < node->mt_nplans; i++)
{
- ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
+ ResultRelInfo *resultRelInfo = node->resultRelInfos[i];
if (!resultRelInfo->ri_usesFdwDirectModify &&
resultRelInfo->ri_FdwRoutine != NULL &&
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 3432bb921d..d3a2fed402 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -215,6 +215,7 @@ _copyModifyTable(const ModifyTable *from)
COPY_BITMAPSET_FIELD(fdwDirectModifyPlans);
COPY_NODE_FIELD(rowMarks);
COPY_SCALAR_FIELD(epqParam);
+ COPY_NODE_FIELD(part_prune_info);
COPY_SCALAR_FIELD(onConflictAction);
COPY_NODE_FIELD(arbiterIndexes);
COPY_NODE_FIELD(onConflictSet);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index b0dcd02ff6..612d41c4fb 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -416,6 +416,7 @@ _outModifyTable(StringInfo str, const ModifyTable *node)
WRITE_BITMAPSET_FIELD(fdwDirectModifyPlans);
WRITE_NODE_FIELD(rowMarks);
WRITE_INT_FIELD(epqParam);
+ WRITE_NODE_FIELD(part_prune_info);
WRITE_ENUM_FIELD(onConflictAction, OnConflictAction);
WRITE_NODE_FIELD(arbiterIndexes);
WRITE_NODE_FIELD(onConflictSet);
@@ -2089,6 +2090,7 @@ _outModifyTablePath(StringInfo str, const ModifyTablePath *node)
WRITE_UINT_FIELD(rootRelation);
WRITE_BOOL_FIELD(partColsUpdated);
WRITE_NODE_FIELD(resultRelations);
+ WRITE_NODE_FIELD(partitioned_rels);
WRITE_NODE_FIELD(subpaths);
WRITE_NODE_FIELD(subroots);
WRITE_NODE_FIELD(withCheckOptionLists);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 764e3bb90c..571587c114 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -1616,6 +1616,7 @@ _readModifyTable(void)
READ_BITMAPSET_FIELD(fdwDirectModifyPlans);
READ_NODE_FIELD(rowMarks);
READ_INT_FIELD(epqParam);
+ READ_NODE_FIELD(part_prune_info);
READ_ENUM_FIELD(onConflictAction, OnConflictAction);
READ_NODE_FIELD(arbiterIndexes);
READ_NODE_FIELD(onConflictSet);
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 0c036209f0..9262f39673 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -289,7 +289,8 @@ static ModifyTable *make_modifytable(PlannerInfo *root,
bool partColsUpdated,
List *resultRelations, List *subplans, List *subroots,
List *withCheckOptionLists, List *returningLists,
- List *rowMarks, OnConflictExpr *onconflict, int epqParam);
+ List *rowMarks, OnConflictExpr *onconflict, int epqParam,
+ PartitionPruneInfo *partpruneinfos);
static GatherMerge *create_gather_merge_plan(PlannerInfo *root,
GatherMergePath *best_path);
@@ -1237,6 +1238,7 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
make_partition_pruneinfo(root, rel,
best_path->subpaths,
best_path->partitioned_rels,
+ NIL,
prunequal);
}
@@ -1402,6 +1404,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
partpruneinfo = make_partition_pruneinfo(root, rel,
best_path->subpaths,
best_path->partitioned_rels,
+ NIL,
prunequal);
}
@@ -2581,6 +2584,7 @@ create_modifytable_plan(PlannerInfo *root, ModifyTablePath *best_path)
List *subplans = NIL;
ListCell *subpaths,
*subroots;
+ PartitionPruneInfo *partpruneinfos = NULL;
/* Build the plan for each input path */
forboth(subpaths, best_path->subpaths,
@@ -2609,6 +2613,30 @@ create_modifytable_plan(PlannerInfo *root, ModifyTablePath *best_path)
subplans = lappend(subplans, subplan);
}
+ if (enable_partition_pruning &&
+ best_path->partitioned_rels != NIL &&
+ !IS_DUMMY_MODIFYTABLE(best_path))
+ {
+ RelOptInfo *rel = best_path->path.parent;
+ List *prunequal = NIL;
+
+ prunequal = extract_actual_clauses(rel->baserestrictinfo, false);
+
+ /*
+ * If any quals exist, then these may be useful to allow us to perform
+ * further partition pruning during execution. We'll generate a
+ * PartitionPruneInfo for each partitioned rel to store these quals
+ * and allow translation of partition indexes into subpath indexes.
+ */
+ if (prunequal != NIL)
+ partpruneinfos = make_partition_pruneinfo(root,
+ best_path->path.parent,
+ best_path->subpaths,
+ best_path->partitioned_rels,
+ best_path->resultRelations,
+ prunequal);
+ }
+
plan = make_modifytable(root,
best_path->operation,
best_path->canSetTag,
@@ -2622,7 +2650,8 @@ create_modifytable_plan(PlannerInfo *root, ModifyTablePath *best_path)
best_path->returningLists,
best_path->rowMarks,
best_path->onconflict,
- best_path->epqParam);
+ best_path->epqParam,
+ partpruneinfos);
copy_generic_path_info(&plan->plan, &best_path->path);
@@ -6616,7 +6645,8 @@ make_modifytable(PlannerInfo *root,
bool partColsUpdated,
List *resultRelations, List *subplans, List *subroots,
List *withCheckOptionLists, List *returningLists,
- List *rowMarks, OnConflictExpr *onconflict, int epqParam)
+ List *rowMarks, OnConflictExpr *onconflict, int epqParam,
+ PartitionPruneInfo *partpruneinfos)
{
ModifyTable *node = makeNode(ModifyTable);
List *fdw_private_list;
@@ -6677,6 +6707,7 @@ make_modifytable(PlannerInfo *root,
node->returningLists = returningLists;
node->rowMarks = rowMarks;
node->epqParam = epqParam;
+ node->part_prune_info = partpruneinfos;
/*
* For each result relation that is a foreign table, allow the FDW to
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 17c5f086fb..bab7175d48 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -1236,6 +1236,9 @@ inheritance_planner(PlannerInfo *root)
RangeTblEntry *parent_rte;
Bitmapset *parent_relids;
Query **parent_parses;
+ PlannerInfo *partition_root = NULL;
+ List *partitioned_rels = NIL;
+ bool dummy_update = false;
/* Should only get here for UPDATE or DELETE */
Assert(parse->commandType == CMD_UPDATE ||
@@ -1347,6 +1350,13 @@ inheritance_planner(PlannerInfo *root)
* expand_partitioned_rtentry for the UPDATE target.)
*/
root->partColsUpdated = subroot->partColsUpdated;
+
+ /*
+ * Save this for later so that we can enable run-time pruning on
+ * the partitioned table(s).
+ */
+ if (parent_rte->relkind == RELKIND_PARTITIONED_TABLE)
+ partition_root = subroot;
}
/*----------
@@ -1741,6 +1751,9 @@ inheritance_planner(PlannerInfo *root)
returningLists = list_make1(parse->returningList);
/* Disable tuple routing, too, just to be safe */
root->partColsUpdated = false;
+
+ /* Mark that we're performing a dummy update */
+ dummy_update = true;
}
else
{
@@ -1779,6 +1792,69 @@ inheritance_planner(PlannerInfo *root)
else
rowMarks = root->rowMarks;
+
+ /*
+ * When performing UPDATE/DELETE on a partitioned table, if the query has
+ * a WHERE clause which supports it, we may be able to perform run-time
+ * partition pruning. The following code sets things up to allow this to
+ * be possible.
+ */
+ if (partition_root && !dummy_update)
+ {
+ RelOptInfo *parent_rel;
+ int i;
+
+ /*
+ * Fetch the target partitioned table from the SELECT version of
+ * the query which we performed above. This may have the base quals
+ * which could allow the run-time pruning to work.
+ */
+ parent_rel = partition_root->simple_rel_array[top_parentRTindex];
+
+ final_rel->baserestrictinfo = parent_rel->baserestrictinfo;
+
+ /* build a list of partitioned rels */
+ i = -1;
+ while ((i = bms_next_member(parent_relids, i)) > 0)
+ partitioned_rels = lappend_int(partitioned_rels, i);
+
+
+ /*
+ * In order to build the run-time pruning data we'll need append rels
+ * any sub-partitioned tables. If there are some of those and the
+ * append_rel_array is not already allocated, then do that now.
+ */
+ if (list_length(partitioned_rels) > 1 &&
+ root->append_rel_array == NULL)
+ root->append_rel_array = palloc0(sizeof(AppendRelInfo *) *
+ root->simple_rel_array_size);
+
+ /*
+ * There can only be a single partition hierarchy, so it's fine to
+ * just make a single element list of the partitioned_rels.
+ */
+ partitioned_rels = list_make1(partitioned_rels);
+
+ i = -1;
+ while ((i = bms_next_member(parent_relids, i)) >= 0)
+ {
+ Assert(root->simple_rel_array[i] == NULL);
+
+ root->simple_rel_array[i] = partition_root->simple_rel_array[i];
+
+ /*
+ * The root partition won't have an append rel entry, so we can
+ * skip that. We'll need to take the partition_root's version for
+ * any sub-partitioned table's
+ */
+ if (i != top_parentRTindex)
+ {
+ Assert(root->append_rel_array[i] == NULL);
+ root->append_rel_array[i] = partition_root->append_rel_array[i];
+ }
+ }
+ }
+
/* Create Path representing a ModifyTable to do the UPDATE/DELETE work */
add_path(final_rel, (Path *)
create_modifytable_path(root, final_rel,
@@ -1788,6 +1864,7 @@ inheritance_planner(PlannerInfo *root)
rootRelation,
root->partColsUpdated,
resultRelations,
+ partitioned_rels,
subpaths,
subroots,
withCheckOptionLists,
@@ -2371,6 +2448,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
rootRelation,
false,
list_make1_int(parse->resultRelation),
+ NIL,
list_make1(path),
list_make1(root),
withCheckOptionLists,
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index 34acb732ee..c1ed3abe9a 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3432,6 +3432,9 @@ create_lockrows_path(PlannerInfo *root, RelOptInfo *rel,
* 'partColsUpdated' is true if any partitioning columns are being updated,
* either from the target relation or a descendent partitioned table.
* 'resultRelations' is an integer list of actual RT indexes of target rel(s)
+ * 'partitioned_rels' is an integer list of RT indexes of non-leaf tables in
+ * the partition tree, if this is an UPDATE/DELETE to a partitioned table.
+ * Otherwise NIL.
* 'subpaths' is a list of Path(s) producing source data (one per rel)
* 'subroots' is a list of PlannerInfo structs (one per rel)
* 'withCheckOptionLists' is a list of WCO lists (one per rel)
@@ -3445,8 +3448,8 @@ create_modifytable_path(PlannerInfo *root, RelOptInfo *rel,
CmdType operation, bool canSetTag,
Index nominalRelation, Index rootRelation,
bool partColsUpdated,
- List *resultRelations, List *subpaths,
- List *subroots,
+ List *resultRelations, List *partitioned_rels,
+ List *subpaths, List *subroots,
List *withCheckOptionLists, List *returningLists,
List *rowMarks, OnConflictExpr *onconflict,
int epqParam)
@@ -3514,6 +3517,7 @@ create_modifytable_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->rootRelation = rootRelation;
pathnode->partColsUpdated = partColsUpdated;
pathnode->resultRelations = resultRelations;
+ pathnode->partitioned_rels = list_copy(partitioned_rels);
pathnode->subpaths = subpaths;
pathnode->subroots = subroots;
pathnode->withCheckOptionLists = withCheckOptionLists;
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index 29f1c11b24..97f3145d16 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -214,7 +214,10 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
* 'parentrel' is the RelOptInfo for an appendrel, and 'subpaths' is the list
* of scan paths for its child rels.
*
- * 'partitioned_rels' is a List containing Lists of relids of partitioned
+ * If 'resultRelations' is non-NIL, then this List of relids is used to build
+ * the mapping structures. Otherwise the 'subpaths' List is used.
+ *
+ * 'partitioned_rels' is a List containing Lists of relids of partitioned
* tables (a/k/a non-leaf partitions) that are parents of some of the child
* rels. Here we attempt to populate the PartitionPruneInfo by adding a
* 'prune_infos' item for each sublist in the 'partitioned_rels' list.
@@ -229,6 +232,7 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
PartitionPruneInfo *
make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
List *subpaths, List *partitioned_rels,
+ List *resultRelations,
List *prunequal)
{
PartitionPruneInfo *pruneinfo;
@@ -246,23 +250,36 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
relid_subplan_map = palloc0(sizeof(int) * root->simple_rel_array_size);
/*
- * relid_subplan_map maps relid of a leaf partition to the index in
- * 'subpaths' of the scan plan for that partition.
+ * If 'resultRelations' are present then map these, otherwise we map the
+ * 'subpaths' List.
*/
- i = 1;
- foreach(lc, subpaths)
+ if (resultRelations != NIL)
{
- Path *path = (Path *) lfirst(lc);
- RelOptInfo *pathrel = path->parent;
+ i = 1;
+ foreach(lc, resultRelations)
+ {
+ int resultrel = lfirst_int(lc);
- Assert(IS_SIMPLE_REL(pathrel));
- Assert(pathrel->relid < root->simple_rel_array_size);
- /* No duplicates please */
- Assert(relid_subplan_map[pathrel->relid] == 0);
-
- relid_subplan_map[pathrel->relid] = i++;
+ Assert(resultrel < root->simple_rel_array_size);
+ relid_subplan_map[resultrel] = i++;
+ }
}
+ else
+ {
+ i = 1;
+ foreach(lc, subpaths)
+ {
+ Path *path = (Path *)lfirst(lc);
+ RelOptInfo *pathrel = path->parent;
+ Assert(IS_SIMPLE_REL(pathrel));
+ Assert(pathrel->relid < root->simple_rel_array_size);
+ /* No duplicates please */
+ Assert(relid_subplan_map[pathrel->relid] == 0);
+
+ relid_subplan_map[pathrel->relid] = i++;
+ }
+ }
/* We now build a PartitionedRelPruneInfo for each partitioned rel. */
prunerelinfos = NIL;
foreach(lc, partitioned_rels)
@@ -404,9 +421,12 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
* an adjust_appendrel_attrs step. But it might not be, and then
* we have to translate. We update the prunequal parameter here,
* because in later iterations of the loop for child partitions,
- * we want to translate from parent to child variables.
+ * we want to translate from parent to child variables. We don't
+ * need to do this when planning a non-SELECT as we're only
+ * working with a single partition hierarchy in that case.
*/
- if (!bms_equal(parentrel->relids, subpart->relids))
+ if (root->parse->commandType == CMD_SELECT &&
+ !bms_equal(parentrel->relids, subpart->relids))
{
int nappinfos;
AppendRelInfo **appinfos = find_appinfos_by_relids(root,
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index b593d22c48..d058716ff8 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1166,7 +1166,7 @@ typedef struct ModifyTableState
int mt_whichplan; /* which one is being executed (0..n-1) */
TupleTableSlot **mt_scans; /* input tuple corresponding to underlying
* plans */
- ResultRelInfo *resultRelInfo; /* per-subplan target relations */
+ ResultRelInfo **resultRelInfos; /* per-subplan target relations */
ResultRelInfo *rootResultRelInfo; /* root target relation (partitioned
* table root) */
List **mt_arowmarks; /* per-subplan ExecAuxRowMark lists */
@@ -1191,6 +1191,14 @@ typedef struct ModifyTableState
/* Per plan map for tuple conversion from child to root */
TupleConversionMap **mt_per_subplan_tupconv_maps;
+
+ /*
+ * Details required to allow partitions to be eliminated from the scan, or
+ * NULL if not possible.
+ */
+ struct PartitionPruneState *mt_prune_state;
+ Bitmapset *mt_valid_subplans; /* for runtime pruning, valid mt_plans
+ * indexes to scan. */
} ModifyTableState;
/* ----------------
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 23a06d718e..37e32166de 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -1380,6 +1380,10 @@ typedef struct AppendPath
#define IS_DUMMY_APPEND(p) \
(IsA((p), AppendPath) && ((AppendPath *) (p))->subpaths == NIL)
+#define IS_DUMMY_MODIFYTABLE(p) \
+ (list_length((p)->subpaths) == 1 && \
+ IS_DUMMY_APPEND(linitial((p)->subpaths)))
+
/*
* A relation that's been proven empty will have one path that is dummy
* (but might have projection paths on top). For historical reasons,
@@ -1775,6 +1779,8 @@ typedef struct ModifyTablePath
Index rootRelation; /* Root RT index, if target is partitioned */
bool partColsUpdated; /* some part key in hierarchy updated */
List *resultRelations; /* integer list of RT indexes */
+ /* RT indexes of non-leaf tables in a partition tree */
+ List *partitioned_rels;
List *subpaths; /* Path(s) producing source data */
List *subroots; /* per-target-table PlannerInfos */
List *withCheckOptionLists; /* per-target-table WCO lists */
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 8e6594e355..444ed14e7c 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -231,6 +231,8 @@ typedef struct ModifyTable
Bitmapset *fdwDirectModifyPlans; /* indices of FDW DM plans */
List *rowMarks; /* PlanRowMarks (non-locking only) */
int epqParam; /* ID of Param for EvalPlanQual re-eval */
+ /* Mapping details for run-time subplan pruning, one per partitioned_rels */
+ struct PartitionPruneInfo *part_prune_info;
OnConflictAction onConflictAction; /* ON CONFLICT action */
List *arbiterIndexes; /* List of ON CONFLICT arbiter index OIDs */
List *onConflictSet; /* SET for INSERT ON CONFLICT DO UPDATE */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index a12af54971..c8f5a165d6 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -252,10 +252,12 @@ extern LockRowsPath *create_lockrows_path(PlannerInfo *root, RelOptInfo *rel,
extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
RelOptInfo *rel,
CmdType operation, bool canSetTag,
- Index nominalRelation, Index rootRelation,
+ Index nominalRelation,
+ Index rootRelation,
bool partColsUpdated,
- List *resultRelations, List *subpaths,
- List *subroots,
+ List *resultRelations,
+ List *partitioned_rels,
+ List *subpaths, List *subroots,
List *withCheckOptionLists, List *returningLists,
List *rowMarks, OnConflictExpr *onconflict,
int epqParam);
diff --git a/src/include/partitioning/partprune.h b/src/include/partitioning/partprune.h
index b3e926865a..a98566f004 100644
--- a/src/include/partitioning/partprune.h
+++ b/src/include/partitioning/partprune.h
@@ -72,6 +72,7 @@ extern PartitionPruneInfo *make_partition_pruneinfo(struct PlannerInfo *root,
struct RelOptInfo *parentrel,
List *subpaths,
List *partitioned_rels,
+ List *resultRelations,
List *prunequal);
extern Bitmapset *prune_append_rel_partitions(struct RelOptInfo *rel);
extern Bitmapset *get_matching_partitions(PartitionPruneContext *context,
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 82b68e793d..ccb72d915a 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -3459,6 +3459,111 @@ explain (analyze, costs off, summary off, timing off) select * from ma_test wher
reset enable_seqscan;
reset enable_sort;
+--
+-- Test run-time pruning of ModifyTable subnodes
+--
+-- Ensure only ma_test_p3 is scanned.
+explain (analyze, costs off, summary off, timing off) delete from ma_test where a = (select 29);
+ QUERY PLAN
+------------------------------------------------------
+ Delete on ma_test (actual rows=0 loops=1)
+ Delete on ma_test_p1
+ Delete on ma_test_p2
+ Delete on ma_test_p3
+ InitPlan 1 (returns $0)
+ -> Result (actual rows=1 loops=1)
+ -> Seq Scan on ma_test_p1 (never executed)
+ Filter: (a = $0)
+ -> Seq Scan on ma_test_p2 (never executed)
+ Filter: (a = $0)
+ -> Seq Scan on ma_test_p3 (actual rows=1 loops=1)
+ Filter: (a = $0)
+ Rows Removed by Filter: 9
+(13 rows)
+
+-- Ensure no partitions are scanned.
+explain (analyze, costs off, summary off, timing off) delete from ma_test where a = (select 30);
+ QUERY PLAN
+-----------------------------------------------
+ Delete on ma_test (actual rows=0 loops=1)
+ Delete on ma_test_p1
+ Delete on ma_test_p2
+ Delete on ma_test_p3
+ InitPlan 1 (returns $0)
+ -> Result (actual rows=1 loops=1)
+ -> Seq Scan on ma_test_p1 (never executed)
+ Filter: (a = $0)
+ -> Seq Scan on ma_test_p2 (never executed)
+ Filter: (a = $0)
+ -> Seq Scan on ma_test_p3 (never executed)
+ Filter: (a = $0)
+(12 rows)
+
+-- Ensure partition pruning works with an update of the partition key.
+explain (analyze, costs off, summary off, timing off) update ma_test set a = 29 where a = (select 1);
+ QUERY PLAN
+------------------------------------------------------
+ Update on ma_test (actual rows=0 loops=1)
+ Update on ma_test_p1
+ Update on ma_test_p2
+ Update on ma_test_p3
+ InitPlan 1 (returns $0)
+ -> Result (actual rows=1 loops=1)
+ -> Seq Scan on ma_test_p1 (actual rows=1 loops=1)
+ Filter: (a = $0)
+ Rows Removed by Filter: 9
+ -> Seq Scan on ma_test_p2 (never executed)
+ Filter: (a = $0)
+ -> Seq Scan on ma_test_p3 (never executed)
+ Filter: (a = $0)
+(13 rows)
+
+-- Verify the above command
+select tableoid::regclass,a from ma_test where a = 29;
+ tableoid | a
+------------+----
+ ma_test_p3 | 29
+(1 row)
+
+truncate ma_test;
+prepare mt_q1 (int) as
+delete from ma_test where a > $1;
+set plan_cache_mode = force_generic_plan;
+explain (analyze, costs off, summary off, timing off) execute mt_q1(15);
+ QUERY PLAN
+------------------------------------------------------
+ Delete on ma_test (actual rows=0 loops=1)
+ Delete on ma_test_p2
+ Delete on ma_test_p3
+ Subplans Removed: 1
+ -> Seq Scan on ma_test_p2 (actual rows=0 loops=1)
+ Filter: (a > $1)
+ -> Seq Scan on ma_test_p3 (actual rows=0 loops=1)
+ Filter: (a > $1)
+(8 rows)
+
+explain (analyze, costs off, summary off, timing off) execute mt_q1(25);
+ QUERY PLAN
+------------------------------------------------------
+ Delete on ma_test (actual rows=0 loops=1)
+ Delete on ma_test_p3
+ Subplans Removed: 2
+ -> Seq Scan on ma_test_p3 (actual rows=0 loops=1)
+ Filter: (a > $1)
+(5 rows)
+
+-- Ensure ModifyTable behaves correctly when no subplans match exec params
+explain (analyze, costs off, summary off, timing off) execute mt_q1(35);
+ QUERY PLAN
+-----------------------------------------------
+ Delete on ma_test (actual rows=0 loops=1)
+ Delete on ma_test_p1
+ Subplans Removed: 2
+ -> Seq Scan on ma_test_p1 (never executed)
+ Filter: (a > $1)
+(5 rows)
+
+reset plan_cache_mode;
drop table ma_test;
reset enable_indexonlyscan;
--
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index 446af3b2c1..2c2a08e6f6 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -908,6 +908,36 @@ explain (analyze, costs off, summary off, timing off) select * from ma_test wher
reset enable_seqscan;
reset enable_sort;
+--
+-- Test run-time pruning of ModifyTable subnodes
+--
+
+-- Ensure only ma_test_p3 is scanned.
+explain (analyze, costs off, summary off, timing off) delete from ma_test where a = (select 29);
+
+-- Ensure no partitions are scanned.
+explain (analyze, costs off, summary off, timing off) delete from ma_test where a = (select 30);
+
+-- Ensure partition pruning works with an update of the partition key.
+explain (analyze, costs off, summary off, timing off) update ma_test set a = 29 where a = (select 1);
+
+-- Verify the above command
+select tableoid::regclass,a from ma_test where a = 29;
+
+truncate ma_test;
+
+prepare mt_q1 (int) as
+delete from ma_test where a > $1;
+
+set plan_cache_mode = force_generic_plan;
+
+explain (analyze, costs off, summary off, timing off) execute mt_q1(15);
+explain (analyze, costs off, summary off, timing off) execute mt_q1(25);
+-- Ensure ModifyTable behaves correctly when no subplans match exec params
+explain (analyze, costs off, summary off, timing off) execute mt_q1(35);
+
+reset plan_cache_mode;
+
drop table ma_test;
reset enable_indexonlyscan;
On Thu, Sep 12, 2019 at 10:10 AM Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:
Here's a rebased version of this patch (it had a trivial conflict).
Hi, FYI partition_prune.sql currently fails (maybe something to do
with commit d52eaa09?):
where s.a = $1 and s.b = $2 and s.c = (select 1);
explain (costs off) execute q (1, 1);
QUERY PLAN
----------------------------------------------------------------
+----------------------------------------------------
Append
InitPlan 1 (returns $0)
-> Result
- Subplans Removed: 1
-> Seq Scan on p1
- Filter: ((a = $1) AND (b = $2) AND (c = $0))
+ Filter: ((a = 1) AND (b = 1) AND (c = $0))
-> Seq Scan on q111
- Filter: ((a = $1) AND (b = $2) AND (c = $0))
+ Filter: ((a = 1) AND (b = 1) AND (c = $0))
-> Result
- One-Time Filter: ((1 = $1) AND (1 = $2) AND (1 = $0))
-(10 rows)
+ One-Time Filter: (1 = $0)
+(9 rows)
execute q (1, 1);
a | b | c
On Tue, Nov 05, 2019 at 04:04:25PM +1300, Thomas Munro wrote:
On Thu, Sep 12, 2019 at 10:10 AM Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:Here's a rebased version of this patch (it had a trivial conflict).
Hi, FYI partition_prune.sql currently fails (maybe something to do
with commit d52eaa09?):
David, perhaps you did not notice that? For now I have moved this
patch to next CF waiting on author to look after the failure.
Amit, Kato-san, both of you are marked as reviewers of this patch.
Are you planning to look at it?
--
Michael
On Wed, Nov 27, 2019 at 05:17:06PM +0900, Michael Paquier wrote:
On Tue, Nov 05, 2019 at 04:04:25PM +1300, Thomas Munro wrote:
On Thu, Sep 12, 2019 at 10:10 AM Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:Here's a rebased version of this patch (it had a trivial conflict).
Hi, FYI partition_prune.sql currently fails (maybe something to do
with commit d52eaa09?):David, perhaps you did not notice that? For now I have moved this
patch to next CF waiting on author to look after the failure.Amit, Kato-san, both of you are marked as reviewers of this patch.
Are you planning to look at it?
David, this patch is marked as "waiting on author" since 11/27, and
there have been no updates or responses since then. Do you plan to
submit a new patch version in this CF? We're already half-way through,
so there's not much time ...
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Thu, Jan 16, 2020 at 10:45:25PM +0100, Tomas Vondra wrote:
David, this patch is marked as "waiting on author" since 11/27, and
there have been no updates or responses since then. Do you plan to
submit a new patch version in this CF? We're already half-way through,
so there's not much time ...
The reason why I moved it to 2020-01 is that there was not enough time
for David to reply back. At this stage, it seems more appropriate to
me to mark it as returned with feedback and move on.
--
Michael
Sorry, I didn't notice this email until now.
On Wed, Nov 27, 2019 at 5:17 PM Michael Paquier <michael@paquier.xyz> wrote:
On Tue, Nov 05, 2019 at 04:04:25PM +1300, Thomas Munro wrote:
On Thu, Sep 12, 2019 at 10:10 AM Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:Here's a rebased version of this patch (it had a trivial conflict).
Hi, FYI partition_prune.sql currently fails (maybe something to do
with commit d52eaa09?):David, perhaps you did not notice that? For now I have moved this
patch to next CF waiting on author to look after the failure.Amit, Kato-san, both of you are marked as reviewers of this patch.
Are you planning to look at it?
Sorry, I never managed to look at the patch closely. As I commented
up-thread, the functionality added by this patch would be unnecessary
if we were to move forward with the other project related to UPDATE
and DELETE over inheritance trees:
/messages/by-id/357.1550612935@sss.pgh.pa.us
I had volunteered to submit a patch in that thread and even managed to
write one but didn't get time to get it in good enough shape to post
it to the list, like I couldn't make it handle foreign child tables.
The gist of the new approach is that ModifyTable will always have
*one* subplan under ModifyTable, not N for N target partitions as
currently. That one subplan being the same plan as one would get if
the query were SELECT instead of UPDATE/DELETE, it would automatically
take care of run-time pruning if needed, freeing ModifyTable itself
from having to do it.
Now, the chances of such a big overhaul of how UPDATEs of inheritance
trees are handled getting into PG 13 seem pretty thin even if I post
the patch in few days, so perhaps it would make sense to get this
patch in so that we can give users run-time pruning for UPDATE/DELETE
in PG 13, provided the code is not very invasive. If and when the
aforesaid overhaul takes place, that code would go away along with a
lot of other code.
Thanks,
Amit
On Thu, Jan 23, 2020 at 4:31 PM Amit Langote <amitlangote09@gmail.com> wrote:
Now, the chances of such a big overhaul of how UPDATEs of inheritance
trees are handled getting into PG 13 seem pretty thin even if I post
the patch in few days, so perhaps it would make sense to get this
patch in so that we can give users run-time pruning for UPDATE/DELETE
in PG 13, provided the code is not very invasive. If and when the
aforesaid overhaul takes place, that code would go away along with a
lot of other code.
Fwiw, I updated the patch, mainly expected/partition_prune.out. Some
tests in it were failing as a fallout of commits d52eaa09 (pointed out
by Thomas upthread) and 6ef77cf46e8, which are not really related to
the code being changed by the patch.
On the patch itself, it seems straightforward enough. It simply takes
the feature we have for Append and MergeAppend nodes and adopts it for
ModifyTable which for the purposes of run-time pruning looks very much
like the aforementioned nodes.
Part of the optimizer patch that looks a bit complex is the changes to
inheritance_planner() which is to be expected, because that function
is a complex beast itself. I have suggestions to modify some comments
around the code added/modified by the patch for clarity; attaching a
delta patch for that.
The executor patch looks pretty benign too. Diffs that looked a bit
suspicious at first are due to replacing
ModifyTableState.resultRelInfo that is a pointer into
EState.es_result_relations array by an array of ResultRelInfo
pointers, but doing that seems to make the relevant code easier to
follow, especially if you consider the changes that the patch makes to
that code.
I'll set the CF entry to Needs Review, because AFAICS there are no
unaddressed comments.
Thanks,
Amit
Attachments:
20200124_amit-delta.patchtext/plain; charset=US-ASCII; name=20200124_amit-delta.patchDownload
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index 30d15291e3..c4244e6d29 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -1802,7 +1802,9 @@ inheritance_planner(PlannerInfo *root)
* When performing UPDATE/DELETE on a partitioned table, if the query has
* a WHERE clause which supports it, we may be able to perform run-time
* partition pruning. The following code sets things up to allow this to
- * be possible.
+ * be possible using the information from partition_root that was used
+ * during planning of the SELECT version of this query which we performed
+ * above.
*/
if (partition_root && !dummy_update)
{
@@ -1810,36 +1812,29 @@ inheritance_planner(PlannerInfo *root)
int i;
/*
- * Fetch the target partitioned table from the SELECT version of
- * the query which we performed above. This may have the base quals
+ * Fetch the target partitioned table which may have the base quals
* which could allow the run-time pruning to work.
*/
parent_rel = partition_root->simple_rel_array[top_parentRTindex];
-
final_rel->baserestrictinfo = parent_rel->baserestrictinfo;
- /* build a list of partitioned rels */
+ /* Collect all non-leaf tables in the partition tree being updated. */
i = -1;
while ((i = bms_next_member(parent_relids, i)) > 0)
partitioned_rels = lappend_int(partitioned_rels, i);
-
/*
- * In order to build the run-time pruning data we'll need append rels
- * any sub-partitioned tables. If there are some of those and the
- * append_rel_array is not already allocated, then do that now.
+ * There can only be a single partition tree, the one whose root is
+ * the query's main target table.
*/
- if (list_length(partitioned_rels) > 1 &&
- root->append_rel_array == NULL)
- root->append_rel_array = palloc0(sizeof(AppendRelInfo *) *
- root->simple_rel_array_size);
+ partitioned_rels = list_make1(partitioned_rels);
/*
- * There can only be a single partition hierarchy, so it's fine to
- * just make a single element list of the partitioned_rels.
+ * Update simple_rel_array and append_rel_array so that runtime
+ * pruning setup logic can find the relavant partitioned relations.
+ * Just use the one that the planning of SELECT version of the query
+ * would have created.
*/
- partitioned_rels = list_make1(partitioned_rels);
-
i = -1;
while ((i = bms_next_member(parent_relids, i)) >= 0)
{
@@ -1847,11 +1842,7 @@ inheritance_planner(PlannerInfo *root)
root->simple_rel_array[i] = partition_root->simple_rel_array[i];
- /*
- * The root partition won't have an append rel entry, so we can
- * skip that. We'll need to take the partition_root's version for
- * any sub-partitioned table's
- */
+ /* Root partitioned table doesn't have an AppendRelInfo. */
if (i != top_parentRTindex)
{
Assert(root->append_rel_array[i] == NULL);
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index ca53f684c2..020bc60fed 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -213,9 +213,6 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
*
* 'parentrel' is the RelOptInfo for an appendrel, and 'subpaths' is the list
* of scan paths for its child rels.
- *
- * If 'resultRelations' is non-NIL, then this List of relids is used to build
- * the mapping structures. Otherwise the 'subpaths' List is used.
*
* 'partitioned_rels' is a List containing Lists of relids of partitioned
* tables (a/k/a non-leaf partitions) that are parents of some of the child
@@ -227,6 +224,12 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
* that set into the PartitionPruneInfo's 'other_subplans' field. Callers
* will likely never want to prune subplans which are mentioned in this field.
*
+ * 'subpaths' (of an Append/MergeAppend for SELECT) and 'resultRelations'
+ * (of a ModifyTable for UPDATE/DELETE) are provided to map a given partition's
+ * index to their corresponding subpath's and result relation's index, resp.
+ * Having these maps allows the executor to easily skip subplans or result
+ * relations based on the indexes of partitions that are pruned.
+ *
* 'prunequal' is a list of potential pruning quals.
*/
PartitionPruneInfo *
@@ -249,10 +252,6 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
*/
relid_subplan_map = palloc0(sizeof(int) * root->simple_rel_array_size);
- /*
- * If 'resultRelations' are present then map these, otherwise we map the
- * 'subpaths' List.
- */
if (resultRelations != NIL)
{
i = 1;
runtime_pruning_for_modifytable_20200124.patchtext/plain; charset=US-ASCII; name=runtime_pruning_for_modifytable_20200124.patchDownload
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 2175dff824..77f7366cf8 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -1939,7 +1939,7 @@ postgresBeginForeignInsert(ModifyTableState *mtstate,
if (plan && plan->operation == CMD_UPDATE &&
(resultRelInfo->ri_usesFdwDirectModify ||
resultRelInfo->ri_FdwState) &&
- resultRelInfo > mtstate->resultRelInfo + mtstate->mt_whichplan)
+ resultRelInfo > mtstate->resultRelInfos[mtstate->mt_whichplan])
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot route tuples into foreign table to be updated \"%s\"",
@@ -1993,7 +1993,7 @@ postgresBeginForeignInsert(ModifyTableState *mtstate,
*/
if (plan && plan->operation == CMD_UPDATE &&
resultRelation == plan->rootRelation)
- resultRelation = mtstate->resultRelInfo[0].ri_RangeTableIndex;
+ resultRelation = mtstate->resultRelInfos[0]->ri_RangeTableIndex;
}
/* Construct the SQL command string. */
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index 40a8ec1abd..e2f348570e 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2854,7 +2854,7 @@ CopyFrom(CopyState cstate)
mtstate->ps.plan = NULL;
mtstate->ps.state = estate;
mtstate->operation = CMD_INSERT;
- mtstate->resultRelInfo = estate->es_result_relations;
+ mtstate->resultRelInfos = &estate->es_result_relations;
if (resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignInsert != NULL)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index d189b8d573..f6371318d0 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -3216,14 +3216,14 @@ show_modifytable_info(ModifyTableState *mtstate, List *ancestors,
/* Should we explicitly label target relations? */
labeltargets = (mtstate->mt_nplans > 1 ||
(mtstate->mt_nplans == 1 &&
- mtstate->resultRelInfo->ri_RangeTableIndex != node->nominalRelation));
+ mtstate->resultRelInfos[0]->ri_RangeTableIndex != node->nominalRelation));
if (labeltargets)
ExplainOpenGroup("Target Tables", "Target Tables", false, es);
for (j = 0; j < mtstate->mt_nplans; j++)
{
- ResultRelInfo *resultRelInfo = mtstate->resultRelInfo + j;
+ ResultRelInfo *resultRelInfo = mtstate->resultRelInfos[j];
FdwRoutine *fdwroutine = resultRelInfo->ri_FdwRoutine;
if (labeltargets)
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index c13b1d3501..f84110a54c 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -470,7 +470,7 @@ ExecHashSubPlanResultRelsByOid(ModifyTableState *mtstate,
/* Hash all subplans by their Oid */
for (i = 0; i < mtstate->mt_nplans; i++)
{
- ResultRelInfo *rri = &mtstate->resultRelInfo[i];
+ ResultRelInfo *rri = mtstate->resultRelInfos[i];
bool found;
Oid partoid = RelationGetRelid(rri->ri_RelationDesc);
SubplanResultRelHashElem *elem;
@@ -507,7 +507,7 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
ModifyTable *node = (ModifyTable *) mtstate->ps.plan;
Relation rootrel = rootResultRelInfo->ri_RelationDesc,
partrel;
- Relation firstResultRel = mtstate->resultRelInfo[0].ri_RelationDesc;
+ Relation firstResultRel = mtstate->resultRelInfos[0]->ri_RelationDesc;
ResultRelInfo *leaf_part_rri;
MemoryContext oldcxt;
AttrMap *part_attmap = NULL;
@@ -555,7 +555,7 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
List *wcoList;
List *wcoExprs = NIL;
ListCell *ll;
- int firstVarno = mtstate->resultRelInfo[0].ri_RangeTableIndex;
+ int firstVarno = mtstate->resultRelInfos[0]->ri_RangeTableIndex;
/*
* In the case of INSERT on a partitioned table, there is only one
@@ -619,7 +619,7 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
TupleTableSlot *slot;
ExprContext *econtext;
List *returningList;
- int firstVarno = mtstate->resultRelInfo[0].ri_RangeTableIndex;
+ int firstVarno = mtstate->resultRelInfos[0]->ri_RangeTableIndex;
/* See the comment above for WCO lists. */
Assert((node->operation == CMD_INSERT &&
@@ -678,7 +678,7 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
*/
if (node && node->onConflictAction != ONCONFLICT_NONE)
{
- int firstVarno = mtstate->resultRelInfo[0].ri_RangeTableIndex;
+ int firstVarno = mtstate->resultRelInfos[0]->ri_RangeTableIndex;
TupleDesc partrelDesc = RelationGetDescr(partrel);
ExprContext *econtext = mtstate->ps.ps_ExprContext;
ListCell *lc;
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 59d1a31c97..806f29f6b3 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -57,6 +57,9 @@
#include "utils/memutils.h"
#include "utils/rel.h"
+ /* Special values for mt_whichplan */
+#define WHICHPLAN_CHOOSE_PARTITIONS -1
+#define WHICHPLAN_NO_MATCHING_PARTITIONS -2
static bool ExecOnConflictUpdate(ModifyTableState *mtstate,
ResultRelInfo *resultRelInfo,
@@ -1260,7 +1263,7 @@ lreplace:;
* retrieve the one for this resultRel, we need to know the
* position of the resultRel in mtstate->resultRelInfo[].
*/
- map_index = resultRelInfo - mtstate->resultRelInfo;
+ map_index = mtstate->mt_whichplan;
Assert(map_index >= 0 && map_index < mtstate->mt_nplans);
tupconv_map = tupconv_map_for_subplan(mtstate, map_index);
if (tupconv_map != NULL)
@@ -1706,12 +1709,12 @@ static void
fireBSTriggers(ModifyTableState *node)
{
ModifyTable *plan = (ModifyTable *) node->ps.plan;
- ResultRelInfo *resultRelInfo = node->resultRelInfo;
+ ResultRelInfo *resultRelInfo = node->resultRelInfos[0];
/*
* If the node modifies a partitioned table, we must fire its triggers.
- * Note that in that case, node->resultRelInfo points to the first leaf
- * partition, not the root table.
+ * Note that in that case, node->resultRelInfos[0] points to the first
+ * leaf partition, not the root table.
*/
if (node->rootResultRelInfo != NULL)
resultRelInfo = node->rootResultRelInfo;
@@ -1749,13 +1752,14 @@ static ResultRelInfo *
getTargetResultRelInfo(ModifyTableState *node)
{
/*
- * Note that if the node modifies a partitioned table, node->resultRelInfo
- * points to the first leaf partition, not the root table.
+ * Note that if the node modifies a partitioned table,
+ * node->resultRelInfos[0] points to the first leaf partition, not the
+ * root table.
*/
if (node->rootResultRelInfo != NULL)
return node->rootResultRelInfo;
else
- return node->resultRelInfo;
+ return node->resultRelInfos[0];
}
/*
@@ -1934,7 +1938,7 @@ static void
ExecSetupChildParentMapForSubplan(ModifyTableState *mtstate)
{
ResultRelInfo *targetRelInfo = getTargetResultRelInfo(mtstate);
- ResultRelInfo *resultRelInfos = mtstate->resultRelInfo;
+ ResultRelInfo **resultRelInfos = mtstate->resultRelInfos;
TupleDesc outdesc;
int numResultRelInfos = mtstate->mt_nplans;
int i;
@@ -1954,7 +1958,7 @@ ExecSetupChildParentMapForSubplan(ModifyTableState *mtstate)
for (i = 0; i < numResultRelInfos; ++i)
{
mtstate->mt_per_subplan_tupconv_maps[i] =
- convert_tuples_by_name(RelationGetDescr(resultRelInfos[i].ri_RelationDesc),
+ convert_tuples_by_name(RelationGetDescr(resultRelInfos[i]->ri_RelationDesc),
outdesc);
}
}
@@ -2030,8 +2034,47 @@ ExecModifyTable(PlanState *pstate)
node->fireBSTriggers = false;
}
+ if (node->mt_whichplan < 0)
+ {
+ /* Handle choosing the valid partitions */
+ if (node->mt_whichplan == WHICHPLAN_CHOOSE_PARTITIONS)
+ {
+ PartitionPruneState *prunestate = node->mt_prune_state;
+
+ /* There should always be at least one */
+ Assert(node->mt_nplans > 0);
+
+ /*
+ * When partition pruning is enabled and exec params match the
+ * partition key then determine the minimum set of matching
+ * subnodes. Otherwise we match to all subnodes.
+ */
+ if (prunestate != NULL && prunestate->do_exec_prune)
+ {
+ node->mt_valid_subplans = ExecFindMatchingSubPlans(prunestate);
+ node->mt_whichplan = bms_next_member(node->mt_valid_subplans, -1);
+
+ /* If no subplan matches these params then we're done */
+ if (node->mt_whichplan < 0)
+ goto done;
+ }
+ else
+ {
+ node->mt_valid_subplans = bms_add_range(NULL, 0,
+ node->mt_nplans - 1);
+ node->mt_whichplan = 0;
+ }
+ }
+
+ /* partition pruning determined that no partitions match */
+ else if (node->mt_whichplan == WHICHPLAN_NO_MATCHING_PARTITIONS)
+ goto done;
+ else
+ elog(ERROR, "invalid subplan index: %d", node->mt_whichplan);
+ }
+
/* Preload local variables */
- resultRelInfo = node->resultRelInfo + node->mt_whichplan;
+ resultRelInfo = node->resultRelInfos[node->mt_whichplan];
subplanstate = node->mt_plans[node->mt_whichplan];
junkfilter = resultRelInfo->ri_junkFilter;
@@ -2073,10 +2116,12 @@ ExecModifyTable(PlanState *pstate)
if (TupIsNull(planSlot))
{
/* advance to next subplan if any */
- node->mt_whichplan++;
- if (node->mt_whichplan < node->mt_nplans)
+ node->mt_whichplan = bms_next_member(node->mt_valid_subplans,
+ node->mt_whichplan);
+
+ if (node->mt_whichplan >= 0)
{
- resultRelInfo++;
+ resultRelInfo = node->resultRelInfos[node->mt_whichplan];
subplanstate = node->mt_plans[node->mt_whichplan];
junkfilter = resultRelInfo->ri_junkFilter;
estate->es_result_relation_info = resultRelInfo;
@@ -2246,6 +2291,8 @@ ExecModifyTable(PlanState *pstate)
/* Restore es_result_relation_info before exiting */
estate->es_result_relation_info = saved_resultRelInfo;
+done:
+
/*
* We're done, but fire AFTER STATEMENT triggers before exiting.
*/
@@ -2268,9 +2315,11 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
int nplans = list_length(node->plans);
ResultRelInfo *saved_resultRelInfo;
ResultRelInfo *resultRelInfo;
+ Bitmapset *validsubplans;
Plan *subplan;
ListCell *l;
- int i;
+ int i,
+ j;
Relation rel;
bool update_tuple_routing_needed = node->partColsUpdated;
@@ -2288,9 +2337,75 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
mtstate->operation = operation;
mtstate->canSetTag = node->canSetTag;
mtstate->mt_done = false;
+ mtstate->mt_whichplan = WHICHPLAN_CHOOSE_PARTITIONS;
+
+ /* If run-time partition pruning is enabled, then set that up now */
+ if (node->part_prune_info != NULL)
+ {
+ PartitionPruneState *prunestate;
+
+ ExecAssignExprContext(estate, &mtstate->ps);
+
+ prunestate = ExecCreatePartitionPruneState(&mtstate->ps,
+ node->part_prune_info);
+ mtstate->mt_prune_state = prunestate;
+
+ /* Perform an initial partition prune, if required. */
+ if (prunestate->do_initial_prune)
+ {
+ /* Determine which subplans match the external params */
+ validsubplans = ExecFindInitialMatchingSubPlans(prunestate,
+ list_length(node->plans));
+
+ /*
+ * The case where no subplans survive pruning must be handled
+ * specially. The problem here is that code in explain.c requires
+ * an Append to have at least one subplan in order for it to
+ * properly determine the Vars in that subplan's targetlist. We
+ * sidestep this issue by just initializing the first subplan and
+ * setting as_whichplan to NO_MATCHING_SUBPLANS to indicate that
+ * we don't really need to scan any subnodes.
+ */
+ if (bms_is_empty(validsubplans))
+ {
+ mtstate->mt_whichplan = WHICHPLAN_NO_MATCHING_PARTITIONS;
+
+ /* Mark the first as valid so that it's initialized below */
+ validsubplans = bms_make_singleton(0);
+ }
+
+ nplans = bms_num_members(validsubplans);
+ }
+ else
+ {
+ /* We'll need to initialize all subplans */
+ nplans = list_length(node->plans);
+ validsubplans = bms_add_range(NULL, 0, nplans - 1);
+ }
+
+ /*
+ * If no runtime pruning is required, we can fill mt_valid_subplans
+ * immediately, preventing later calls to ExecFindMatchingSubPlans.
+ */
+ if (!prunestate->do_exec_prune)
+ mtstate->mt_valid_subplans = bms_add_range(NULL, 0, nplans - 1);
+ }
+ else
+ {
+ nplans = list_length(node->plans);
+
+ /*
+ * When run-time partition pruning is not enabled we can just mark all
+ * plans as valid, they must also all be initialized.
+ */
+ validsubplans = bms_add_range(NULL, 0, nplans - 1);
+ mtstate->mt_prune_state = NULL;
+ }
+
mtstate->mt_plans = (PlanState **) palloc0(sizeof(PlanState *) * nplans);
- mtstate->resultRelInfo = estate->es_result_relations + node->resultRelIndex;
+ mtstate->resultRelInfos = (ResultRelInfo **)
+ palloc(sizeof(ResultRelInfo *) * nplans);
mtstate->mt_scans = (TupleTableSlot **) palloc0(sizeof(TupleTableSlot *) * nplans);
/* If modifying a partitioned table, initialize the root table info */
@@ -2317,11 +2432,18 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
*/
saved_resultRelInfo = estate->es_result_relation_info;
- resultRelInfo = mtstate->resultRelInfo;
- i = 0;
+ j = i = 0;
foreach(l, node->plans)
{
+ if (!bms_is_member(i, validsubplans))
+ {
+ i++;
+ continue;
+ }
+
subplan = (Plan *) lfirst(l);
+ resultRelInfo = estate->es_result_relations + node->resultRelIndex + i;
+ mtstate->resultRelInfos[j] = resultRelInfo;
/* Initialize the usesFdwDirectModify flag */
resultRelInfo->ri_usesFdwDirectModify = bms_is_member(i,
@@ -2359,9 +2481,9 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
/* Now init the plan for this result rel */
estate->es_result_relation_info = resultRelInfo;
- mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
- mtstate->mt_scans[i] =
- ExecInitExtraTupleSlot(mtstate->ps.state, ExecGetResultType(mtstate->mt_plans[i]),
+ mtstate->mt_plans[j] = ExecInitNode(subplan, estate, eflags);
+ mtstate->mt_scans[j] =
+ ExecInitExtraTupleSlot(mtstate->ps.state, ExecGetResultType(mtstate->mt_plans[j]),
table_slot_callbacks(resultRelInfo->ri_RelationDesc));
/* Also let FDWs init themselves for foreign-table result rels */
@@ -2377,9 +2499,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
i,
eflags);
}
-
- resultRelInfo++;
i++;
+ j++;
}
estate->es_result_relation_info = saved_resultRelInfo;
@@ -2428,14 +2549,21 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
/*
* Initialize any WITH CHECK OPTION constraints if needed.
*/
- resultRelInfo = mtstate->resultRelInfo;
- i = 0;
+ j = i = 0;
foreach(l, node->withCheckOptionLists)
{
- List *wcoList = (List *) lfirst(l);
+ List *wcoList;
List *wcoExprs = NIL;
ListCell *ll;
+ if (!bms_is_member(i, validsubplans))
+ {
+ i++;
+ continue;
+ }
+
+ wcoList = (List *) lfirst(l);
+
foreach(ll, wcoList)
{
WithCheckOption *wco = (WithCheckOption *) lfirst(ll);
@@ -2445,9 +2573,10 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
wcoExprs = lappend(wcoExprs, wcoExpr);
}
+ resultRelInfo = mtstate->resultRelInfos[j];
resultRelInfo->ri_WithCheckOptions = wcoList;
resultRelInfo->ri_WithCheckOptionExprs = wcoExprs;
- resultRelInfo++;
+ j++;
i++;
}
@@ -2477,16 +2606,25 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
/*
* Build a projection for each result rel.
*/
- resultRelInfo = mtstate->resultRelInfo;
+ j = i = 0;
foreach(l, node->returningLists)
{
- List *rlist = (List *) lfirst(l);
+ List *rlist;
+
+ if (!bms_is_member(i, validsubplans))
+ {
+ i++;
+ continue;
+ }
+ rlist = (List *) lfirst(l);
+
+ resultRelInfo = mtstate->resultRelInfos[j];
resultRelInfo->ri_returningList = rlist;
resultRelInfo->ri_projectReturning =
ExecBuildProjectionInfo(rlist, econtext, slot, &mtstate->ps,
resultRelInfo->ri_RelationDesc->rd_att);
- resultRelInfo++;
+ j++;
}
}
else
@@ -2497,12 +2635,10 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
*/
mtstate->ps.plan->targetlist = NIL;
ExecInitResultTypeTL(&mtstate->ps);
-
- mtstate->ps.ps_ExprContext = NULL;
}
/* Set the list of arbiter indexes if needed for ON CONFLICT */
- resultRelInfo = mtstate->resultRelInfo;
+ resultRelInfo = mtstate->resultRelInfos[0];
if (node->onConflictAction != ONCONFLICT_NONE)
resultRelInfo->ri_onConflictArbiterIndexes = node->arbiterIndexes;
@@ -2593,7 +2729,6 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
}
/* select first subplan */
- mtstate->mt_whichplan = 0;
subplan = (Plan *) linitial(node->plans);
EvalPlanQualSetPlan(&mtstate->mt_epqstate, subplan,
mtstate->mt_arowmarks[0]);
@@ -2642,12 +2777,12 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
if (junk_filter_needed)
{
- resultRelInfo = mtstate->resultRelInfo;
for (i = 0; i < nplans; i++)
{
JunkFilter *j;
TupleTableSlot *junkresslot;
+ resultRelInfo = mtstate->resultRelInfos[i];
subplan = mtstate->mt_plans[i]->plan;
if (operation == CMD_INSERT || operation == CMD_UPDATE)
ExecCheckPlanOutput(resultRelInfo->ri_RelationDesc,
@@ -2690,13 +2825,12 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
}
resultRelInfo->ri_junkFilter = j;
- resultRelInfo++;
}
}
else
{
if (operation == CMD_INSERT)
- ExecCheckPlanOutput(mtstate->resultRelInfo->ri_RelationDesc,
+ ExecCheckPlanOutput(mtstate->resultRelInfos[0]->ri_RelationDesc,
subplan->targetlist);
}
}
@@ -2735,7 +2869,7 @@ ExecEndModifyTable(ModifyTableState *node)
*/
for (i = 0; i < node->mt_nplans; i++)
{
- ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
+ ResultRelInfo *resultRelInfo = node->resultRelInfos[i];
if (!resultRelInfo->ri_usesFdwDirectModify &&
resultRelInfo->ri_FdwRoutine != NULL &&
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 54ad62bb7f..84b0be7a79 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -216,6 +216,7 @@ _copyModifyTable(const ModifyTable *from)
COPY_BITMAPSET_FIELD(fdwDirectModifyPlans);
COPY_NODE_FIELD(rowMarks);
COPY_SCALAR_FIELD(epqParam);
+ COPY_NODE_FIELD(part_prune_info);
COPY_SCALAR_FIELD(onConflictAction);
COPY_NODE_FIELD(arbiterIndexes);
COPY_NODE_FIELD(onConflictSet);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index d76fae44b8..e673c7fd17 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -417,6 +417,7 @@ _outModifyTable(StringInfo str, const ModifyTable *node)
WRITE_BITMAPSET_FIELD(fdwDirectModifyPlans);
WRITE_NODE_FIELD(rowMarks);
WRITE_INT_FIELD(epqParam);
+ WRITE_NODE_FIELD(part_prune_info);
WRITE_ENUM_FIELD(onConflictAction, OnConflictAction);
WRITE_NODE_FIELD(arbiterIndexes);
WRITE_NODE_FIELD(onConflictSet);
@@ -2092,6 +2093,7 @@ _outModifyTablePath(StringInfo str, const ModifyTablePath *node)
WRITE_UINT_FIELD(rootRelation);
WRITE_BOOL_FIELD(partColsUpdated);
WRITE_NODE_FIELD(resultRelations);
+ WRITE_NODE_FIELD(partitioned_rels);
WRITE_NODE_FIELD(subpaths);
WRITE_NODE_FIELD(subroots);
WRITE_NODE_FIELD(withCheckOptionLists);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 551ce6c41c..cbd6ea7a6b 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -1647,6 +1647,7 @@ _readModifyTable(void)
READ_BITMAPSET_FIELD(fdwDirectModifyPlans);
READ_NODE_FIELD(rowMarks);
READ_INT_FIELD(epqParam);
+ READ_NODE_FIELD(part_prune_info);
READ_ENUM_FIELD(onConflictAction, OnConflictAction);
READ_NODE_FIELD(arbiterIndexes);
READ_NODE_FIELD(onConflictSet);
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index dff826a828..6b0b46d9f3 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -289,7 +289,8 @@ static ModifyTable *make_modifytable(PlannerInfo *root,
bool partColsUpdated,
List *resultRelations, List *subplans, List *subroots,
List *withCheckOptionLists, List *returningLists,
- List *rowMarks, OnConflictExpr *onconflict, int epqParam);
+ List *rowMarks, OnConflictExpr *onconflict, int epqParam,
+ PartitionPruneInfo *partpruneinfos);
static GatherMerge *create_gather_merge_plan(PlannerInfo *root,
GatherMergePath *best_path);
@@ -1238,6 +1239,7 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
make_partition_pruneinfo(root, rel,
best_path->subpaths,
best_path->partitioned_rels,
+ NIL,
prunequal);
}
@@ -1404,6 +1406,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
partpruneinfo = make_partition_pruneinfo(root, rel,
best_path->subpaths,
best_path->partitioned_rels,
+ NIL,
prunequal);
}
@@ -2586,6 +2589,7 @@ create_modifytable_plan(PlannerInfo *root, ModifyTablePath *best_path)
List *subplans = NIL;
ListCell *subpaths,
*subroots;
+ PartitionPruneInfo *partpruneinfos = NULL;
/* Build the plan for each input path */
forboth(subpaths, best_path->subpaths,
@@ -2614,6 +2618,30 @@ create_modifytable_plan(PlannerInfo *root, ModifyTablePath *best_path)
subplans = lappend(subplans, subplan);
}
+ if (enable_partition_pruning &&
+ best_path->partitioned_rels != NIL &&
+ !IS_DUMMY_MODIFYTABLE(best_path))
+ {
+ RelOptInfo *rel = best_path->path.parent;
+ List *prunequal = NIL;
+
+ prunequal = extract_actual_clauses(rel->baserestrictinfo, false);
+
+ /*
+ * If any quals exist, then these may be useful to allow us to perform
+ * further partition pruning during execution. We'll generate a
+ * PartitionPruneInfo for each partitioned rel to store these quals
+ * and allow translation of partition indexes into subpath indexes.
+ */
+ if (prunequal != NIL)
+ partpruneinfos = make_partition_pruneinfo(root,
+ best_path->path.parent,
+ best_path->subpaths,
+ best_path->partitioned_rels,
+ best_path->resultRelations,
+ prunequal);
+ }
+
plan = make_modifytable(root,
best_path->operation,
best_path->canSetTag,
@@ -2627,7 +2655,8 @@ create_modifytable_plan(PlannerInfo *root, ModifyTablePath *best_path)
best_path->returningLists,
best_path->rowMarks,
best_path->onconflict,
- best_path->epqParam);
+ best_path->epqParam,
+ partpruneinfos);
copy_generic_path_info(&plan->plan, &best_path->path);
@@ -6621,7 +6650,8 @@ make_modifytable(PlannerInfo *root,
bool partColsUpdated,
List *resultRelations, List *subplans, List *subroots,
List *withCheckOptionLists, List *returningLists,
- List *rowMarks, OnConflictExpr *onconflict, int epqParam)
+ List *rowMarks, OnConflictExpr *onconflict, int epqParam,
+ PartitionPruneInfo *partpruneinfos)
{
ModifyTable *node = makeNode(ModifyTable);
List *fdw_private_list;
@@ -6682,6 +6712,7 @@ make_modifytable(PlannerInfo *root,
node->returningLists = returningLists;
node->rowMarks = rowMarks;
node->epqParam = epqParam;
+ node->part_prune_info = partpruneinfos;
/*
* For each result relation that is a foreign table, allow the FDW to
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index d6f2153593..30d15291e3 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -1239,6 +1239,9 @@ inheritance_planner(PlannerInfo *root)
RangeTblEntry *parent_rte;
Bitmapset *parent_relids;
Query **parent_parses;
+ PlannerInfo *partition_root = NULL;
+ List *partitioned_rels = NIL;
+ bool dummy_update = false;
/* Should only get here for UPDATE or DELETE */
Assert(parse->commandType == CMD_UPDATE ||
@@ -1350,6 +1353,13 @@ inheritance_planner(PlannerInfo *root)
* expand_partitioned_rtentry for the UPDATE target.)
*/
root->partColsUpdated = subroot->partColsUpdated;
+
+ /*
+ * Save this for later so that we can enable run-time pruning on
+ * the partitioned table(s).
+ */
+ if (parent_rte->relkind == RELKIND_PARTITIONED_TABLE)
+ partition_root = subroot;
}
/*----------
@@ -1745,6 +1755,9 @@ inheritance_planner(PlannerInfo *root)
returningLists = list_make1(parse->returningList);
/* Disable tuple routing, too, just to be safe */
root->partColsUpdated = false;
+
+ /* Mark that we're performing a dummy update */
+ dummy_update = true;
}
else
{
@@ -1784,6 +1797,69 @@ inheritance_planner(PlannerInfo *root)
else
rowMarks = root->rowMarks;
+
+ /*
+ * When performing UPDATE/DELETE on a partitioned table, if the query has
+ * a WHERE clause which supports it, we may be able to perform run-time
+ * partition pruning. The following code sets things up to allow this to
+ * be possible.
+ */
+ if (partition_root && !dummy_update)
+ {
+ RelOptInfo *parent_rel;
+ int i;
+
+ /*
+ * Fetch the target partitioned table from the SELECT version of
+ * the query which we performed above. This may have the base quals
+ * which could allow the run-time pruning to work.
+ */
+ parent_rel = partition_root->simple_rel_array[top_parentRTindex];
+
+ final_rel->baserestrictinfo = parent_rel->baserestrictinfo;
+
+ /* build a list of partitioned rels */
+ i = -1;
+ while ((i = bms_next_member(parent_relids, i)) > 0)
+ partitioned_rels = lappend_int(partitioned_rels, i);
+
+
+ /*
+ * In order to build the run-time pruning data we'll need append rels
+ * any sub-partitioned tables. If there are some of those and the
+ * append_rel_array is not already allocated, then do that now.
+ */
+ if (list_length(partitioned_rels) > 1 &&
+ root->append_rel_array == NULL)
+ root->append_rel_array = palloc0(sizeof(AppendRelInfo *) *
+ root->simple_rel_array_size);
+
+ /*
+ * There can only be a single partition hierarchy, so it's fine to
+ * just make a single element list of the partitioned_rels.
+ */
+ partitioned_rels = list_make1(partitioned_rels);
+
+ i = -1;
+ while ((i = bms_next_member(parent_relids, i)) >= 0)
+ {
+ Assert(root->simple_rel_array[i] == NULL);
+
+ root->simple_rel_array[i] = partition_root->simple_rel_array[i];
+
+ /*
+ * The root partition won't have an append rel entry, so we can
+ * skip that. We'll need to take the partition_root's version for
+ * any sub-partitioned table's
+ */
+ if (i != top_parentRTindex)
+ {
+ Assert(root->append_rel_array[i] == NULL);
+ root->append_rel_array[i] = partition_root->append_rel_array[i];
+ }
+ }
+ }
+
/* Create Path representing a ModifyTable to do the UPDATE/DELETE work */
add_path(final_rel, (Path *)
create_modifytable_path(root, final_rel,
@@ -1793,6 +1869,7 @@ inheritance_planner(PlannerInfo *root)
rootRelation,
root->partColsUpdated,
resultRelations,
+ partitioned_rels,
subpaths,
subroots,
withCheckOptionLists,
@@ -2376,6 +2453,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
rootRelation,
false,
list_make1_int(parse->resultRelation),
+ NIL,
list_make1(path),
list_make1(root),
withCheckOptionLists,
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index e6d08aede5..63da0396a8 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3431,6 +3431,9 @@ create_lockrows_path(PlannerInfo *root, RelOptInfo *rel,
* 'partColsUpdated' is true if any partitioning columns are being updated,
* either from the target relation or a descendent partitioned table.
* 'resultRelations' is an integer list of actual RT indexes of target rel(s)
+ * 'partitioned_rels' is an integer list of RT indexes of non-leaf tables in
+ * the partition tree, if this is an UPDATE/DELETE to a partitioned table.
+ * Otherwise NIL.
* 'subpaths' is a list of Path(s) producing source data (one per rel)
* 'subroots' is a list of PlannerInfo structs (one per rel)
* 'withCheckOptionLists' is a list of WCO lists (one per rel)
@@ -3444,8 +3447,8 @@ create_modifytable_path(PlannerInfo *root, RelOptInfo *rel,
CmdType operation, bool canSetTag,
Index nominalRelation, Index rootRelation,
bool partColsUpdated,
- List *resultRelations, List *subpaths,
- List *subroots,
+ List *resultRelations, List *partitioned_rels,
+ List *subpaths, List *subroots,
List *withCheckOptionLists, List *returningLists,
List *rowMarks, OnConflictExpr *onconflict,
int epqParam)
@@ -3513,6 +3516,7 @@ create_modifytable_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->rootRelation = rootRelation;
pathnode->partColsUpdated = partColsUpdated;
pathnode->resultRelations = resultRelations;
+ pathnode->partitioned_rels = list_copy(partitioned_rels);
pathnode->subpaths = subpaths;
pathnode->subroots = subroots;
pathnode->withCheckOptionLists = withCheckOptionLists;
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index eac52e6ec8..ca53f684c2 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -214,7 +214,10 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
* 'parentrel' is the RelOptInfo for an appendrel, and 'subpaths' is the list
* of scan paths for its child rels.
*
- * 'partitioned_rels' is a List containing Lists of relids of partitioned
+ * If 'resultRelations' is non-NIL, then this List of relids is used to build
+ * the mapping structures. Otherwise the 'subpaths' List is used.
+ *
+ * 'partitioned_rels' is a List containing Lists of relids of partitioned
* tables (a/k/a non-leaf partitions) that are parents of some of the child
* rels. Here we attempt to populate the PartitionPruneInfo by adding a
* 'prune_infos' item for each sublist in the 'partitioned_rels' list.
@@ -229,6 +232,7 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
PartitionPruneInfo *
make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
List *subpaths, List *partitioned_rels,
+ List *resultRelations,
List *prunequal)
{
PartitionPruneInfo *pruneinfo;
@@ -246,23 +250,36 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
relid_subplan_map = palloc0(sizeof(int) * root->simple_rel_array_size);
/*
- * relid_subplan_map maps relid of a leaf partition to the index in
- * 'subpaths' of the scan plan for that partition.
+ * If 'resultRelations' are present then map these, otherwise we map the
+ * 'subpaths' List.
*/
- i = 1;
- foreach(lc, subpaths)
+ if (resultRelations != NIL)
{
- Path *path = (Path *) lfirst(lc);
- RelOptInfo *pathrel = path->parent;
-
- Assert(IS_SIMPLE_REL(pathrel));
- Assert(pathrel->relid < root->simple_rel_array_size);
- /* No duplicates please */
- Assert(relid_subplan_map[pathrel->relid] == 0);
+ i = 1;
+ foreach(lc, resultRelations)
+ {
+ int resultrel = lfirst_int(lc);
- relid_subplan_map[pathrel->relid] = i++;
+ Assert(resultrel < root->simple_rel_array_size);
+ relid_subplan_map[resultrel] = i++;
+ }
}
+ else
+ {
+ i = 1;
+ foreach(lc, subpaths)
+ {
+ Path *path = (Path *)lfirst(lc);
+ RelOptInfo *pathrel = path->parent;
+ Assert(IS_SIMPLE_REL(pathrel));
+ Assert(pathrel->relid < root->simple_rel_array_size);
+ /* No duplicates please */
+ Assert(relid_subplan_map[pathrel->relid] == 0);
+
+ relid_subplan_map[pathrel->relid] = i++;
+ }
+ }
/* We now build a PartitionedRelPruneInfo for each partitioned rel. */
prunerelinfos = NIL;
foreach(lc, partitioned_rels)
@@ -404,9 +421,12 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
* an adjust_appendrel_attrs step. But it might not be, and then
* we have to translate. We update the prunequal parameter here,
* because in later iterations of the loop for child partitions,
- * we want to translate from parent to child variables.
+ * we want to translate from parent to child variables. We don't
+ * need to do this when planning a non-SELECT as we're only
+ * working with a single partition hierarchy in that case.
*/
- if (!bms_equal(parentrel->relids, subpart->relids))
+ if (root->parse->commandType == CMD_SELECT &&
+ !bms_equal(parentrel->relids, subpart->relids))
{
int nappinfos;
AppendRelInfo **appinfos = find_appinfos_by_relids(root,
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 1f6f5bbc20..c9ad649189 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1165,7 +1165,7 @@ typedef struct ModifyTableState
int mt_whichplan; /* which one is being executed (0..n-1) */
TupleTableSlot **mt_scans; /* input tuple corresponding to underlying
* plans */
- ResultRelInfo *resultRelInfo; /* per-subplan target relations */
+ ResultRelInfo **resultRelInfos; /* per-subplan target relations */
ResultRelInfo *rootResultRelInfo; /* root target relation (partitioned
* table root) */
List **mt_arowmarks; /* per-subplan ExecAuxRowMark lists */
@@ -1189,6 +1189,14 @@ typedef struct ModifyTableState
/* Per plan map for tuple conversion from child to root */
TupleConversionMap **mt_per_subplan_tupconv_maps;
+
+ /*
+ * Details required to allow partitions to be eliminated from the scan, or
+ * NULL if not possible.
+ */
+ struct PartitionPruneState *mt_prune_state;
+ Bitmapset *mt_valid_subplans; /* for runtime pruning, valid mt_plans
+ * indexes to scan. */
} ModifyTableState;
/* ----------------
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 3d3be197e0..1de53e40f5 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -1382,6 +1382,10 @@ typedef struct AppendPath
#define IS_DUMMY_APPEND(p) \
(IsA((p), AppendPath) && ((AppendPath *) (p))->subpaths == NIL)
+#define IS_DUMMY_MODIFYTABLE(p) \
+ (list_length((p)->subpaths) == 1 && \
+ IS_DUMMY_APPEND(linitial((p)->subpaths)))
+
/*
* A relation that's been proven empty will have one path that is dummy
* (but might have projection paths on top). For historical reasons,
@@ -1777,6 +1781,8 @@ typedef struct ModifyTablePath
Index rootRelation; /* Root RT index, if target is partitioned */
bool partColsUpdated; /* some part key in hierarchy updated */
List *resultRelations; /* integer list of RT indexes */
+ /* RT indexes of non-leaf tables in a partition tree */
+ List *partitioned_rels;
List *subpaths; /* Path(s) producing source data */
List *subroots; /* per-target-table PlannerInfos */
List *withCheckOptionLists; /* per-target-table WCO lists */
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 32c0d87f80..02a0302e12 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -233,6 +233,8 @@ typedef struct ModifyTable
Bitmapset *fdwDirectModifyPlans; /* indices of FDW DM plans */
List *rowMarks; /* PlanRowMarks (non-locking only) */
int epqParam; /* ID of Param for EvalPlanQual re-eval */
+ /* Mapping details for run-time subplan pruning, one per partitioned_rels */
+ struct PartitionPruneInfo *part_prune_info;
OnConflictAction onConflictAction; /* ON CONFLICT action */
List *arbiterIndexes; /* List of ON CONFLICT arbiter index OIDs */
List *onConflictSet; /* SET for INSERT ON CONFLICT DO UPDATE */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index e450fe112a..145506a634 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -252,10 +252,12 @@ extern LockRowsPath *create_lockrows_path(PlannerInfo *root, RelOptInfo *rel,
extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
RelOptInfo *rel,
CmdType operation, bool canSetTag,
- Index nominalRelation, Index rootRelation,
+ Index nominalRelation,
+ Index rootRelation,
bool partColsUpdated,
- List *resultRelations, List *subpaths,
- List *subroots,
+ List *resultRelations,
+ List *partitioned_rels,
+ List *subpaths, List *subroots,
List *withCheckOptionLists, List *returningLists,
List *rowMarks, OnConflictExpr *onconflict,
int epqParam);
diff --git a/src/include/partitioning/partprune.h b/src/include/partitioning/partprune.h
index babdad2c3e..eb84e176eb 100644
--- a/src/include/partitioning/partprune.h
+++ b/src/include/partitioning/partprune.h
@@ -72,6 +72,7 @@ extern PartitionPruneInfo *make_partition_pruneinfo(struct PlannerInfo *root,
struct RelOptInfo *parentrel,
List *subpaths,
List *partitioned_rels,
+ List *resultRelations,
List *prunequal);
extern Bitmapset *prune_append_rel_partitions(struct RelOptInfo *rel);
extern Bitmapset *get_matching_partitions(PartitionPruneContext *context,
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 424d51d521..9921b68554 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -3196,6 +3196,110 @@ explain (analyze, costs off, summary off, timing off) select * from ma_test wher
reset enable_seqscan;
reset enable_sort;
+--
+-- Test run-time pruning of ModifyTable subnodes
+--
+-- Ensure only ma_test_p3 is scanned.
+explain (analyze, costs off, summary off, timing off) delete from ma_test where a = (select 29);
+ QUERY PLAN
+----------------------------------------------------------------
+ Delete on ma_test (actual rows=0 loops=1)
+ Delete on ma_test_p1 ma_test_1
+ Delete on ma_test_p2 ma_test_2
+ Delete on ma_test_p3 ma_test_3
+ InitPlan 1 (returns $0)
+ -> Result (actual rows=1 loops=1)
+ -> Seq Scan on ma_test_p1 ma_test_1 (never executed)
+ Filter: (a = $0)
+ -> Seq Scan on ma_test_p2 ma_test_2 (never executed)
+ Filter: (a = $0)
+ -> Seq Scan on ma_test_p3 ma_test_3 (actual rows=1 loops=1)
+ Filter: (a = $0)
+ Rows Removed by Filter: 9
+(13 rows)
+
+-- Ensure no partitions are scanned.
+explain (analyze, costs off, summary off, timing off) delete from ma_test where a = (select 30);
+ QUERY PLAN
+---------------------------------------------------------
+ Delete on ma_test (actual rows=0 loops=1)
+ Delete on ma_test_p1 ma_test_1
+ Delete on ma_test_p2 ma_test_2
+ Delete on ma_test_p3 ma_test_3
+ InitPlan 1 (returns $0)
+ -> Result (actual rows=1 loops=1)
+ -> Seq Scan on ma_test_p1 ma_test_1 (never executed)
+ Filter: (a = $0)
+ -> Seq Scan on ma_test_p2 ma_test_2 (never executed)
+ Filter: (a = $0)
+ -> Seq Scan on ma_test_p3 ma_test_3 (never executed)
+ Filter: (a = $0)
+(12 rows)
+
+-- Ensure partition pruning works with an update of the partition key.
+explain (analyze, costs off, summary off, timing off) update ma_test set a = 29 where a = (select 1);
+ QUERY PLAN
+----------------------------------------------------------------
+ Update on ma_test (actual rows=0 loops=1)
+ Update on ma_test_p1 ma_test_1
+ Update on ma_test_p2 ma_test_2
+ Update on ma_test_p3 ma_test_3
+ InitPlan 1 (returns $0)
+ -> Result (actual rows=1 loops=1)
+ -> Seq Scan on ma_test_p1 ma_test_1 (actual rows=1 loops=1)
+ Filter: (a = $0)
+ Rows Removed by Filter: 9
+ -> Seq Scan on ma_test_p2 ma_test_2 (never executed)
+ Filter: (a = $0)
+ -> Seq Scan on ma_test_p3 ma_test_3 (never executed)
+ Filter: (a = $0)
+(13 rows)
+
+-- Verify the above command
+select tableoid::regclass,a from ma_test where a = 29;
+ tableoid | a
+------------+----
+ ma_test_p3 | 29
+(1 row)
+
+truncate ma_test;
+prepare mt_q1 (int) as
+delete from ma_test where a > $1;
+set plan_cache_mode = force_generic_plan;
+explain (analyze, costs off, summary off, timing off) execute mt_q1(15);
+ QUERY PLAN
+----------------------------------------------------------------
+ Delete on ma_test (actual rows=0 loops=1)
+ Delete on ma_test_p2 ma_test_1
+ Delete on ma_test_p3 ma_test_2
+ Subplans Removed: 1
+ -> Seq Scan on ma_test_p2 ma_test_1 (actual rows=0 loops=1)
+ Filter: (a > $1)
+ -> Seq Scan on ma_test_p3 ma_test_2 (actual rows=0 loops=1)
+ Filter: (a > $1)
+(8 rows)
+
+explain (analyze, costs off, summary off, timing off) execute mt_q1(25);
+ QUERY PLAN
+----------------------------------------------------------------
+ Delete on ma_test (actual rows=0 loops=1)
+ Delete on ma_test_p3 ma_test_1
+ Subplans Removed: 2
+ -> Seq Scan on ma_test_p3 ma_test_1 (actual rows=0 loops=1)
+ Filter: (a > $1)
+(5 rows)
+
+-- Ensure ModifyTable behaves correctly when no subplans match exec params
+explain (analyze, costs off, summary off, timing off) execute mt_q1(35);
+ QUERY PLAN
+---------------------------------------------------------
+ Delete on ma_test (actual rows=0 loops=1)
+ Delete on ma_test_p1 ma_test_1
+ Subplans Removed: 2
+ -> Seq Scan on ma_test_p1 ma_test_1 (never executed)
+ Filter: (a > $1)
+(5 rows)
+
drop table ma_test;
reset enable_indexonlyscan;
--
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index d9daba3af3..047ed00ed3 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -851,6 +851,34 @@ explain (analyze, costs off, summary off, timing off) select * from ma_test wher
reset enable_seqscan;
reset enable_sort;
+--
+-- Test run-time pruning of ModifyTable subnodes
+--
+
+-- Ensure only ma_test_p3 is scanned.
+explain (analyze, costs off, summary off, timing off) delete from ma_test where a = (select 29);
+
+-- Ensure no partitions are scanned.
+explain (analyze, costs off, summary off, timing off) delete from ma_test where a = (select 30);
+
+-- Ensure partition pruning works with an update of the partition key.
+explain (analyze, costs off, summary off, timing off) update ma_test set a = 29 where a = (select 1);
+
+-- Verify the above command
+select tableoid::regclass,a from ma_test where a = 29;
+
+truncate ma_test;
+
+prepare mt_q1 (int) as
+delete from ma_test where a > $1;
+
+set plan_cache_mode = force_generic_plan;
+
+explain (analyze, costs off, summary off, timing off) execute mt_q1(15);
+explain (analyze, costs off, summary off, timing off) execute mt_q1(25);
+-- Ensure ModifyTable behaves correctly when no subplans match exec params
+explain (analyze, costs off, summary off, timing off) execute mt_q1(35);
+
drop table ma_test;
reset enable_indexonlyscan;
Thanks for having a look at this, Amit.
On Fri, 24 Jan 2020 at 21:57, Amit Langote <amitlangote09@gmail.com> wrote:
On Thu, Jan 23, 2020 at 4:31 PM Amit Langote <amitlangote09@gmail.com> wrote:
Part of the optimizer patch that looks a bit complex is the changes to
inheritance_planner() which is to be expected, because that function
is a complex beast itself. I have suggestions to modify some comments
around the code added/modified by the patch for clarity; attaching a
delta patch for that.
I've made another pass over the patch and made various changes. The
biggest of which was the required modifications to nodeModifyTable.c
so that it can now prune all partitions. Append and MergeAppend were
modified to allow this in 5935917ce59 (Thanks for pushing that Tom).
I've also slightly simplified the code in ExecModifyTable() and added
slightly more code to ExecInitModifyTable(). We now only set
mt_whichplan to WHICHPLAN_CHOOSE_SUBPLAN when run-time pruning is
enabled and do_exec_prune is true. I also made it so when all
partitions are pruned that we set mt_whichplan to
WHICHPLAN_CHOOSE_SUBPLAN as this saves an additional run-time check
during execution.
Over in inheritance_planner(), I noticed that the RT index of the
SELECT query and the UPDATE/DELETE query can differ. There was some
code that performed translations. I changed that code slightly so that
it's a bit more optimal. It was building two lists, one for the old
RT index and one for the new. It added elements to this list
regardless of if the RT indexes were the same or not. I've now changed
that to only add to the list if they differ, which I feel should never
be slower and most likely always faster. I'm also now building a
translation map between the old and new RT indexes, however, I only
found one test in the regression tests which require any sort of
translation of these RT indexes. This was with an inheritance table,
so I need to do a bit more work to find a case where this happens with
a partitioned table to ensure all this works.
The executor patch looks pretty benign too. Diffs that looked a bit
suspicious at first are due to replacing
ModifyTableState.resultRelInfo that is a pointer into
EState.es_result_relations array by an array of ResultRelInfo
pointers, but doing that seems to make the relevant code easier to
follow, especially if you consider the changes that the patch makes to
that code.
Yeah, that's because the ModifyTableState's resultRelInfo field was
just a pointer to the estate->es_result_relations array offset by the
ModifyTable's resultRelIndex. This was fine previously because we
always initialised the plans for each ResultRelInfo. However, now
that we might be pruning some of those that array can't be used as
it'll still contain ResultRelInfos for relations we're not going to
touch. Changing this to an array of pointers allows us to point to the
elements in estate->es_result_relations that we're going to use. I
also renamed the field just to ensure nothing can compile (thinking of
extensions here) that's not got updated code.
Tom, I'm wondering if you wouldn't mind looking over my changes to
inheritance_planner()?
David
Attachments:
modifytable_runtime_prune_2020-03-10.patchapplication/octet-stream; name=modifytable_runtime_prune_2020-03-10.patchDownload
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 2175dff824..77f7366cf8 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -1939,7 +1939,7 @@ postgresBeginForeignInsert(ModifyTableState *mtstate,
if (plan && plan->operation == CMD_UPDATE &&
(resultRelInfo->ri_usesFdwDirectModify ||
resultRelInfo->ri_FdwState) &&
- resultRelInfo > mtstate->resultRelInfo + mtstate->mt_whichplan)
+ resultRelInfo > mtstate->resultRelInfos[mtstate->mt_whichplan])
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot route tuples into foreign table to be updated \"%s\"",
@@ -1993,7 +1993,7 @@ postgresBeginForeignInsert(ModifyTableState *mtstate,
*/
if (plan && plan->operation == CMD_UPDATE &&
resultRelation == plan->rootRelation)
- resultRelation = mtstate->resultRelInfo[0].ri_RangeTableIndex;
+ resultRelation = mtstate->resultRelInfos[0]->ri_RangeTableIndex;
}
/* Construct the SQL command string. */
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index e79ede4cb8..f97d62efc8 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2854,7 +2854,7 @@ CopyFrom(CopyState cstate)
mtstate->ps.plan = NULL;
mtstate->ps.state = estate;
mtstate->operation = CMD_INSERT;
- mtstate->resultRelInfo = estate->es_result_relations;
+ mtstate->resultRelInfos = &estate->es_result_relations;
if (resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignInsert != NULL)
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index d901dc4a50..18b0697591 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -1988,6 +1988,11 @@ ExplainNode(PlanState *planstate, List *ancestors,
list_length(((MergeAppend *) plan)->mergeplans),
es);
break;
+ case T_ModifyTable:
+ ExplainMissingMembers(((ModifyTableState *) planstate)->mt_nplans,
+ list_length(((ModifyTable *) plan)->plans),
+ es);
+ break;
default:
break;
}
@@ -3244,14 +3249,14 @@ show_modifytable_info(ModifyTableState *mtstate, List *ancestors,
/* Should we explicitly label target relations? */
labeltargets = (mtstate->mt_nplans > 1 ||
(mtstate->mt_nplans == 1 &&
- mtstate->resultRelInfo->ri_RangeTableIndex != node->nominalRelation));
+ mtstate->resultRelInfos[0]->ri_RangeTableIndex != node->nominalRelation));
if (labeltargets)
ExplainOpenGroup("Target Tables", "Target Tables", false, es);
for (j = 0; j < mtstate->mt_nplans; j++)
{
- ResultRelInfo *resultRelInfo = mtstate->resultRelInfo + j;
+ ResultRelInfo *resultRelInfo = mtstate->resultRelInfos[j];
FdwRoutine *fdwroutine = resultRelInfo->ri_FdwRoutine;
if (labeltargets)
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index c13b1d3501..f84110a54c 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -470,7 +470,7 @@ ExecHashSubPlanResultRelsByOid(ModifyTableState *mtstate,
/* Hash all subplans by their Oid */
for (i = 0; i < mtstate->mt_nplans; i++)
{
- ResultRelInfo *rri = &mtstate->resultRelInfo[i];
+ ResultRelInfo *rri = mtstate->resultRelInfos[i];
bool found;
Oid partoid = RelationGetRelid(rri->ri_RelationDesc);
SubplanResultRelHashElem *elem;
@@ -507,7 +507,7 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
ModifyTable *node = (ModifyTable *) mtstate->ps.plan;
Relation rootrel = rootResultRelInfo->ri_RelationDesc,
partrel;
- Relation firstResultRel = mtstate->resultRelInfo[0].ri_RelationDesc;
+ Relation firstResultRel = mtstate->resultRelInfos[0]->ri_RelationDesc;
ResultRelInfo *leaf_part_rri;
MemoryContext oldcxt;
AttrMap *part_attmap = NULL;
@@ -555,7 +555,7 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
List *wcoList;
List *wcoExprs = NIL;
ListCell *ll;
- int firstVarno = mtstate->resultRelInfo[0].ri_RangeTableIndex;
+ int firstVarno = mtstate->resultRelInfos[0]->ri_RangeTableIndex;
/*
* In the case of INSERT on a partitioned table, there is only one
@@ -619,7 +619,7 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
TupleTableSlot *slot;
ExprContext *econtext;
List *returningList;
- int firstVarno = mtstate->resultRelInfo[0].ri_RangeTableIndex;
+ int firstVarno = mtstate->resultRelInfos[0]->ri_RangeTableIndex;
/* See the comment above for WCO lists. */
Assert((node->operation == CMD_INSERT &&
@@ -678,7 +678,7 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
*/
if (node && node->onConflictAction != ONCONFLICT_NONE)
{
- int firstVarno = mtstate->resultRelInfo[0].ri_RangeTableIndex;
+ int firstVarno = mtstate->resultRelInfos[0]->ri_RangeTableIndex;
TupleDesc partrelDesc = RelationGetDescr(partrel);
ExprContext *econtext = mtstate->ps.ps_ExprContext;
ListCell *lc;
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index d71c0a4322..0d987e9651 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -57,6 +57,8 @@
#include "utils/memutils.h"
#include "utils/rel.h"
+ /* Special value for mt_whichplan */
+#define WHICHPLAN_CHOOSE_SUBPLAN -1
static bool ExecOnConflictUpdate(ModifyTableState *mtstate,
ResultRelInfo *resultRelInfo,
@@ -1198,7 +1200,6 @@ lreplace:;
TupleTableSlot *ret_slot;
TupleTableSlot *epqslot = NULL;
PartitionTupleRouting *proute = mtstate->mt_partition_tuple_routing;
- int map_index;
TupleConversionMap *tupconv_map;
/*
@@ -1277,17 +1278,8 @@ lreplace:;
if (mtstate->mt_transition_capture)
saved_tcs_map = mtstate->mt_transition_capture->tcs_map;
- /*
- * resultRelInfo is one of the per-subplan resultRelInfos. So we
- * should convert the tuple into root's tuple descriptor, since
- * ExecInsert() starts the search from root. The tuple conversion
- * map list is in the order of mtstate->resultRelInfo[], so to
- * retrieve the one for this resultRel, we need to know the
- * position of the resultRel in mtstate->resultRelInfo[].
- */
- map_index = resultRelInfo - mtstate->resultRelInfo;
- Assert(map_index >= 0 && map_index < mtstate->mt_nplans);
- tupconv_map = tupconv_map_for_subplan(mtstate, map_index);
+ /* Fetch the tuple conversion map for the current subplan */
+ tupconv_map = tupconv_map_for_subplan(mtstate, mtstate->mt_whichplan);
if (tupconv_map != NULL)
slot = execute_attr_map_slot(tupconv_map->attrMap,
slot,
@@ -1731,12 +1723,12 @@ static void
fireBSTriggers(ModifyTableState *node)
{
ModifyTable *plan = (ModifyTable *) node->ps.plan;
- ResultRelInfo *resultRelInfo = node->resultRelInfo;
+ ResultRelInfo *resultRelInfo = node->resultRelInfos[0];
/*
* If the node modifies a partitioned table, we must fire its triggers.
- * Note that in that case, node->resultRelInfo points to the first leaf
- * partition, not the root table.
+ * Note that in that case, node->resultRelInfos[0] points to the first
+ * leaf partition, not the root table.
*/
if (node->rootResultRelInfo != NULL)
resultRelInfo = node->rootResultRelInfo;
@@ -1774,13 +1766,14 @@ static ResultRelInfo *
getTargetResultRelInfo(ModifyTableState *node)
{
/*
- * Note that if the node modifies a partitioned table, node->resultRelInfo
- * points to the first leaf partition, not the root table.
+ * Note that if the node modifies a partitioned table,
+ * node->resultRelInfos[0] points to the first leaf partition, not the
+ * root table.
*/
if (node->rootResultRelInfo != NULL)
return node->rootResultRelInfo;
else
- return node->resultRelInfo;
+ return node->resultRelInfos[0];
}
/*
@@ -1959,7 +1952,7 @@ static void
ExecSetupChildParentMapForSubplan(ModifyTableState *mtstate)
{
ResultRelInfo *targetRelInfo = getTargetResultRelInfo(mtstate);
- ResultRelInfo *resultRelInfos = mtstate->resultRelInfo;
+ ResultRelInfo **resultRelInfos = mtstate->resultRelInfos;
TupleDesc outdesc;
int numResultRelInfos = mtstate->mt_nplans;
int i;
@@ -1979,7 +1972,7 @@ ExecSetupChildParentMapForSubplan(ModifyTableState *mtstate)
for (i = 0; i < numResultRelInfos; ++i)
{
mtstate->mt_per_subplan_tupconv_maps[i] =
- convert_tuples_by_name(RelationGetDescr(resultRelInfos[i].ri_RelationDesc),
+ convert_tuples_by_name(RelationGetDescr(resultRelInfos[i]->ri_RelationDesc),
outdesc);
}
}
@@ -2055,8 +2048,33 @@ ExecModifyTable(PlanState *pstate)
node->fireBSTriggers = false;
}
+
+ /* Handle choosing the valid subplans */
+ if (node->mt_whichplan == WHICHPLAN_CHOOSE_SUBPLAN)
+ {
+ PartitionPruneState *prunestate = node->mt_prune_state;
+
+ /* Bail if all partitions were run-time pruned */
+ if (node->mt_nplans == 0)
+ goto done;
+
+ /*
+ * mt_whichplan should never get set to WHICHPLAN_CHOOSE_SUBPLAN
+ * unless we're performing do_exec_prune
+ */
+ Assert(prunestate != NULL && prunestate->do_exec_prune);
+
+ /* Determine the minimum set of matching subplans */
+ node->mt_valid_subplans = ExecFindMatchingSubPlans(prunestate);
+ node->mt_whichplan = bms_next_member(node->mt_valid_subplans, -1);
+
+ /* If no subplan matches these params then we're done */
+ if (node->mt_whichplan < 0)
+ goto done;
+ }
+
/* Preload local variables */
- resultRelInfo = node->resultRelInfo + node->mt_whichplan;
+ resultRelInfo = node->resultRelInfos[node->mt_whichplan];
subplanstate = node->mt_plans[node->mt_whichplan];
junkfilter = resultRelInfo->ri_junkFilter;
@@ -2097,11 +2115,13 @@ ExecModifyTable(PlanState *pstate)
if (TupIsNull(planSlot))
{
- /* advance to next subplan if any */
- node->mt_whichplan++;
- if (node->mt_whichplan < node->mt_nplans)
+ /* advance to the next valid subplan if any */
+ node->mt_whichplan = bms_next_member(node->mt_valid_subplans,
+ node->mt_whichplan);
+
+ if (node->mt_whichplan >= 0)
{
- resultRelInfo++;
+ resultRelInfo = node->resultRelInfos[node->mt_whichplan];
subplanstate = node->mt_plans[node->mt_whichplan];
junkfilter = resultRelInfo->ri_junkFilter;
estate->es_result_relation_info = resultRelInfo;
@@ -2271,6 +2291,8 @@ ExecModifyTable(PlanState *pstate)
/* Restore es_result_relation_info before exiting */
estate->es_result_relation_info = saved_resultRelInfo;
+done:
+
/*
* We're done, but fire AFTER STATEMENT triggers before exiting.
*/
@@ -2293,9 +2315,11 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
int nplans = list_length(node->plans);
ResultRelInfo *saved_resultRelInfo;
ResultRelInfo *resultRelInfo;
+ Bitmapset *validsubplans;
Plan *subplan;
ListCell *l;
- int i;
+ int i,
+ j;
Relation rel;
bool update_tuple_routing_needed = node->partColsUpdated;
@@ -2314,8 +2338,74 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
mtstate->canSetTag = node->canSetTag;
mtstate->mt_done = false;
+ /* If run-time partition pruning is enabled, then set that up now */
+ if (node->part_prune_info != NULL)
+ {
+ PartitionPruneState *prunestate;
+
+ ExecAssignExprContext(estate, &mtstate->ps);
+
+ prunestate = ExecCreatePartitionPruneState(&mtstate->ps,
+ node->part_prune_info);
+ mtstate->mt_prune_state = prunestate;
+
+ /* Perform an initial partition prune, if required. */
+ if (prunestate->do_initial_prune)
+ {
+ /* Determine which subplans match the external params */
+ validsubplans = ExecFindInitialMatchingSubPlans(prunestate,
+ list_length(node->plans));
+
+ nplans = bms_num_members(validsubplans);
+ }
+ else
+ {
+ /* We'll need to initialize all subplans */
+ nplans = list_length(node->plans);
+ validsubplans = bms_add_range(NULL, 0, nplans - 1);
+ }
+
+ /*
+ * When no run-time pruning is required and there's at least one
+ * subplan, we can fill mt_valid_subplans immediately, preventing
+ * later calls to ExecFindMatchingSubPlans. Additionally, we can also
+ * select the first subplan to execute.
+ */
+ if (!prunestate->do_exec_prune && nplans > 0)
+ {
+ mtstate->mt_valid_subplans = bms_add_range(NULL, 0, nplans - 1);
+ mtstate->mt_whichplan = 0;
+ }
+ else
+ {
+ /*
+ * When performing do_exec_prune or we pruned all subplans above
+ * then we set mt_whichplan to WHICHPLAN_CHOOSE_SUBPLAN so that
+ * we properly handle those cases during execution.
+ */
+ mtstate->mt_valid_subplans = NULL;
+ mtstate->mt_whichplan = WHICHPLAN_CHOOSE_SUBPLAN;
+ }
+ }
+ else
+ {
+ nplans = list_length(node->plans);
+
+ /*
+ * When run-time partition pruning is not enabled we can just mark all
+ * plans as valid, they must also all be initialized.
+ */
+ validsubplans = bms_add_range(NULL, 0, nplans - 1);
+ mtstate->mt_valid_subplans = validsubplans;
+ mtstate->mt_prune_state = NULL;
+
+ /* Also set the plan to execute first */
+ mtstate->mt_whichplan = 0;
+ }
+
mtstate->mt_plans = (PlanState **) palloc0(sizeof(PlanState *) * nplans);
- mtstate->resultRelInfo = estate->es_result_relations + node->resultRelIndex;
+ mtstate->resultRelInfos = (ResultRelInfo **)
+ palloc(sizeof(ResultRelInfo *) * nplans);
mtstate->mt_scans = (TupleTableSlot **) palloc0(sizeof(TupleTableSlot *) * nplans);
/* If modifying a partitioned table, initialize the root table info */
@@ -2342,11 +2432,14 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
*/
saved_resultRelInfo = estate->es_result_relation_info;
- resultRelInfo = mtstate->resultRelInfo;
- i = 0;
- foreach(l, node->plans)
+ j = 0;
+ i = -1;
+ while ((i = bms_next_member(validsubplans, i)) >= 0)
{
- subplan = (Plan *) lfirst(l);
+ subplan = (Plan *) list_nth(node->plans, i);
+
+ resultRelInfo = &estate->es_result_relations[node->resultRelIndex + i];
+ mtstate->resultRelInfos[j] = resultRelInfo;
/* Initialize the usesFdwDirectModify flag */
resultRelInfo->ri_usesFdwDirectModify = bms_is_member(i,
@@ -2384,9 +2477,9 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
/* Now init the plan for this result rel */
estate->es_result_relation_info = resultRelInfo;
- mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
- mtstate->mt_scans[i] =
- ExecInitExtraTupleSlot(mtstate->ps.state, ExecGetResultType(mtstate->mt_plans[i]),
+ mtstate->mt_plans[j] = ExecInitNode(subplan, estate, eflags);
+ mtstate->mt_scans[j] =
+ ExecInitExtraTupleSlot(mtstate->ps.state, ExecGetResultType(mtstate->mt_plans[j]),
table_slot_callbacks(resultRelInfo->ri_RelationDesc));
/* Also let FDWs init themselves for foreign-table result rels */
@@ -2402,9 +2495,7 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
i,
eflags);
}
-
- resultRelInfo++;
- i++;
+ j++;
}
estate->es_result_relation_info = saved_resultRelInfo;
@@ -2453,33 +2544,35 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
/*
* Initialize any WITH CHECK OPTION constraints if needed.
*/
- resultRelInfo = mtstate->resultRelInfo;
- i = 0;
- foreach(l, node->withCheckOptionLists)
+ if (node->withCheckOptionLists != NIL)
{
- List *wcoList = (List *) lfirst(l);
- List *wcoExprs = NIL;
- ListCell *ll;
-
- foreach(ll, wcoList)
+ j = 0;
+ i = -1;
+ while ((i = bms_next_member(validsubplans, i)) >= 0)
{
- WithCheckOption *wco = (WithCheckOption *) lfirst(ll);
- ExprState *wcoExpr = ExecInitQual((List *) wco->qual,
- &mtstate->ps);
+ List *wcoList = (List *) list_nth(node->withCheckOptionLists, i);;
+ List *wcoExprs = NIL;
- wcoExprs = lappend(wcoExprs, wcoExpr);
- }
+ foreach(l, wcoList)
+ {
+ WithCheckOption *wco = (WithCheckOption *) lfirst(l);
+ ExprState *wcoExpr = ExecInitQual((List *) wco->qual,
+ &mtstate->ps);
+
+ wcoExprs = lappend(wcoExprs, wcoExpr);
+ }
- resultRelInfo->ri_WithCheckOptions = wcoList;
- resultRelInfo->ri_WithCheckOptionExprs = wcoExprs;
- resultRelInfo++;
- i++;
+ resultRelInfo = mtstate->resultRelInfos[j];
+ resultRelInfo->ri_WithCheckOptions = wcoList;
+ resultRelInfo->ri_WithCheckOptionExprs = wcoExprs;
+ j++;
+ }
}
/*
* Initialize RETURNING projections if needed.
*/
- if (node->returningLists)
+ if (node->returningLists != NIL)
{
TupleTableSlot *slot;
ExprContext *econtext;
@@ -2502,16 +2595,18 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
/*
* Build a projection for each result rel.
*/
- resultRelInfo = mtstate->resultRelInfo;
- foreach(l, node->returningLists)
+ j = 0;
+ i = -1;
+ while ((i = bms_next_member(validsubplans, i)) >= 0)
{
- List *rlist = (List *) lfirst(l);
+ List *rlist = (List *) list_nth(node->returningLists, i);
+ resultRelInfo = mtstate->resultRelInfos[j];
resultRelInfo->ri_returningList = rlist;
resultRelInfo->ri_projectReturning =
ExecBuildProjectionInfo(rlist, econtext, slot, &mtstate->ps,
resultRelInfo->ri_RelationDesc->rd_att);
- resultRelInfo++;
+ j++;
}
}
else
@@ -2522,12 +2617,10 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
*/
mtstate->ps.plan->targetlist = NIL;
ExecInitResultTypeTL(&mtstate->ps);
-
- mtstate->ps.ps_ExprContext = NULL;
}
/* Set the list of arbiter indexes if needed for ON CONFLICT */
- resultRelInfo = mtstate->resultRelInfo;
+ resultRelInfo = mtstate->resultRelInfos[0];
if (node->onConflictAction != ONCONFLICT_NONE)
resultRelInfo->ri_onConflictArbiterIndexes = node->arbiterIndexes;
@@ -2618,7 +2711,6 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
}
/* select first subplan */
- mtstate->mt_whichplan = 0;
subplan = (Plan *) linitial(node->plans);
EvalPlanQualSetPlan(&mtstate->mt_epqstate, subplan,
mtstate->mt_arowmarks[0]);
@@ -2667,12 +2759,12 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
if (junk_filter_needed)
{
- resultRelInfo = mtstate->resultRelInfo;
for (i = 0; i < nplans; i++)
{
JunkFilter *j;
TupleTableSlot *junkresslot;
+ resultRelInfo = mtstate->resultRelInfos[i];
subplan = mtstate->mt_plans[i]->plan;
if (operation == CMD_INSERT || operation == CMD_UPDATE)
ExecCheckPlanOutput(resultRelInfo->ri_RelationDesc,
@@ -2715,13 +2807,12 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
}
resultRelInfo->ri_junkFilter = j;
- resultRelInfo++;
}
}
else
{
if (operation == CMD_INSERT)
- ExecCheckPlanOutput(mtstate->resultRelInfo->ri_RelationDesc,
+ ExecCheckPlanOutput(mtstate->resultRelInfos[0]->ri_RelationDesc,
subplan->targetlist);
}
}
@@ -2760,7 +2851,7 @@ ExecEndModifyTable(ModifyTableState *node)
*/
for (i = 0; i < node->mt_nplans; i++)
{
- ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
+ ResultRelInfo *resultRelInfo = node->resultRelInfos[i];
if (!resultRelInfo->ri_usesFdwDirectModify &&
resultRelInfo->ri_FdwRoutine != NULL &&
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index eaab97f753..8270bf53eb 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -216,6 +216,7 @@ _copyModifyTable(const ModifyTable *from)
COPY_BITMAPSET_FIELD(fdwDirectModifyPlans);
COPY_NODE_FIELD(rowMarks);
COPY_SCALAR_FIELD(epqParam);
+ COPY_NODE_FIELD(part_prune_info);
COPY_SCALAR_FIELD(onConflictAction);
COPY_NODE_FIELD(arbiterIndexes);
COPY_NODE_FIELD(onConflictSet);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index e084c3f069..0e9da7f086 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -417,6 +417,7 @@ _outModifyTable(StringInfo str, const ModifyTable *node)
WRITE_BITMAPSET_FIELD(fdwDirectModifyPlans);
WRITE_NODE_FIELD(rowMarks);
WRITE_INT_FIELD(epqParam);
+ WRITE_NODE_FIELD(part_prune_info);
WRITE_ENUM_FIELD(onConflictAction, OnConflictAction);
WRITE_NODE_FIELD(arbiterIndexes);
WRITE_NODE_FIELD(onConflictSet);
@@ -2095,6 +2096,7 @@ _outModifyTablePath(StringInfo str, const ModifyTablePath *node)
WRITE_UINT_FIELD(rootRelation);
WRITE_BOOL_FIELD(partColsUpdated);
WRITE_NODE_FIELD(resultRelations);
+ WRITE_NODE_FIELD(partitioned_rels);
WRITE_NODE_FIELD(subpaths);
WRITE_NODE_FIELD(subroots);
WRITE_NODE_FIELD(withCheckOptionLists);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index d5b23a3479..03ecf56fef 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -1647,6 +1647,7 @@ _readModifyTable(void)
READ_BITMAPSET_FIELD(fdwDirectModifyPlans);
READ_NODE_FIELD(rowMarks);
READ_INT_FIELD(epqParam);
+ READ_NODE_FIELD(part_prune_info);
READ_ENUM_FIELD(onConflictAction, OnConflictAction);
READ_NODE_FIELD(arbiterIndexes);
READ_NODE_FIELD(onConflictSet);
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index fc25908dc6..c1096023c6 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -289,7 +289,8 @@ static ModifyTable *make_modifytable(PlannerInfo *root,
bool partColsUpdated,
List *resultRelations, List *subplans, List *subroots,
List *withCheckOptionLists, List *returningLists,
- List *rowMarks, OnConflictExpr *onconflict, int epqParam);
+ List *rowMarks, OnConflictExpr *onconflict, int epqParam,
+ PartitionPruneInfo *partpruneinfos);
static GatherMerge *create_gather_merge_plan(PlannerInfo *root,
GatherMergePath *best_path);
@@ -1236,6 +1237,7 @@ create_append_plan(PlannerInfo *root, AppendPath *best_path, int flags)
if (prunequal != NIL)
partpruneinfo =
make_partition_pruneinfo(root, rel,
+ NIL,
best_path->subpaths,
best_path->partitioned_rels,
prunequal);
@@ -1402,6 +1404,7 @@ create_merge_append_plan(PlannerInfo *root, MergeAppendPath *best_path,
if (prunequal != NIL)
partpruneinfo = make_partition_pruneinfo(root, rel,
+ NIL,
best_path->subpaths,
best_path->partitioned_rels,
prunequal);
@@ -2587,6 +2590,7 @@ static ModifyTable *
create_modifytable_plan(PlannerInfo *root, ModifyTablePath *best_path)
{
ModifyTable *plan;
+ PartitionPruneInfo *partpruneinfos = NULL;
List *subplans = NIL;
ListCell *subpaths,
*subroots;
@@ -2618,6 +2622,30 @@ create_modifytable_plan(PlannerInfo *root, ModifyTablePath *best_path)
subplans = lappend(subplans, subplan);
}
+ if (enable_partition_pruning &&
+ best_path->partitioned_rels != NIL &&
+ !IS_DUMMY_MODIFYTABLE(best_path))
+ {
+ RelOptInfo *rel = best_path->path.parent;
+ List *prunequal = NIL;
+
+ prunequal = extract_actual_clauses(rel->baserestrictinfo, false);
+
+ /*
+ * If any quals exist, then these may be useful to allow us to perform
+ * further partition pruning during execution. We'll generate a
+ * PartitionPruneInfo for each partitioned rel to store these quals
+ * and allow translation of partition indexes into subpath indexes.
+ */
+ if (prunequal != NIL)
+ partpruneinfos = make_partition_pruneinfo(root,
+ best_path->path.parent,
+ best_path->resultRelations,
+ best_path->subpaths,
+ best_path->partitioned_rels,
+ prunequal);
+ }
+
plan = make_modifytable(root,
best_path->operation,
best_path->canSetTag,
@@ -2631,7 +2659,8 @@ create_modifytable_plan(PlannerInfo *root, ModifyTablePath *best_path)
best_path->returningLists,
best_path->rowMarks,
best_path->onconflict,
- best_path->epqParam);
+ best_path->epqParam,
+ partpruneinfos);
copy_generic_path_info(&plan->plan, &best_path->path);
@@ -6624,7 +6653,8 @@ make_modifytable(PlannerInfo *root,
bool partColsUpdated,
List *resultRelations, List *subplans, List *subroots,
List *withCheckOptionLists, List *returningLists,
- List *rowMarks, OnConflictExpr *onconflict, int epqParam)
+ List *rowMarks, OnConflictExpr *onconflict, int epqParam,
+ PartitionPruneInfo *partpruneinfos)
{
ModifyTable *node = makeNode(ModifyTable);
List *fdw_private_list;
@@ -6685,6 +6715,7 @@ make_modifytable(PlannerInfo *root,
node->returningLists = returningLists;
node->rowMarks = rowMarks;
node->epqParam = epqParam;
+ node->part_prune_info = partpruneinfos;
/*
* For each result relation that is a foreign table, allow the FDW to
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index b44efd6314..6e1300f792 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -1239,6 +1239,9 @@ inheritance_planner(PlannerInfo *root)
RangeTblEntry *parent_rte;
Bitmapset *parent_relids;
Query **parent_parses;
+ PlannerInfo *partition_root = NULL;
+ List *partitioned_rels = NIL;
+ bool dummy_modifytbl = false;
/* Should only get here for UPDATE or DELETE */
Assert(parse->commandType == CMD_UPDATE ||
@@ -1350,6 +1353,13 @@ inheritance_planner(PlannerInfo *root)
* expand_partitioned_rtentry for the UPDATE target.)
*/
root->partColsUpdated = subroot->partColsUpdated;
+
+ /*
+ * Save this for later so that we can enable run-time pruning on
+ * the partitioned table(s).
+ */
+ if (parent_rte->relkind == RELKIND_PARTITIONED_TABLE)
+ partition_root = subroot;
}
/*----------
@@ -1386,6 +1396,9 @@ inheritance_planner(PlannerInfo *root)
{
AppendRelInfo *appinfo = lfirst_node(AppendRelInfo, lc);
RangeTblEntry *child_rte;
+ int old_rti;
+ int new_rti;
+
/* append_rel_list contains all append rels; ignore others */
if (!bms_is_member(appinfo->parent_relid, parent_relids))
@@ -1400,11 +1413,18 @@ inheritance_planner(PlannerInfo *root)
/* and append it to the original rtable */
parse->rtable = lappend(parse->rtable, child_rte);
- /* remember child's index in the SELECT rtable */
- old_child_rtis = lappend_int(old_child_rtis, appinfo->child_relid);
+ old_rti = appinfo->child_relid;
+ new_rti = list_length(parse->rtable);
- /* and its new index in the final rtable */
- new_child_rtis = lappend_int(new_child_rtis, list_length(parse->rtable));
+ /* record RT indexes for any RTEs that have changed RT index */
+ if (old_rti != new_rti)
+ {
+ /* remember child's index in the SELECT rtable */
+ old_child_rtis = lappend_int(old_child_rtis, old_rti);
+
+ /* and its new index in the final rtable */
+ new_child_rtis = lappend_int(new_child_rtis, new_rti);
+ }
/* if child is itself partitioned, update parent_relids */
if (child_rte->inh)
@@ -1426,9 +1446,6 @@ inheritance_planner(PlannerInfo *root)
int old_child_rti = lfirst_int(lc);
int new_child_rti = lfirst_int(lc2);
- if (old_child_rti == new_child_rti)
- continue; /* nothing to do */
-
Assert(old_child_rti > new_child_rti);
ChangeVarNodes((Node *) child_appinfos,
@@ -1745,6 +1762,9 @@ inheritance_planner(PlannerInfo *root)
returningLists = list_make1(parse->returningList);
/* Disable tuple routing, too, just to be safe */
root->partColsUpdated = false;
+
+ /* Mark that we're performing a dummy modify table */
+ dummy_modifytbl = true;
}
else
{
@@ -1784,6 +1804,105 @@ inheritance_planner(PlannerInfo *root)
else
rowMarks = root->rowMarks;
+
+ /*
+ * When performing UPDATE/DELETE on a partitioned table, if the query has
+ * a WHERE clause which supports it, we may be able to perform run-time
+ * partition pruning. The following code makes use of PlannerInfo fields
+ * from the SELECT planning invocation we performed above. We must
+ * translate the RT indexes to allow us to use these in 'root'.
+ */
+ if (partition_root && !dummy_modifytbl)
+ {
+ RelOptInfo *parent_rel;
+ int *relid_map;
+ int i;
+
+ /* create a map to translate old RT indexes into new ones */
+ relid_map = palloc(sizeof(int) * partition_root->simple_rel_array_size);
+
+ /*
+ * initialize the map assuming everything is the same. We'll make
+ * the required adjustments by looking at the old_child_rtis and
+ * new_child_rtis lists shortly.
+ */
+ for (i = 1; i < partition_root->simple_rel_array_size; i++)
+ relid_map[i] = i;
+
+ /*
+ * Fetch the target partitioned table's RelOptInfo. This may have
+ * base quals which we can use for run-time pruning.
+ */
+ parent_rel = partition_root->simple_rel_array[top_parentRTindex];
+
+ final_rel->baserestrictinfo = parent_rel->baserestrictinfo;
+
+ /* adjust varnos of any RTEs that have changed their RT index */
+ forboth(lc, old_child_rtis, lc2, new_child_rtis)
+ {
+ int old_child_rti = lfirst_int(lc);
+ int new_child_rti = lfirst_int(lc2);
+
+ Assert(old_child_rti > new_child_rti);
+
+ ChangeVarNodes((Node *) final_rel->baserestrictinfo,
+ old_child_rti, new_child_rti, 0);
+
+ /*
+ * While we're at it, adjust the map to account for the changed RT
+ * index.
+ */
+ relid_map[old_child_rti] = new_child_rti;
+ }
+
+ /*
+ * Build a list of each non-leaf partitioned rels which are being
+ * UPDATEd / DELETEd.
+ */
+ i = -1;
+ while ((i = bms_next_member(parent_relids, i)) > 0)
+ partitioned_rels = lappend_int(partitioned_rels, relid_map[i]);
+
+ /*
+ * In order to build the run-time pruning data we'll need append rels
+ * of any sub-partitioned tables. If there are some of those and the
+ * append_rel_array is not already allocated, then do that now.
+ */
+ if (list_length(partitioned_rels) > 1 &&
+ root->append_rel_array == NULL)
+ root->append_rel_array = palloc0(sizeof(AppendRelInfo *) *
+ root->simple_rel_array_size);
+
+ /*
+ * There can only be a single partition hierarchy, so it's fine to
+ * just make a single element list of the partitioned_rels.
+ */
+ partitioned_rels = list_make1(partitioned_rels);
+
+ i = -1;
+ while ((i = bms_next_member(parent_relids, i)) >= 0)
+ {
+ int new_rti = relid_map[i];
+
+ Assert(root->simple_rel_array[new_rti] == NULL);
+
+ root->simple_rel_array[new_rti] = partition_root->simple_rel_array[i];
+
+ /*
+ * The root partition won't have an append rel entry, so we can
+ * skip that. We'll need to take the partition_root's version for
+ * any sub-partitioned table's
+ */
+ if (i != top_parentRTindex)
+ {
+ Assert(root->append_rel_array[new_rti] == NULL);
+ root->append_rel_array[new_rti] = partition_root->append_rel_array[i];
+ }
+ }
+
+ pfree(relid_map);
+ }
+
/* Create Path representing a ModifyTable to do the UPDATE/DELETE work */
add_path(final_rel, (Path *)
create_modifytable_path(root, final_rel,
@@ -1793,6 +1912,7 @@ inheritance_planner(PlannerInfo *root)
rootRelation,
root->partColsUpdated,
resultRelations,
+ partitioned_rels,
subpaths,
subroots,
withCheckOptionLists,
@@ -2376,6 +2496,7 @@ grouping_planner(PlannerInfo *root, bool inheritance_update,
rootRelation,
false,
list_make1_int(parse->resultRelation),
+ NIL,
list_make1(path),
list_make1(root),
withCheckOptionLists,
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index d9ce516211..6fa7de546d 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3433,6 +3433,9 @@ create_lockrows_path(PlannerInfo *root, RelOptInfo *rel,
* 'partColsUpdated' is true if any partitioning columns are being updated,
* either from the target relation or a descendent partitioned table.
* 'resultRelations' is an integer list of actual RT indexes of target rel(s)
+ * 'partitioned_rels' is an integer list of RT indexes of non-leaf tables in
+ * the partition tree, if this is an UPDATE/DELETE to a partitioned table.
+ * Otherwise NIL.
* 'subpaths' is a list of Path(s) producing source data (one per rel)
* 'subroots' is a list of PlannerInfo structs (one per rel)
* 'withCheckOptionLists' is a list of WCO lists (one per rel)
@@ -3446,8 +3449,8 @@ create_modifytable_path(PlannerInfo *root, RelOptInfo *rel,
CmdType operation, bool canSetTag,
Index nominalRelation, Index rootRelation,
bool partColsUpdated,
- List *resultRelations, List *subpaths,
- List *subroots,
+ List *resultRelations, List *partitioned_rels,
+ List *subpaths, List *subroots,
List *withCheckOptionLists, List *returningLists,
List *rowMarks, OnConflictExpr *onconflict,
int epqParam)
@@ -3515,6 +3518,7 @@ create_modifytable_path(PlannerInfo *root, RelOptInfo *rel,
pathnode->rootRelation = rootRelation;
pathnode->partColsUpdated = partColsUpdated;
pathnode->resultRelations = resultRelations;
+ pathnode->partitioned_rels = list_copy(partitioned_rels);
pathnode->subpaths = subpaths;
pathnode->subroots = subroots;
pathnode->withCheckOptionLists = withCheckOptionLists;
diff --git a/src/backend/partitioning/partprune.c b/src/backend/partitioning/partprune.c
index eac52e6ec8..ecda23f878 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -214,7 +214,12 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
* 'parentrel' is the RelOptInfo for an appendrel, and 'subpaths' is the list
* of scan paths for its child rels.
*
- * 'partitioned_rels' is a List containing Lists of relids of partitioned
+ * If 'resultRelations' is non-NIL, then this List of RT indexes is used to
+ * build the mapping structures. Otherwise the 'subpaths' List is used. Both
+ * of these lists correspond to the planner node's subplans list that we're
+ * building the PartitionPruneInfo for.
+ *
+ * 'partitioned_rels' is a List containing Lists of RT indexes of partitioned
* tables (a/k/a non-leaf partitions) that are parents of some of the child
* rels. Here we attempt to populate the PartitionPruneInfo by adding a
* 'prune_infos' item for each sublist in the 'partitioned_rels' list.
@@ -228,8 +233,8 @@ static void partkey_datum_from_expr(PartitionPruneContext *context,
*/
PartitionPruneInfo *
make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
- List *subpaths, List *partitioned_rels,
- List *prunequal)
+ List *resultRelations, List *subpaths,
+ List *partitioned_rels, List *prunequal)
{
PartitionPruneInfo *pruneinfo;
Bitmapset *allmatchedsubplans = NULL;
@@ -246,21 +251,39 @@ make_partition_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
relid_subplan_map = palloc0(sizeof(int) * root->simple_rel_array_size);
/*
- * relid_subplan_map maps relid of a leaf partition to the index in
- * 'subpaths' of the scan plan for that partition.
+ * If 'resultRelations' are present then map these, otherwise, we map the
+ * 'subpaths' List. Later, this will allow us to map partition indexes
+ * into the executor node's subpath list index during query execution.
*/
- i = 1;
- foreach(lc, subpaths)
+ if (resultRelations != NIL)
{
- Path *path = (Path *) lfirst(lc);
- RelOptInfo *pathrel = path->parent;
+ i = 1;
+ foreach(lc, resultRelations)
+ {
+ int resultrel = lfirst_int(lc);
- Assert(IS_SIMPLE_REL(pathrel));
- Assert(pathrel->relid < root->simple_rel_array_size);
- /* No duplicates please */
- Assert(relid_subplan_map[pathrel->relid] == 0);
+ Assert(resultrel < root->simple_rel_array_size);
+ /* No duplicates please */
+ Assert(relid_subplan_map[resultrel] == 0);
+
+ relid_subplan_map[resultrel] = i++;
+ }
+ }
+ else
+ {
+ i = 1;
+ foreach(lc, subpaths)
+ {
+ Path *path = (Path *) lfirst(lc);
+ RelOptInfo *pathrel = path->parent;
+
+ Assert(IS_SIMPLE_REL(pathrel));
+ Assert(pathrel->relid < root->simple_rel_array_size);
+ /* No duplicates please */
+ Assert(relid_subplan_map[pathrel->relid] == 0);
- relid_subplan_map[pathrel->relid] = i++;
+ relid_subplan_map[pathrel->relid] = i++;
+ }
}
/* We now build a PartitionedRelPruneInfo for each partitioned rel. */
@@ -404,9 +427,12 @@ make_partitionedrel_pruneinfo(PlannerInfo *root, RelOptInfo *parentrel,
* an adjust_appendrel_attrs step. But it might not be, and then
* we have to translate. We update the prunequal parameter here,
* because in later iterations of the loop for child partitions,
- * we want to translate from parent to child variables.
+ * we want to translate from parent to child variables. We don't
+ * need to do this when planning a non-SELECT as, in that case,
+ * we're only working with a single partition hierarchy.
*/
- if (!bms_equal(parentrel->relids, subpart->relids))
+ if (root->parse->commandType == CMD_SELECT &&
+ !bms_equal(parentrel->relids, subpart->relids))
{
int nappinfos;
AppendRelInfo **appinfos = find_appinfos_by_relids(root,
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index cd3ddf781f..eb0975f47d 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1169,7 +1169,7 @@ typedef struct ModifyTableState
int mt_whichplan; /* which one is being executed (0..n-1) */
TupleTableSlot **mt_scans; /* input tuple corresponding to underlying
* plans */
- ResultRelInfo *resultRelInfo; /* per-subplan target relations */
+ ResultRelInfo **resultRelInfos; /* per-subplan target relations */
ResultRelInfo *rootResultRelInfo; /* root target relation (partitioned
* table root) */
List **mt_arowmarks; /* per-subplan ExecAuxRowMark lists */
@@ -1193,6 +1193,14 @@ typedef struct ModifyTableState
/* Per plan map for tuple conversion from child to root */
TupleConversionMap **mt_per_subplan_tupconv_maps;
+
+ /*
+ * Details required to allow partitions to be eliminated from the scan, or
+ * NULL if not possible.
+ */
+ struct PartitionPruneState *mt_prune_state;
+ Bitmapset *mt_valid_subplans; /* for runtime pruning, valid mt_plans
+ * indexes to execute. */
} ModifyTableState;
/* ----------------
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index 0ceb809644..31b0f8fde0 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -1779,6 +1779,8 @@ typedef struct ModifyTablePath
Index rootRelation; /* Root RT index, if target is partitioned */
bool partColsUpdated; /* some part key in hierarchy updated */
List *resultRelations; /* integer list of RT indexes */
+ /* RT indexes of non-leaf tables in a partition tree */
+ List *partitioned_rels;
List *subpaths; /* Path(s) producing source data */
List *subroots; /* per-target-table PlannerInfos */
List *withCheckOptionLists; /* per-target-table WCO lists */
@@ -1788,6 +1790,11 @@ typedef struct ModifyTablePath
int epqParam; /* ID of Param for EvalPlanQual re-eval */
} ModifyTablePath;
+#define IS_DUMMY_MODIFYTABLE(p) \
+ (IsA((p), ModifyTablePath) && \
+ list_length((p)->subpaths) == 1 && \
+ IS_DUMMY_APPEND(linitial((p)->subpaths)))
+
/*
* LimitPath represents applying LIMIT/OFFSET restrictions
*/
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 4869fe7b6d..e350acd7ff 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -233,6 +233,8 @@ typedef struct ModifyTable
Bitmapset *fdwDirectModifyPlans; /* indices of FDW DM plans */
List *rowMarks; /* PlanRowMarks (non-locking only) */
int epqParam; /* ID of Param for EvalPlanQual re-eval */
+ /* Info for run-time subplan pruning; NULL if we're not doing that */
+ struct PartitionPruneInfo *part_prune_info;
OnConflictAction onConflictAction; /* ON CONFLICT action */
List *arbiterIndexes; /* List of ON CONFLICT arbiter index OIDs */
List *onConflictSet; /* SET for INSERT ON CONFLICT DO UPDATE */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index e450fe112a..e6e1240135 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -254,8 +254,9 @@ extern ModifyTablePath *create_modifytable_path(PlannerInfo *root,
CmdType operation, bool canSetTag,
Index nominalRelation, Index rootRelation,
bool partColsUpdated,
- List *resultRelations, List *subpaths,
- List *subroots,
+ List *resultRelations,
+ List *partitioned_rels,
+ List *subpaths, List *subroots,
List *withCheckOptionLists, List *returningLists,
List *rowMarks, OnConflictExpr *onconflict,
int epqParam);
diff --git a/src/include/partitioning/partprune.h b/src/include/partitioning/partprune.h
index babdad2c3e..92bf3522a2 100644
--- a/src/include/partitioning/partprune.h
+++ b/src/include/partitioning/partprune.h
@@ -70,6 +70,7 @@ typedef struct PartitionPruneContext
extern PartitionPruneInfo *make_partition_pruneinfo(struct PlannerInfo *root,
struct RelOptInfo *parentrel,
+ List *resultRelations,
List *subpaths,
List *partitioned_rels,
List *prunequal);
diff --git a/src/test/regress/expected/insert_conflict.out b/src/test/regress/expected/insert_conflict.out
index 1338b2b23e..b6a13fa596 100644
--- a/src/test/regress/expected/insert_conflict.out
+++ b/src/test/regress/expected/insert_conflict.out
@@ -211,6 +211,7 @@ explain (costs off, format json) insert into insertconflicttest values (0, 'Bilb
"Conflict Resolution": "UPDATE", +
"Conflict Arbiter Indexes": ["key_index"], +
"Conflict Filter": "(insertconflicttest.fruit <> 'Lime'::text)",+
+ "Subplans Removed": 0, +
"Plans": [ +
{ +
"Node Type": "Result", +
diff --git a/src/test/regress/expected/partition_prune.out b/src/test/regress/expected/partition_prune.out
index 9c8f80da87..33afaa03f6 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -3196,6 +3196,107 @@ explain (analyze, costs off, summary off, timing off) select * from ma_test wher
reset enable_seqscan;
reset enable_sort;
+--
+-- Test run-time pruning of ModifyTable subnodes
+--
+-- Ensure only ma_test_p3 is scanned.
+explain (analyze, costs off, summary off, timing off) delete from ma_test where a = (select 29);
+ QUERY PLAN
+----------------------------------------------------------------
+ Delete on ma_test (actual rows=0 loops=1)
+ Delete on ma_test_p1 ma_test_1
+ Delete on ma_test_p2 ma_test_2
+ Delete on ma_test_p3 ma_test_3
+ InitPlan 1 (returns $0)
+ -> Result (actual rows=1 loops=1)
+ -> Seq Scan on ma_test_p1 ma_test_1 (never executed)
+ Filter: (a = $0)
+ -> Seq Scan on ma_test_p2 ma_test_2 (never executed)
+ Filter: (a = $0)
+ -> Seq Scan on ma_test_p3 ma_test_3 (actual rows=1 loops=1)
+ Filter: (a = $0)
+ Rows Removed by Filter: 9
+(13 rows)
+
+-- Ensure no partitions are scanned.
+explain (analyze, costs off, summary off, timing off) delete from ma_test where a = (select 30);
+ QUERY PLAN
+---------------------------------------------------------
+ Delete on ma_test (actual rows=0 loops=1)
+ Delete on ma_test_p1 ma_test_1
+ Delete on ma_test_p2 ma_test_2
+ Delete on ma_test_p3 ma_test_3
+ InitPlan 1 (returns $0)
+ -> Result (actual rows=1 loops=1)
+ -> Seq Scan on ma_test_p1 ma_test_1 (never executed)
+ Filter: (a = $0)
+ -> Seq Scan on ma_test_p2 ma_test_2 (never executed)
+ Filter: (a = $0)
+ -> Seq Scan on ma_test_p3 ma_test_3 (never executed)
+ Filter: (a = $0)
+(12 rows)
+
+-- Ensure partition pruning works with an update of the partition key.
+explain (analyze, costs off, summary off, timing off) update ma_test set a = 29 where a = (select 1);
+ QUERY PLAN
+----------------------------------------------------------------
+ Update on ma_test (actual rows=0 loops=1)
+ Update on ma_test_p1 ma_test_1
+ Update on ma_test_p2 ma_test_2
+ Update on ma_test_p3 ma_test_3
+ InitPlan 1 (returns $0)
+ -> Result (actual rows=1 loops=1)
+ -> Seq Scan on ma_test_p1 ma_test_1 (actual rows=1 loops=1)
+ Filter: (a = $0)
+ Rows Removed by Filter: 9
+ -> Seq Scan on ma_test_p2 ma_test_2 (never executed)
+ Filter: (a = $0)
+ -> Seq Scan on ma_test_p3 ma_test_3 (never executed)
+ Filter: (a = $0)
+(13 rows)
+
+-- Verify the above command
+select tableoid::regclass,a from ma_test where a = 29;
+ tableoid | a
+------------+----
+ ma_test_p3 | 29
+(1 row)
+
+truncate ma_test;
+prepare mt_q1 (int) as
+delete from ma_test where a > $1;
+set plan_cache_mode = force_generic_plan;
+explain (analyze, costs off, summary off, timing off) execute mt_q1(15);
+ QUERY PLAN
+----------------------------------------------------------------
+ Delete on ma_test (actual rows=0 loops=1)
+ Delete on ma_test_p2 ma_test_1
+ Delete on ma_test_p3 ma_test_2
+ Subplans Removed: 1
+ -> Seq Scan on ma_test_p2 ma_test_1 (actual rows=0 loops=1)
+ Filter: (a > $1)
+ -> Seq Scan on ma_test_p3 ma_test_2 (actual rows=0 loops=1)
+ Filter: (a > $1)
+(8 rows)
+
+explain (analyze, costs off, summary off, timing off) execute mt_q1(25);
+ QUERY PLAN
+----------------------------------------------------------------
+ Delete on ma_test (actual rows=0 loops=1)
+ Delete on ma_test_p3 ma_test_1
+ Subplans Removed: 2
+ -> Seq Scan on ma_test_p3 ma_test_1 (actual rows=0 loops=1)
+ Filter: (a > $1)
+(5 rows)
+
+-- Ensure ModifyTable behaves correctly when no subplans match exec params
+explain (analyze, costs off, summary off, timing off) execute mt_q1(35);
+ QUERY PLAN
+-------------------------------------------
+ Delete on ma_test (actual rows=0 loops=1)
+ Subplans Removed: 3
+(2 rows)
+
drop table ma_test;
reset enable_indexonlyscan;
--
diff --git a/src/test/regress/sql/partition_prune.sql b/src/test/regress/sql/partition_prune.sql
index d9daba3af3..047ed00ed3 100644
--- a/src/test/regress/sql/partition_prune.sql
+++ b/src/test/regress/sql/partition_prune.sql
@@ -851,6 +851,34 @@ explain (analyze, costs off, summary off, timing off) select * from ma_test wher
reset enable_seqscan;
reset enable_sort;
+--
+-- Test run-time pruning of ModifyTable subnodes
+--
+
+-- Ensure only ma_test_p3 is scanned.
+explain (analyze, costs off, summary off, timing off) delete from ma_test where a = (select 29);
+
+-- Ensure no partitions are scanned.
+explain (analyze, costs off, summary off, timing off) delete from ma_test where a = (select 30);
+
+-- Ensure partition pruning works with an update of the partition key.
+explain (analyze, costs off, summary off, timing off) update ma_test set a = 29 where a = (select 1);
+
+-- Verify the above command
+select tableoid::regclass,a from ma_test where a = 29;
+
+truncate ma_test;
+
+prepare mt_q1 (int) as
+delete from ma_test where a > $1;
+
+set plan_cache_mode = force_generic_plan;
+
+explain (analyze, costs off, summary off, timing off) execute mt_q1(15);
+explain (analyze, costs off, summary off, timing off) execute mt_q1(25);
+-- Ensure ModifyTable behaves correctly when no subplans match exec params
+explain (analyze, costs off, summary off, timing off) execute mt_q1(35);
+
drop table ma_test;
reset enable_indexonlyscan;
On Tue, 10 Mar 2020 at 00:13, David Rowley <dgrowleyml@gmail.com> wrote:
Over in inheritance_planner(), I noticed that the RT index of the
SELECT query and the UPDATE/DELETE query can differ. There was some
code that performed translations. I changed that code slightly so that
it's a bit more optimal. It was building two lists, one for the old
RT index and one for the new. It added elements to this list
regardless of if the RT indexes were the same or not. I've now changed
that to only add to the list if they differ, which I feel should never
be slower and most likely always faster. I'm also now building a
translation map between the old and new RT indexes, however, I only
found one test in the regression tests which require any sort of
translation of these RT indexes. This was with an inheritance table,
so I need to do a bit more work to find a case where this happens with
a partitioned table to ensure all this works.
I had a closer look at this today and the code I have in
inheritance_planner() is certainly not right.
It's pretty easy to made the SELECT and UPDATE/DELETE's RT indexes
differ with something like:
drop table part_t cascade;
create table part_t (a int, b int, c int) partition by list (a);
create table part_t12 partition of part_t for values in(1,2) partition
by list (a);
create table part_t12_1 partition of part_t12 for values in(1);
create table part_t12_2 partition of part_t12 for values in(2);
create table part_t3 partition of part_t for values in(3);
create view vw_part_t as select * from part_t;
explain analyze update vw_part_t set a = t2.a +0 from part_t t2 where
t2.a = vw_part_t.a and vw_part_t.a = (select 1);
In this case, the sub-partitioned table changes RT index. I can't
just take the RelOptInfo's from the partition_root's simple_rel_array
and put them in the correct element in the root's simple_rel_array as
they RT indexes stored within also need to be translated.
I'll be having another look at this to see what the best fix is going to be.
David
David Rowley <dgrowleyml@gmail.com> writes:
I had a closer look at this today and the code I have in
inheritance_planner() is certainly not right.
Although I didn't get around to it for v13, there's still a plan on the
table for inheritance_planner() to get nuked from orbit [1]/messages/by-id/357.1550612935@sss.pgh.pa.us.
Maybe this improvement should be put on hold till that's done?
regards, tom lane
On Wed, 25 Mar 2020 at 13:00, Tom Lane <tgl@sss.pgh.pa.us> wrote:
David Rowley <dgrowleyml@gmail.com> writes:
I had a closer look at this today and the code I have in
inheritance_planner() is certainly not right.Although I didn't get around to it for v13, there's still a plan on the
table for inheritance_planner() to get nuked from orbit [1].Maybe this improvement should be put on hold till that's done?
Possibly. I'm not really wedded to the idea of getting it in. However,
it would really only be the inheritance planner part that would need
to be changed later. I don't think any of the other code would need to
be adjusted.
Amit shared his thoughts in [1]/messages/by-id/CA+HiwqGhD7ieKsv5+GkmHgs-XhP2DoUhtESVb3MU-4j14=G6LA@mail.gmail.com. If you'd rather I held off, then I will.
David
[1]: /messages/by-id/CA+HiwqGhD7ieKsv5+GkmHgs-XhP2DoUhtESVb3MU-4j14=G6LA@mail.gmail.com
Hi David,
Sorry I couldn't get to this sooner.
On Wed, Mar 25, 2020 at 9:49 AM David Rowley <dgrowleyml@gmail.com> wrote:
On Wed, 25 Mar 2020 at 13:00, Tom Lane <tgl@sss.pgh.pa.us> wrote:
David Rowley <dgrowleyml@gmail.com> writes:
I had a closer look at this today and the code I have in
inheritance_planner() is certainly not right.Although I didn't get around to it for v13, there's still a plan on the
table for inheritance_planner() to get nuked from orbit [1].Maybe this improvement should be put on hold till that's done?
Possibly. I'm not really wedded to the idea of getting it in. However,
it would really only be the inheritance planner part that would need
to be changed later. I don't think any of the other code would need to
be adjusted.Amit shared his thoughts in [1]. If you'd rather I held off, then I will.
David
[1] /messages/by-id/CA+HiwqGhD7ieKsv5+GkmHgs-XhP2DoUhtESVb3MU-4j14=G6LA@mail.gmail.com
Actually, I was saying in that email that the update/delete planning
overhaul being talked about will make the entirety of the
functionality this patch is adding, which is ModifyTable node being
able to prune its subplans based on run-time parameter values,
redundant. That's because, with the overhaul, there won't be multiple
subplans under ModifyTable, only one which would take care of any
pruning that's necessary.
What I did say in favor of this patch though is that it doesn not seem
that invasive, so maybe okay to get in for v13.
--
Thank you,
Amit Langote
EnterpriseDB: http://www.enterprisedb.com
On Wed, 25 Mar 2020 at 15:37, Amit Langote <amitlangote09@gmail.com> wrote:
Actually, I was saying in that email that the update/delete planning
overhaul being talked about will make the entirety of the
functionality this patch is adding, which is ModifyTable node being
able to prune its subplans based on run-time parameter values,
redundant. That's because, with the overhaul, there won't be multiple
subplans under ModifyTable, only one which would take care of any
pruning that's necessary.
Thanks for explaining. I've not read over any patch for that yet, so
wasn't aware of exactly what was planned.
With your explanation, I imagine some sort of Append / MergeAppend
that runs the query as if it were a SELECT, but each
Append/MergeAppend subnode is tagged somehow with an index of which
ModifyTable subnode that it belongs to. Basically, just one complete
plan, rather than a plan per ModifyTable subnode.
What I did say in favor of this patch though is that it doesn not seem
that invasive, so maybe okay to get in for v13.
Since it seems there's much less code that will be useful after the
rewrite than I thought, combined with the fact that I'm not entirely
sure the best way to reuse the partitioned table's RelOptInfo from the
SELECT's PlannerInfo, then I'm going to return this one with feedback.
David
On Tue, Apr 7, 2020 at 8:52 PM David Rowley <dgrowleyml@gmail.com> wrote:
On Wed, 25 Mar 2020 at 15:37, Amit Langote <amitlangote09@gmail.com> wrote:
Actually, I was saying in that email that the update/delete planning
overhaul being talked about will make the entirety of the
functionality this patch is adding, which is ModifyTable node being
able to prune its subplans based on run-time parameter values,
redundant. That's because, with the overhaul, there won't be multiple
subplans under ModifyTable, only one which would take care of any
pruning that's necessary.Thanks for explaining. I've not read over any patch for that yet, so
wasn't aware of exactly what was planned.With your explanation, I imagine some sort of Append / MergeAppend
that runs the query as if it were a SELECT, but each
Append/MergeAppend subnode is tagged somehow with an index of which
ModifyTable subnode that it belongs to. Basically, just one complete
plan, rather than a plan per ModifyTable subnode.
That's correct, although I don't think Append/MergeAppend will need to
look any different structurally, except its subnodes will need to
produce a targetlist member to identify partition/child for a given
output row. There will still be N result relations, but not the N
plans created separately for each, as inheritance_planner() currently
does.
What I did say in favor of this patch though is that it doesn not seem
that invasive, so maybe okay to get in for v13.Since it seems there's much less code that will be useful after the
rewrite than I thought, combined with the fact that I'm not entirely
sure the best way to reuse the partitioned table's RelOptInfo from the
SELECT's PlannerInfo, then I'm going to return this one with feedback.
That makes sense. I am thinking to spend some time working on this
early in PG 14 cycle.
--
Thank you,
Amit Langote
EnterpriseDB: http://www.enterprisedb.com