Optimization for updating foreign tables in Postgres FDW
Attached is a WIP patch for the following:
/*
* postgresPlanForeignModify
* Plan an insert/update/delete operation on a foreign table
*
* Note: currently, the plan tree generated for UPDATE/DELETE will always
* include a ForeignScan that retrieves ctids (using SELECT FOR UPDATE)
* and then the ModifyTable node will have to execute individual remote
* UPDATE/DELETE commands. If there are no local conditions or joins
* needed, it'd be better to let the scan node do UPDATE/DELETE RETURNING
* and then do nothing at ModifyTable. Room for future optimization ...
*/
In the patch postgresPlanForeignModify has been modified so that if, in
addition to the above condition, the followings are satisfied, then the
ForeignScan and ModifyTable node will work that way.
- There are no local BEFORE/AFTER triggers.
- In UPDATE it's safe to evaluate expressions to assign to the target
columns on the remote server.
Here is a simple performance test.
On remote side:
postgres=# create table t (id serial primary key, inserted timestamp
default clock_timestamp(), data text);
CREATE TABLE
postgres=# insert into t(data) select random() from generate_series(0,
99999);
INSERT 0 100000
postgres=# vacuum t;
VACUUM
On local side:
postgres=# create foreign table ft (id integer, inserted timestamp, data
text) server myserver options (table_name 't');
CREATE FOREIGN TABLE
Unpatched:
postgres=# explain analyze verbose delete from ft where id < 10000;
QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------
Delete on public.ft (cost=100.00..162.32 rows=910 width=6) (actual
time=1275.255..1275.255 rows=0 loops=1)
Remote SQL: DELETE FROM public.t WHERE ctid = $1
-> Foreign Scan on public.ft (cost=100.00..162.32 rows=910 width=6)
(actual time=1.180..52.095 rows=9999 loops=1)
Output: ctid
Remote SQL: SELECT ctid FROM public.t WHERE ((id < 10000)) FOR
UPDATE
Planning time: 0.112 ms
Execution time: 1275.733 ms
(7 rows)
Patched (Note that the DELETE command has been pushed down.):
postgres=# explain analyze verbose delete from ft where id < 10000;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------
Delete on public.ft (cost=100.00..162.32 rows=910 width=6) (actual
time=0.006..0.006 rows=0 loops=1)
-> Foreign Scan on public.ft (cost=100.00..162.32 rows=910 width=6)
(actual time=0.001..0.001 rows=0 loops=1)
Output: ctid
Remote SQL: DELETE FROM public.t WHERE ((id < 10000))
Planning time: 0.101 ms
Execution time: 8.808 ms
(6 rows)
I'll add this to the next CF. Comments are welcome.
Thanks,
Best regards,
Etsuro Fujita
Attachments:
postgres_fdw-update-v1.patchtext/plain; charset=Shift_JIS; name=postgres_fdw-update-v1.patchDownload
*** a/contrib/postgres_fdw/deparse.c
--- b/contrib/postgres_fdw/deparse.c
***************
*** 189,198 **** is_foreign_expr(PlannerInfo *root,
if (!foreign_expr_walker((Node *) expr, &glob_cxt, &loc_cxt))
return false;
- /* Expressions examined here should be boolean, ie noncollatable */
- Assert(loc_cxt.collation == InvalidOid);
- Assert(loc_cxt.state == FDW_COLLATE_NONE);
-
/*
* An expression which includes any mutable functions can't be sent over
* because its result is not stable. For example, sending now() remote
--- 189,194 ----
***************
*** 928,933 **** deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 924,982 ----
}
/*
+ * deparse remote UPDATE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparseDirectUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List *targetlist,
+ List *targetAttrs, List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ List *params_list = NIL;
+ deparse_expr_cxt context;
+ bool first;
+ ListCell *lc;
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = NULL;
+
+ appendStringInfoString(buf, "UPDATE ");
+ deparseRelation(buf, rel);
+ appendStringInfoString(buf, " SET ");
+
+ first = true;
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(targetlist, attnum);
+
+ if (!first)
+ appendStringInfoString(buf, ", ");
+ first = false;
+
+ deparseColumnRef(buf, rtindex, attnum, root);
+ appendStringInfo(buf, " = ");
+ deparseExpr((Expr *) tle->expr, &context);
+ }
+ if (remote_conds)
+ appendWhereClause(buf, root, baserel, remote_conds,
+ true, ¶ms_list);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* deparse remote DELETE statement
*
* The statement text is appended to buf, and we also create an integer List
***************
*** 950,955 **** deparseDeleteSql(StringInfo buf, PlannerInfo *root,
--- 999,1031 ----
}
/*
+ * deparse remote DELETE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparseDirectDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ List *params_list = NIL;
+
+ appendStringInfoString(buf, "DELETE FROM ");
+ deparseRelation(buf, rel);
+ if (remote_conds)
+ appendWhereClause(buf, root, baserel, remote_conds,
+ true, ¶ms_list);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* Add a RETURNING clause, if needed, to an INSERT/UPDATE/DELETE.
*/
static void
*** a/contrib/postgres_fdw/expected/postgres_fdw.out
--- b/contrib/postgres_fdw/expected/postgres_fdw.out
***************
*** 1124,1138 **** UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
! QUERY PLAN
! ----------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
- Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1 RETURNING "C 1", c4
-> Foreign Scan on public.ft2
Output: ctid
! Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) FOR UPDATE
! (6 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
--- 1124,1137 ----
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
! QUERY PLAN
! --------------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
-> Foreign Scan on public.ft2
Output: ctid
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) RETURNING "C 1", c4
! (5 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
***************
*** 2227,2233 **** CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 2226,2232 ----
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 2386,2392 **** savepoint s3;
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 2385,2391 ----
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (-2) WHERE ((c2 = 42)) AND (("C 1" = 10))
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
***************
*** 87,93 **** typedef struct PgFdwRelationInfo
* planner to executor. Currently we store:
*
* 1) SELECT statement text to be sent to the remote server
! * 2) Integer list of attribute numbers retrieved by the SELECT
*
* These items are indexed with the enum FdwScanPrivateIndex, so an item
* can be fetched with list_nth(). For example, to get the SELECT statement:
--- 87,98 ----
* planner to executor. Currently we store:
*
* 1) SELECT statement text to be sent to the remote server
! * 2) List of restriction clauses that can be executed remotely
! * 3) Integer list of attribute numbers retrieved by the SELECT
! * 4) UPDATE/DELETE statement text to be sent to the remote server
! * 5) Boolean flag showing if we set the command es_processed
! * 6) Boolean flag showing if the remote query has a RETURNING clause
! * 7) Integer list of attribute numbers retrieved by RETURNING, if any
*
* These items are indexed with the enum FdwScanPrivateIndex, so an item
* can be fetched with list_nth(). For example, to get the SELECT statement:
***************
*** 97,106 **** enum FdwScanPrivateIndex
{
/* SQL statement to execute remotely (as a String node) */
FdwScanPrivateSelectSql,
! /* Integer list of attribute numbers retrieved by the SELECT */
! FdwScanPrivateRetrievedAttrs
};
/*
* Similarly, this enum describes what's kept in the fdw_private list for
* a ModifyTable node referencing a postgres_fdw foreign table. We store:
--- 102,124 ----
{
/* SQL statement to execute remotely (as a String node) */
FdwScanPrivateSelectSql,
! /* List of restriction clauses that can be executed remotely */
! FdwScanPrivateRemoteConds,
! /* Integer list of attribute numbers retrieved by SELECT */
! FdwScanPrivateRetrievedAttrsBySelect,
! /* UPDATE/DELETE statement to execute remotely (as a String node) */
! FdwScanPrivateUpdateSql,
! /* set-processed flag (as an integer Value node) */
! FdwScanPrivateSetProcessed,
! /* has-returning flag (as an integer Value node) */
! FdwScanPrivateHasReturning,
! /* Integer list of attribute numbers retrieved by RETURNING */
! FdwScanPrivateRetrievedAttrsByReturning
};
+ #define MinFdwScanFdwPrivateLength 3
+ #define MaxFdwScanFdwPrivateLength 7
+
/*
* Similarly, this enum describes what's kept in the fdw_private list for
* a ModifyTable node referencing a postgres_fdw foreign table. We store:
***************
*** 132,139 **** typedef struct PgFdwScanState
AttInMetadata *attinmeta; /* attribute datatype conversion metadata */
/* extracted fdw_private data */
! char *query; /* text of SELECT command */
List *retrieved_attrs; /* list of retrieved attribute numbers */
/* for remote query execution */
PGconn *conn; /* connection for the scan */
--- 150,159 ----
AttInMetadata *attinmeta; /* attribute datatype conversion metadata */
/* extracted fdw_private data */
! char *query; /* text of SELECT or UPDATE/DELETE command */
List *retrieved_attrs; /* list of retrieved attribute numbers */
+ bool set_processed; /* do we set the command es_processed? */
+ bool has_returning; /* is there a RETURNING clause? */
/* for remote query execution */
PGconn *conn; /* connection for the scan */
***************
*** 153,158 **** typedef struct PgFdwScanState
--- 173,183 ----
int fetch_ct_2; /* Min(# of fetches done, 2) */
bool eof_reached; /* true if last fetch reached EOF */
+ /* for direct update */
+ bool direct_update; /* do we update the foreign table directly? */
+ PGresult *result; /* result of an UPDATE/DELETE query */;
+ TupleTableSlot *rslot; /* slot containing the result tuple */
+
/* working memory contexts */
MemoryContext batch_cxt; /* context holding current batch of tuples */
MemoryContext temp_cxt; /* context for per-tuple temporary data */
***************
*** 181,186 **** typedef struct PgFdwModifyState
--- 206,215 ----
int p_nums; /* number of parameters to transmit */
FmgrInfo *p_flinfo; /* output conversion functions for them */
+ /* for direct update */
+ bool direct_update; /* do we update the foreign table directly? */
+ PgFdwScanState *fsstate; /* execution state of a foreign scan */
+
/* working memory context */
MemoryContext temp_cxt; /* context for per-tuple temporary data */
} PgFdwModifyState;
***************
*** 307,318 **** static bool ec_member_matches_foreign(PlannerInfo *root, RelOptInfo *rel,
static void create_cursor(ForeignScanState *node);
static void fetch_more_data(ForeignScanState *node);
static void close_cursor(PGconn *conn, unsigned int cursor_number);
static void prepare_foreign_modify(PgFdwModifyState *fmstate);
static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
ItemPointer tupleid,
TupleTableSlot *slot);
! static void store_returning_result(PgFdwModifyState *fmstate,
! TupleTableSlot *slot, PGresult *res);
static int postgresAcquireSampleRowsFunc(Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
--- 336,364 ----
static void create_cursor(ForeignScanState *node);
static void fetch_more_data(ForeignScanState *node);
static void close_cursor(PGconn *conn, unsigned int cursor_number);
+ static bool check_direct_update(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs);
+ static List *plan_direct_update(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs);
static void prepare_foreign_modify(PgFdwModifyState *fmstate);
static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
ItemPointer tupleid,
TupleTableSlot *slot);
! static void store_returning_result(TupleTableSlot *slot,
! PGresult *res,
! int row,
! Relation rel,
! AttInMetadata *attinmeta,
! List *retrieved_attrs,
! MemoryContext temp_context);
static int postgresAcquireSampleRowsFunc(Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
***************
*** 848,854 **** postgresGetForeignPlan(PlannerInfo *root,
* Build the fdw_private list that will be available to the executor.
* Items in the list must match enum FdwScanPrivateIndex, above.
*/
! fdw_private = list_make2(makeString(sql.data),
retrieved_attrs);
/*
--- 894,901 ----
* Build the fdw_private list that will be available to the executor.
* Items in the list must match enum FdwScanPrivateIndex, above.
*/
! fdw_private = list_make3(makeString(sql.data),
! remote_conds,
retrieved_attrs);
/*
***************
*** 910,931 **** postgresBeginForeignScan(ForeignScanState *node, int eflags)
server = GetForeignServer(table->serverid);
user = GetUserMapping(userid, server->serverid);
/*
* Get connection to the foreign server. Connection manager will
* establish new connection if necessary.
*/
fsstate->conn = GetConnection(server, user, false);
- /* Assign a unique ID for my cursor */
- fsstate->cursor_number = GetCursorNumber(fsstate->conn);
- fsstate->cursor_exists = false;
-
- /* Get private info created by planner functions. */
- fsstate->query = strVal(list_nth(fsplan->fdw_private,
- FdwScanPrivateSelectSql));
- fsstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
- FdwScanPrivateRetrievedAttrs);
-
/* Create contexts for batches of tuples and per-tuple temp workspace. */
fsstate->batch_cxt = AllocSetContextCreate(estate->es_query_cxt,
"postgres_fdw tuple data",
--- 957,996 ----
server = GetForeignServer(table->serverid);
user = GetUserMapping(userid, server->serverid);
+ /* Get private info created by planner functions. */
+ if (list_length(fsplan->fdw_private) >= MaxFdwScanFdwPrivateLength)
+ {
+ Assert(list_length(fsplan->fdw_private) == MaxFdwScanFdwPrivateLength);
+
+ fsstate->query = strVal(list_nth(fsplan->fdw_private,
+ FdwScanPrivateUpdateSql));
+ fsstate->set_processed = intVal(list_nth(fsplan->fdw_private,
+ FdwScanPrivateSetProcessed));
+ fsstate->has_returning = intVal(list_nth(fsplan->fdw_private,
+ FdwScanPrivateHasReturning));
+ fsstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
+ FdwScanPrivateRetrievedAttrsByReturning);
+
+ fsstate->direct_update = true;
+ }
+ else
+ {
+ Assert(list_length(fsplan->fdw_private) == MinFdwScanFdwPrivateLength);
+
+ fsstate->query = strVal(list_nth(fsplan->fdw_private,
+ FdwScanPrivateSelectSql));
+ fsstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
+ FdwScanPrivateRetrievedAttrsBySelect);
+
+ fsstate->direct_update = false;
+ }
+
/*
* Get connection to the foreign server. Connection manager will
* establish new connection if necessary.
*/
fsstate->conn = GetConnection(server, user, false);
/* Create contexts for batches of tuples and per-tuple temp workspace. */
fsstate->batch_cxt = AllocSetContextCreate(estate->es_query_cxt,
"postgres_fdw tuple data",
***************
*** 941,946 **** postgresBeginForeignScan(ForeignScanState *node, int eflags)
--- 1006,1037 ----
/* Get info we'll need for input data conversion. */
fsstate->attinmeta = TupleDescGetAttInMetadata(RelationGetDescr(fsstate->rel));
+ if (fsstate->direct_update)
+ {
+ /*
+ * Execute the update statement, and check for success.
+ *
+ * We don't use a PG_TRY block here, so be careful not to throw error
+ * without releasing the PGresult.
+ */
+ fsstate->result = PQexec(fsstate->conn, fsstate->query);
+ if (PQresultStatus(fsstate->result) !=
+ (fsstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
+ pgfdw_report_error(ERROR, fsstate->result, fsstate->conn, true,
+ fsstate->query);
+
+ /* Check number of rows affected. */
+ if (fsstate->has_returning)
+ fsstate->num_tuples = PQntuples(fsstate->result);
+ else
+ fsstate->num_tuples = atoi(PQcmdTuples(fsstate->result));
+ return;
+ }
+
+ /* Assign a unique ID for my cursor */
+ fsstate->cursor_number = GetCursorNumber(fsstate->conn);
+ fsstate->cursor_exists = false;
+
/* Prepare for output conversion of parameters used in remote query. */
numParams = list_length(fsplan->fdw_exprs);
fsstate->numParams = numParams;
***************
*** 990,995 **** postgresIterateForeignScan(ForeignScanState *node)
--- 1081,1129 ----
PgFdwScanState *fsstate = (PgFdwScanState *) node->fdw_state;
TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+ if (fsstate->direct_update)
+ {
+ MemoryContext oldcontext;
+
+ if (!fsstate->has_returning)
+ {
+ EState *estate = node->ss.ps.state;
+
+ if (fsstate->set_processed)
+ estate->es_processed += fsstate->num_tuples;
+ return ExecClearTuple(slot);
+ }
+
+ /* If we didn't get any tuples, must be end of data. */
+ if (fsstate->next_tuple >= fsstate->num_tuples)
+ return ExecClearTuple(slot);
+
+ /* We'll store RETURNING tuples in the batch_cxt. */
+ oldcontext = MemoryContextSwitchTo(fsstate->batch_cxt);
+
+ /* Fetch the next tuple. */
+ store_returning_result(slot,
+ fsstate->result,
+ fsstate->next_tuple,
+ fsstate->rel,
+ fsstate->attinmeta,
+ fsstate->retrieved_attrs,
+ fsstate->temp_cxt);
+
+ MemoryContextSwitchTo(oldcontext);
+
+ /* Save the result. */
+ fsstate->rslot = slot;
+ fsstate->next_tuple++;
+
+ /*
+ * Return slot. Note that this is safe because that there are no local
+ * conditions and because that the tuple contained in the slot is
+ * projected safely and then ignored (see also plan_direct_update).
+ */
+ return slot;
+ }
+
/*
* If this is the first call after Begin or ReScan, we need to create the
* cursor on the remote side.
***************
*** 1090,1095 **** postgresEndForeignScan(ForeignScanState *node)
--- 1224,1236 ----
if (fsstate == NULL)
return;
+ if (fsstate->direct_update)
+ {
+ /* Clean up */
+ if (fsstate->result)
+ PQclear(fsstate->result);
+ }
+
/* Close the cursor if open, to prevent accumulation of cursors */
if (fsstate->cursor_exists)
close_cursor(fsstate->conn, fsstate->cursor_number);
***************
*** 1206,1211 **** postgresPlanForeignModify(PlannerInfo *root,
--- 1347,1378 ----
}
/*
+ * For an UPDATE/DELETE command, if there are no local conditions or joins
+ * needed (see check_direct_update for more details), let the scan node do
+ * UPDATE/DELETE RETURNING and then do nothing at ModifyTable.
+ */
+ if (operation == CMD_UPDATE || operation == CMD_DELETE)
+ {
+ /* Check whether it's safe to do direct update. */
+ if (check_direct_update(root, plan,
+ resultRelation,
+ subplan_index,
+ rel, targetAttrs))
+ {
+ List *fdw_private;
+
+ /* OK, generate a plan to do direct update. */
+ fdw_private = plan_direct_update(root, plan,
+ resultRelation,
+ subplan_index,
+ rel, targetAttrs);
+
+ heap_close(rel, NoLock);
+ return fdw_private;
+ }
+ }
+
+ /*
* Extract the relevant RETURNING list if any.
*/
if (plan->returningLists)
***************
*** 1284,1289 **** postgresBeginForeignModify(ModifyTableState *mtstate,
--- 1451,1489 ----
fmstate = (PgFdwModifyState *) palloc0(sizeof(PgFdwModifyState));
fmstate->rel = rel;
+ /* Deconstruct fdw_private data. */
+ fmstate->query = strVal(list_nth(fdw_private,
+ FdwModifyPrivateUpdateSql));
+ fmstate->target_attrs = (List *) list_nth(fdw_private,
+ FdwModifyPrivateTargetAttnums);
+ fmstate->has_returning = intVal(list_nth(fdw_private,
+ FdwModifyPrivateHasReturning));
+ fmstate->retrieved_attrs = (List *) list_nth(fdw_private,
+ FdwModifyPrivateRetrievedAttrs);
+
+ if (fmstate->query == NULL)
+ {
+ PlanState *node = mtstate->mt_plans[subplan_index];
+ PgFdwScanState *fsstate;
+
+ Assert(fmstate->target_attrs == NIL);
+ Assert(fmstate->has_returning == false);
+ Assert(fmstate->retrieved_attrs == NIL);
+
+ Assert(nodeTag(node) == T_ForeignScanState);
+ fsstate = ((ForeignScanState *) node)->fdw_state;
+
+ Assert(fsstate->direct_update);
+ fmstate->direct_update = true;
+ if (fsstate->has_returning)
+ {
+ fmstate->has_returning = true;
+ fmstate->fsstate = fsstate;
+ }
+ resultRelInfo->ri_FdwState = fmstate;
+ return;
+ }
+
/*
* Identify which user to do the remote access as. This should match what
* ExecCheckRTEPerms() does.
***************
*** 1300,1315 **** postgresBeginForeignModify(ModifyTableState *mtstate,
fmstate->conn = GetConnection(server, user, true);
fmstate->p_name = NULL; /* prepared statement not made yet */
- /* Deconstruct fdw_private data. */
- fmstate->query = strVal(list_nth(fdw_private,
- FdwModifyPrivateUpdateSql));
- fmstate->target_attrs = (List *) list_nth(fdw_private,
- FdwModifyPrivateTargetAttnums);
- fmstate->has_returning = intVal(list_nth(fdw_private,
- FdwModifyPrivateHasReturning));
- fmstate->retrieved_attrs = (List *) list_nth(fdw_private,
- FdwModifyPrivateRetrievedAttrs);
-
/* Create context for per-tuple temp workspace. */
fmstate->temp_cxt = AllocSetContextCreate(estate->es_query_cxt,
"postgres_fdw temporary data",
--- 1500,1505 ----
***************
*** 1407,1413 **** postgresExecForeignInsert(EState *estate,
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(fmstate, slot, res);
}
else
n_rows = atoi(PQcmdTuples(res));
--- 1597,1607 ----
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(slot, res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
}
else
n_rows = atoi(PQcmdTuples(res));
***************
*** 1438,1443 **** postgresExecForeignUpdate(EState *estate,
--- 1632,1645 ----
PGresult *res;
int n_rows;
+ /* Return slot created in the ForeignScan node when doing direct update. */
+ if (fmstate->direct_update)
+ {
+ Assert(fmstate->has_returning);
+ Assert(fmstate->fsstate->rslot);
+ return fmstate->fsstate->rslot;
+ }
+
/* Set up the prepared statement on the remote server, if we didn't yet */
if (!fmstate->p_name)
prepare_foreign_modify(fmstate);
***************
*** 1477,1483 **** postgresExecForeignUpdate(EState *estate,
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(fmstate, slot, res);
}
else
n_rows = atoi(PQcmdTuples(res));
--- 1679,1689 ----
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(slot, res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
}
else
n_rows = atoi(PQcmdTuples(res));
***************
*** 1508,1513 **** postgresExecForeignDelete(EState *estate,
--- 1714,1727 ----
PGresult *res;
int n_rows;
+ /* Return slot created in the ForeignScan node when doing direct update. */
+ if (fmstate->direct_update)
+ {
+ Assert(fmstate->has_returning);
+ Assert(fmstate->fsstate->rslot);
+ return fmstate->fsstate->rslot;
+ }
+
/* Set up the prepared statement on the remote server, if we didn't yet */
if (!fmstate->p_name)
prepare_foreign_modify(fmstate);
***************
*** 1547,1553 **** postgresExecForeignDelete(EState *estate,
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(fmstate, slot, res);
}
else
n_rows = atoi(PQcmdTuples(res));
--- 1761,1771 ----
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(slot, res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
}
else
n_rows = atoi(PQcmdTuples(res));
***************
*** 1575,1580 **** postgresEndForeignModify(EState *estate,
--- 1793,1802 ----
if (fmstate == NULL)
return;
+ /* If doing direct update, there is nothing to do */
+ if (fmstate->direct_update)
+ return;
+
/* If we created a prepared statement, destroy it */
if (fmstate->p_name)
{
***************
*** 1653,1663 **** postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
{
List *fdw_private;
char *sql;
if (es->verbose)
{
fdw_private = ((ForeignScan *) node->ss.ps.plan)->fdw_private;
! sql = strVal(list_nth(fdw_private, FdwScanPrivateSelectSql));
ExplainPropertyText("Remote SQL", sql, es);
}
}
--- 1875,1900 ----
{
List *fdw_private;
char *sql;
+ bool direct_update;
if (es->verbose)
{
fdw_private = ((ForeignScan *) node->ss.ps.plan)->fdw_private;
! if (list_length(fdw_private) >= MaxFdwScanFdwPrivateLength)
! {
! Assert(list_length(fdw_private) == MaxFdwScanFdwPrivateLength);
! direct_update = true;
! }
! else
! {
! Assert(list_length(fdw_private) == MinFdwScanFdwPrivateLength);
! direct_update = false;
! }
!
! if (direct_update)
! sql = strVal(list_nth(fdw_private, FdwScanPrivateUpdateSql));
! else
! sql = strVal(list_nth(fdw_private, FdwScanPrivateSelectSql));
ExplainPropertyText("Remote SQL", sql, es);
}
}
***************
*** 1678,1684 **** postgresExplainForeignModify(ModifyTableState *mtstate,
char *sql = strVal(list_nth(fdw_private,
FdwModifyPrivateUpdateSql));
! ExplainPropertyText("Remote SQL", sql, es);
}
}
--- 1915,1922 ----
char *sql = strVal(list_nth(fdw_private,
FdwModifyPrivateUpdateSql));
! if (sql != NULL)
! ExplainPropertyText("Remote SQL", sql, es);
}
}
***************
*** 1907,1912 **** ec_member_matches_foreign(PlannerInfo *root, RelOptInfo *rel,
--- 2145,2338 ----
}
/*
+ * Check whether it's safe to update the foreign table directly.
+ */
+ static bool
+ check_direct_update(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[resultRelation];
+ Plan *subplan = (Plan *) list_nth(plan->plans, subplan_index);
+ ListCell *lc;
+
+ if (rel->trigdesc &&
+ (rel->trigdesc->trig_update_after_row ||
+ rel->trigdesc->trig_update_before_row))
+ return false;
+
+ if (nodeTag(subplan) != T_ForeignScan || subplan->qual != NIL)
+ return false;
+
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(subplan->targetlist,
+ attnum);
+
+ if (!is_foreign_expr(root, baserel, (Expr *) tle->expr))
+ return false;
+ }
+
+ return true;
+ }
+
+ /*
+ * Generate a plan to update the foreign table directly.
+ */
+ static List *
+ plan_direct_update(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs)
+ {
+ CmdType operation = plan->operation;
+ bool canSetTag = plan->canSetTag;
+ Plan *subplan = (Plan *) list_nth(plan->plans, subplan_index);
+ ForeignScan *fscan = (ForeignScan *) subplan;
+ StringInfoData sql;
+ List *remote_conds;
+ List *returningList = NIL;
+ List *retrieved_attrs = NIL;
+ List *new_tlist = NIL;
+ List *fdw_private;
+
+ Assert(operation == CMD_UPDATE || operation == CMD_DELETE);
+
+ initStringInfo(&sql);
+
+ /*
+ * Extract the baserestrictinfo clauses that can be evaluated remotely.
+ */
+ remote_conds = (List *) list_nth(fscan->fdw_private,
+ FdwScanPrivateRemoteConds);
+
+ /*
+ * Extract the relevant RETURNING list if any.
+ */
+ if (plan->returningLists)
+ returningList = (List *) list_nth(plan->returningLists, subplan_index);
+
+ /*
+ * Construct the SQL command string.
+ */
+ if (operation == CMD_UPDATE)
+ {
+ List *targetlist = subplan->targetlist;
+
+ deparseDirectUpdateSql(&sql, root, resultRelation, rel,
+ remote_conds,
+ targetlist,
+ targetAttrs,
+ returningList,
+ &retrieved_attrs);
+ }
+ else
+ {
+ Assert(operation == CMD_DELETE);
+
+ deparseDirectDeleteSql(&sql, root, resultRelation, rel,
+ remote_conds,
+ returningList,
+ &retrieved_attrs);
+ }
+
+ /*
+ * Update the fdw_private list that will be available to the executor.
+ * Items in the list must match enum FdwScanPrivateIndex, above.
+ */
+ fscan->fdw_private = lappend(fscan->fdw_private, makeString(sql.data));
+ fscan->fdw_private = lappend(fscan->fdw_private, makeInteger(canSetTag));
+ fscan->fdw_private = lappend(fscan->fdw_private,
+ makeInteger((retrieved_attrs != NIL)));
+ fscan->fdw_private = lappend(fscan->fdw_private, retrieved_attrs);
+
+ /*
+ * Rrewrite the targetlist for an UPDATE command for safety of ExecProject.
+ * Note we ignore and do not reference result tuples in direct update case.
+ */
+ if (operation == CMD_UPDATE)
+ {
+ ListCell *lc;
+ int attrno = 1;
+ int numattrs = RelationGetNumberOfAttributes(rel);
+
+ foreach(lc, subplan->targetlist)
+ {
+ TargetEntry *tle = (TargetEntry *) lfirst(lc);
+
+ if (tle->resjunk)
+ {
+ new_tlist = lappend(new_tlist, tle);
+ continue;
+ }
+
+ if (attrno > numattrs)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("table row type and query-specified row type do not match"),
+ errdetail("Query has too many columns.")));
+
+ if (!list_member_int(targetAttrs, attrno))
+ new_tlist = lappend(new_tlist, tle);
+ else
+ {
+ Form_pg_attribute attr;
+ Oid atttype;
+ int32 atttypmod;
+ Oid attcollation;
+ Node *new_expr;
+ TargetEntry *new_tle;
+
+ attr = rel->rd_att->attrs[attrno - 1];
+
+ Assert(!attr->attisdropped);
+ atttype = attr->atttypid;
+ atttypmod = attr->atttypmod;
+ attcollation = attr->attcollation;
+
+ new_expr = (Node *) makeVar(resultRelation,
+ attrno,
+ atttype,
+ atttypmod,
+ attcollation,
+ 0);
+
+ new_tle = makeTargetEntry((Expr *) new_expr,
+ attrno,
+ pstrdup(NameStr(attr->attname)),
+ false);
+
+ new_tlist = lappend(new_tlist, new_tle);
+ }
+
+ attrno++;
+ }
+
+ if (attrno != numattrs + 1)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("table row type and query-specified row type do not match"),
+ errdetail("Query has too few columns.")));
+
+ subplan->targetlist = new_tlist;
+ }
+
+ /*
+ * Build the fdw_private list that will be available to the executor.
+ * Items in the list must match enum FdwModifyPrivateIndex, above.
+ */
+ fdw_private = list_make4(makeString(NULL), NIL, makeInteger(false), NIL);
+
+ return fdw_private;
+ }
+
+ /*
* Create cursor for node's query with current parameter values.
*/
static void
***************
*** 2254,2272 **** convert_prep_stmt_params(PgFdwModifyState *fmstate,
* have PG_TRY blocks to ensure this happens.
*/
static void
! store_returning_result(PgFdwModifyState *fmstate,
! TupleTableSlot *slot, PGresult *res)
{
/* PGresult must be released before leaving this function. */
PG_TRY();
{
HeapTuple newtup;
! newtup = make_tuple_from_result_row(res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
/* tuple will be deleted when it is cleared from the slot */
ExecStoreTuple(newtup, slot, InvalidBuffer, true);
}
--- 2680,2703 ----
* have PG_TRY blocks to ensure this happens.
*/
static void
! store_returning_result(TupleTableSlot *slot,
! PGresult *res,
! int row,
! Relation rel,
! AttInMetadata *attinmeta,
! List *retrieved_attrs,
! MemoryContext temp_context)
{
/* PGresult must be released before leaving this function. */
PG_TRY();
{
HeapTuple newtup;
! newtup = make_tuple_from_result_row(res, row,
! rel,
! attinmeta,
! retrieved_attrs,
! temp_context);
/* tuple will be deleted when it is cleared from the slot */
ExecStoreTuple(newtup, slot, InvalidBuffer, true);
}
*** a/contrib/postgres_fdw/postgres_fdw.h
--- b/contrib/postgres_fdw/postgres_fdw.h
***************
*** 66,75 **** extern void deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 66,87 ----
Index rtindex, Relation rel,
List *targetAttrs, List *returningList,
List **retrieved_attrs);
+ extern void deparseDirectUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List *targetlist,
+ List *targetAttrs,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseDeleteSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *returningList,
List **retrieved_attrs);
+ extern void deparseDirectDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseAnalyzeSizeSql(StringInfo buf, Relation rel);
extern void deparseAnalyzeSql(StringInfo buf, Relation rel,
List **retrieved_attrs);
On Tue, Jul 8, 2014 at 3:07 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
Attached is a WIP patch for the following:
/*
* postgresPlanForeignModify
* Plan an insert/update/delete operation on a foreign table
*
* Note: currently, the plan tree generated for UPDATE/DELETE will always
* include a ForeignScan that retrieves ctids (using SELECT FOR UPDATE)
* and then the ModifyTable node will have to execute individual remote
* UPDATE/DELETE commands. If there are no local conditions or joins
* needed, it'd be better to let the scan node do UPDATE/DELETE RETURNING
* and then do nothing at ModifyTable. Room for future optimization ...
*/In the patch postgresPlanForeignModify has been modified so that if, in
addition to the above condition, the followings are satisfied, then the
ForeignScan and ModifyTable node will work that way.- There are no local BEFORE/AFTER triggers.
- In UPDATE it's safe to evaluate expressions to assign to the target
columns on the remote server.Here is a simple performance test.
On remote side:
postgres=# create table t (id serial primary key, inserted timestamp
default clock_timestamp(), data text);
CREATE TABLE
postgres=# insert into t(data) select random() from generate_series(0,
99999);
INSERT 0 100000
postgres=# vacuum t;
VACUUMOn local side:
postgres=# create foreign table ft (id integer, inserted timestamp, data
text) server myserver options (table_name 't');
CREATE FOREIGN TABLEUnpatched:
postgres=# explain analyze verbose delete from ft where id < 10000;
QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------
Delete on public.ft (cost=100.00..162.32 rows=910 width=6) (actual
time=1275.255..1275.255 rows=0 loops=1)
Remote SQL: DELETE FROM public.t WHERE ctid = $1
-> Foreign Scan on public.ft (cost=100.00..162.32 rows=910 width=6)
(actual time=1.180..52.095 rows=9999 loops=1)
Output: ctid
Remote SQL: SELECT ctid FROM public.t WHERE ((id < 10000)) FOR
UPDATE
Planning time: 0.112 ms
Execution time: 1275.733 ms
(7 rows)Patched (Note that the DELETE command has been pushed down.):
postgres=# explain analyze verbose delete from ft where id < 10000;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------
Delete on public.ft (cost=100.00..162.32 rows=910 width=6) (actual
time=0.006..0.006 rows=0 loops=1)
-> Foreign Scan on public.ft (cost=100.00..162.32 rows=910 width=6)
(actual time=0.001..0.001 rows=0 loops=1)
Output: ctid
Remote SQL: DELETE FROM public.t WHERE ((id < 10000))
Planning time: 0.101 ms
Execution time: 8.808 ms
(6 rows)I'll add this to the next CF. Comments are welcome.
I haven't looked at the code, but +1 for the general idea. The
concept seems good to me, and that's a very large performance
improvement.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hi Fujita-san,
My coworker Takaaki EITOKU reviewed the patch, and here are some
comments from him.
2014-07-08 16:07 GMT+09:00 Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>:
...
In the patch postgresPlanForeignModify has been modified so that if, in
addition to the above condition, the followings are satisfied, then the
ForeignScan and ModifyTable node will work that way.- There are no local BEFORE/AFTER triggers.
The reason the check ignores INSTEAD OF triggers here is that INSTEAD
OF trigger would prevent executing UPDATE/DELETE statement against a
foreign tables at all, right?
- In UPDATE it's safe to evaluate expressions to assign to the target
columns on the remote server.
IIUC, in addition to target expressions, whole WHERE clause should be
safe to be pushed-down. Though that is obviously required, but
mentioning that in documents and source comments would help users and
developers.
Here is a simple performance test.
On remote side:
postgres=# create table t (id serial primary key, inserted timestamp
default clock_timestamp(), data text);
CREATE TABLE
postgres=# insert into t(data) select random() from generate_series(0,
99999);
INSERT 0 100000
postgres=# vacuum t;
VACUUMOn local side:
postgres=# create foreign table ft (id integer, inserted timestamp, data
text) server myserver options (table_name 't');
CREATE FOREIGN TABLEUnpatched:
postgres=# explain analyze verbose delete from ft where id < 10000;
QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------
Delete on public.ft (cost=100.00..162.32 rows=910 width=6) (actual
time=1275.255..1275.255 rows=0 loops=1)
Remote SQL: DELETE FROM public.t WHERE ctid = $1
-> Foreign Scan on public.ft (cost=100.00..162.32 rows=910 width=6)
(actual time=1.180..52.095 rows=9999 loops=1)
Output: ctid
Remote SQL: SELECT ctid FROM public.t WHERE ((id < 10000)) FOR
UPDATE
Planning time: 0.112 ms
Execution time: 1275.733 ms
(7 rows)Patched (Note that the DELETE command has been pushed down.):
postgres=# explain analyze verbose delete from ft where id < 10000;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------
Delete on public.ft (cost=100.00..162.32 rows=910 width=6) (actual
time=0.006..0.006 rows=0 loops=1)
-> Foreign Scan on public.ft (cost=100.00..162.32 rows=910 width=6)
(actual time=0.001..0.001 rows=0 loops=1)
Output: ctid
Remote SQL: DELETE FROM public.t WHERE ((id < 10000))
Planning time: 0.101 ms
Execution time: 8.808 ms
(6 rows)
We found that this patch speeds up DELETE case remarkably, as you
describe above, but we saw only less than 2x speed on UPDATE cases.
Do you have any numbers of UPDATE cases?
Some more random thoughts:
* Naming of new behavior
You named this optimization "Direct Update", but I'm not sure that
this is intuitive enough to express this behavior. I would like to
hear opinions of native speakers.
* Macros for # of fdw_private elements
In postgres_fdw.c you defined MaxFdwScanFdwPrivateLength and
MinFdwScanFdwPrivateLength for determining the mode, but number of
fdw_private elements is not a ranged value but an enum value (I mean
that fdw_private holds 3 or 7, not 4~6, so Max and Min seems
inappropriate for prefix. IMO those macros should be named so that
the names represent the behavior, or define a macro to determine the
mode, say IS_SHIPPABLE(foo).
* Comparison of Macros
Comparison against MaxFdwScanFdwPrivateLength and
MinFdwScanFdwPrivateLength should be == instead of >= or <= to detect
unexpected value. Adding assert macro seems good too.
--
Shigeru HANADA
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
(2014/07/24 18:30), Shigeru Hanada wrote:
My coworker Takaaki EITOKU reviewed the patch, and here are some
comments from him.
Thank you for the review, Eitoku-san!
2014-07-08 16:07 GMT+09:00 Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>:
In the patch postgresPlanForeignModify has been modified so that if, in
addition to the above condition, the followings are satisfied, then the
ForeignScan and ModifyTable node will work that way.- There are no local BEFORE/AFTER triggers.
The reason the check ignores INSTEAD OF triggers here is that INSTEAD
OF trigger would prevent executing UPDATE/DELETE statement against a
foreign tables at all, right?
I'm not sure that I understand your question correctly, but the reason
for that is because foreign tables cannot have INSTEAD OF triggers.
- In UPDATE it's safe to evaluate expressions to assign to the target
columns on the remote server.IIUC, in addition to target expressions, whole WHERE clause should be
safe to be pushed-down. Though that is obviously required, but
mentioning that in documents and source comments would help users and
developers.
OK, I'll add the comments and documentation notes.
(I intentionaly didn't mention that because I think the comment for
postgresPlanForeignModify has already said the equivalent condition, ie,
"there are no local conditions", which means all conditions are executed
remotely.)
We found that this patch speeds up DELETE case remarkably, as you
describe above, but we saw only less than 2x speed on UPDATE cases.
Do you have any numbers of UPDATE cases?
Here is the result for an UPDATE case.
On remote side:
postgres=# create table t (id serial primary key, inserted timestamp
default clock_timestamp(), data text);
CREATE TABLE
postgres=# insert into t(data) select random() from generate_series(0,
99999);
INSERT 0 100000
postgres=# vacuum t;
VACUUMOn local side:
postgres=# create foreign table ft (id integer, inserted timestamp, data
text) server myserver options (table_name 't');
CREATE FOREIGN TABLE
Unpatched:
postgres=# explain analyze verbose update ft set data = 'notsolongtext'
where id < 10000;
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------
Update on public.ft (cost=100.00..147.38 rows=650 width=18) (actual
time=1463.944..1463.944 rows=0 loops=1)
Remote SQL: UPDATE public.t SET data = $2 WHERE ctid = $1
-> Foreign Scan on public.ft (cost=100.00..147.38 rows=650
width=18) (actual time=2.069..83.220 rows=9999 loops=1)
Output: id, inserted, 'notsolongtext'::text, ctid
Remote SQL: SELECT id, inserted, ctid FROM public.t WHERE ((id
< 10000)) FOR UPDATE
Planning time: 0.355 ms
Execution time: 1470.032 ms
(7 rows)
Patched:
postgres=# explain analyze verbose update ft set data = 'notsolongtext'
where id < 10000;
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------
Update on public.ft (cost=100.00..147.38 rows=650 width=18) (actual
time=0.005..0.005 rows=0 loops=1)
-> Foreign Scan on public.ft (cost=100.00..147.38 rows=650
width=18) (actual time=0.001..0.001 rows=0 loops=1)
Output: id, inserted, data, ctid
Remote SQL: UPDATE public.t SET data = 'notsolongtext'::text
WHERE ((id < 10000))
Planning time: 0.197 ms
Execution time: 61.519 ms
(6 rows)
I think that the precise effect of this optimization for DELETE/UPDATE
would depend on eg, data, queries (inc. w/ or w/o RETRUNING clauses) and
server/network performance. Could you tell me these information about
the UPDATE evaluation?
Some more random thoughts:
* Naming of new behavior
You named this optimization "Direct Update", but I'm not sure that
this is intuitive enough to express this behavior. I would like to
hear opinions of native speakers.
+1
* Macros for # of fdw_private elements
In postgres_fdw.c you defined MaxFdwScanFdwPrivateLength and
MinFdwScanFdwPrivateLength for determining the mode, but number of
fdw_private elements is not a ranged value but an enum value (I mean
that fdw_private holds 3 or 7, not 4~6, so Max and Min seems
inappropriate for prefix. IMO those macros should be named so that
the names represent the behavior, or define a macro to determine the
mode, say IS_SHIPPABLE(foo).
OK, Will fix.
* Comparison of Macros
Comparison against MaxFdwScanFdwPrivateLength and
MinFdwScanFdwPrivateLength should be == instead of >= or <= to detect
unexpected value. Adding assert macro seems good too.
Will fix this too.
Thanks,
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Shigeru Hanada wrote:
* Naming of new behavior
You named this optimization "Direct Update", but I'm not sure that
this is intuitive enough to express this behavior. I would like to
hear opinions of native speakers.
How about "batch foreign update" or "batch foreign modification"?
(Disclaimer: I'm not a native speaker either.)
Yours,
Laurenz Albe
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Jul 25, 2014 at 3:39 AM, Albe Laurenz <laurenz.albe@wien.gv.at> wrote:
Shigeru Hanada wrote:
* Naming of new behavior
You named this optimization "Direct Update", but I'm not sure that
this is intuitive enough to express this behavior. I would like to
hear opinions of native speakers.How about "batch foreign update" or "batch foreign modification"?
(Disclaimer: I'm not a native speaker either.)
I think direct update sounds pretty good. "Batch" does not sound as
good to me, since it doesn't clearly describe what makes this patch
special as opposed to some other grouping of updates that happens to
produce a speedup.
Another term that might be used is "update pushdown", since we are
pushing the whole update to the remote server instead of having the
local server participate. Without looking at the patch, I don't have
a strong opinion on whether that's better than "direct update" in this
context.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
(2014/07/29 0:58), Robert Haas wrote:
On Fri, Jul 25, 2014 at 3:39 AM, Albe Laurenz <laurenz.albe@wien.gv.at> wrote:
Shigeru Hanada wrote:
* Naming of new behavior
You named this optimization "Direct Update", but I'm not sure that
this is intuitive enough to express this behavior. I would like to
hear opinions of native speakers.How about "batch foreign update" or "batch foreign modification"?
(Disclaimer: I'm not a native speaker either.)I think direct update sounds pretty good. "Batch" does not sound as
good to me, since it doesn't clearly describe what makes this patch
special as opposed to some other grouping of updates that happens to
produce a speedup.
I agree with Robert on that point.
Another term that might be used is "update pushdown", since we are
pushing the whole update to the remote server instead of having the
local server participate. Without looking at the patch, I don't have
a strong opinion on whether that's better than "direct update" in this
context.
"Update Pushdown" is fine with me.
If there are no objections of others, I'll change the name from "Direct
Update" to "Update Pushdown".
Thanks,
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hi
2014-07-30 10:22 GMT+02:00 Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>:
(2014/07/29 0:58), Robert Haas wrote:
On Fri, Jul 25, 2014 at 3:39 AM, Albe Laurenz <laurenz.albe@wien.gv.at>
wrote:Shigeru Hanada wrote:
* Naming of new behavior
You named this optimization "Direct Update", but I'm not sure that
this is intuitive enough to express this behavior. I would like to
hear opinions of native speakers.How about "batch foreign update" or "batch foreign modification"?
(Disclaimer: I'm not a native speaker either.)I think direct update sounds pretty good. "Batch" does not sound as
good to me, since it doesn't clearly describe what makes this patch
special as opposed to some other grouping of updates that happens to
produce a speedup.I agree with Robert on that point.
Another term that might be used is "update pushdown", since we are
pushing the whole update to the remote server instead of having the
local server participate. Without looking at the patch, I don't have
a strong opinion on whether that's better than "direct update" in this
context."Update Pushdown" is fine with me.
If there are no objections of others, I'll change the name from "Direct
Update" to "Update Pushdown".
I like "Update Pushdown" - it is simple without other semantic
Regards
Pavel
Show quoted text
Thanks,
Best regards,
Etsuro Fujita--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hi Fujita-san,
Here is a new review result from Eitoku-san.
2014-07-25 16:30 GMT+09:00 Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>:
(2014/07/24 18:30), Shigeru Hanada wrote:
I'm not sure that I understand your question correctly, but the reason for
that is because foreign tables cannot have INSTEAD OF triggers.
Now I see the reason, but then I worry (though it unlikely happens) a
case that new trigger type might be added in future. The code says
that "only BEFORE and AFTER triggers are unsafe for direct update",
but it would be more safe to code that "any trigger other than event
trigger is unsafe for direct update".
We found that this patch speeds up DELETE case remarkably, as you
describe above, but we saw only less than 2x speed on UPDATE cases.
Do you have any numbers of UPDATE cases?Here is the result for an UPDATE case.
Hmm, performance gain on UPDATE cases seems similar to our results,
except planning times. In your environment the patch reduces planning
time too, but we got longer planning times with your patch (in only
once in six trial, we got shorter planning time than average of
patched version). Could you try multiple times on your environment?
I think that the precise effect of this optimization for DELETE/UPDATE would
depend on eg, data, queries (inc. w/ or w/o RETRUNING clauses) and
server/network performance. Could you tell me these information about the
UPDATE evaluation?
I tried on a CentOS 6.5 on VMware on a Note PC with Core i3 1.17GHz,
2.0GB memory and single HDD, so the performance is poor.
The SQLs used for performance test are quite simple, update 10
thousands rows at a time, and repeat it for different section of the
table for six times. The definition of foreign table ft is same as
the one in your case.
EXPLAIN ANALYZE VERBOSE UPDATe ft SET data = 'abcdefg' WHERE id >= 0
AND id < 10000;
EXPLAIN ANALYZE VERBOSE UPDATe ft SET data = 'abcdefg' WHERE id >=
10000 AND id < 20000;
EXPLAIN ANALYZE VERBOSE UPDATe ft SET data = 'abcdefg' WHERE id >=
20000 AND id < 30000;
EXPLAIN ANALYZE VERBOSE UPDATe ft SET data = 'abcdefg' WHERE id >=
30000 AND id < 40000;
EXPLAIN ANALYZE VERBOSE UPDATe ft SET data = 'abcdefg' WHERE id >=
40000 AND id < 50000;
EXPLAIN ANALYZE VERBOSE UPDATe ft SET data = 'abcdefg' WHERE id >=
50000 AND id < 60000;
Some more random thoughts:
* Naming of new behavior
You named this optimization "Direct Update", but I'm not sure that
this is intuitive enough to express this behavior. I would like to
hear opinions of native speakers.
Update push-down seems nice with according to others.
--
Shigeru HANADA
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
(2014/07/30 17:22), Etsuro Fujita wrote:
(2014/07/29 0:58), Robert Haas wrote:
On Fri, Jul 25, 2014 at 3:39 AM, Albe Laurenz
<laurenz.albe@wien.gv.at> wrote:Shigeru Hanada wrote:
* Naming of new behavior
You named this optimization "Direct Update", but I'm not sure that
this is intuitive enough to express this behavior. I would like to
hear opinions of native speakers.How about "batch foreign update" or "batch foreign modification"?
(Disclaimer: I'm not a native speaker either.)I think direct update sounds pretty good. "Batch" does not sound as
good to me, since it doesn't clearly describe what makes this patch
special as opposed to some other grouping of updates that happens to
produce a speedup.I agree with Robert on that point.
Another term that might be used is "update pushdown", since we are
pushing the whole update to the remote server instead of having the
local server participate. Without looking at the patch, I don't have
a strong opinion on whether that's better than "direct update" in this
context."Update Pushdown" is fine with me.
If there are no objections of others, I'll change the name from "Direct
Update" to "Update Pushdown".
Done. (I've left deparseDirectUpdateSql/deparseDirectDeleteSql as-is,
though.)
Other changes:
* Address the comments from Eitoku-san.
* Add regression tests.
* Fix a bug, which fails to show the actual row counts in EXPLAIN
ANALYZE for UPDATE/DELETE without a RETURNING clause.
* Rebase to HEAD.
Please find attached an updated version of the patch.
Thanks,
Best regards,
Etsuro Fujita
Attachments:
postgres_fdw-update-v2.patchtext/x-diff; name=postgres_fdw-update-v2.patchDownload
*** a/contrib/postgres_fdw/deparse.c
--- b/contrib/postgres_fdw/deparse.c
***************
*** 188,197 **** is_foreign_expr(PlannerInfo *root,
if (!foreign_expr_walker((Node *) expr, &glob_cxt, &loc_cxt))
return false;
- /* Expressions examined here should be boolean, ie noncollatable */
- Assert(loc_cxt.collation == InvalidOid);
- Assert(loc_cxt.state == FDW_COLLATE_NONE);
-
/*
* An expression which includes any mutable functions can't be sent over
* because its result is not stable. For example, sending now() remote
--- 188,193 ----
***************
*** 927,932 **** deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 923,981 ----
}
/*
+ * deparse remote UPDATE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparseDirectUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List *targetlist,
+ List *targetAttrs, List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ List *params_list = NIL;
+ deparse_expr_cxt context;
+ bool first;
+ ListCell *lc;
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = NULL;
+
+ appendStringInfoString(buf, "UPDATE ");
+ deparseRelation(buf, rel);
+ appendStringInfoString(buf, " SET ");
+
+ first = true;
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(targetlist, attnum);
+
+ if (!first)
+ appendStringInfoString(buf, ", ");
+ first = false;
+
+ deparseColumnRef(buf, rtindex, attnum, root);
+ appendStringInfo(buf, " = ");
+ deparseExpr((Expr *) tle->expr, &context);
+ }
+ if (remote_conds)
+ appendWhereClause(buf, root, baserel, remote_conds,
+ true, ¶ms_list);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* deparse remote DELETE statement
*
* The statement text is appended to buf, and we also create an integer List
***************
*** 949,954 **** deparseDeleteSql(StringInfo buf, PlannerInfo *root,
--- 998,1030 ----
}
/*
+ * deparse remote DELETE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparseDirectDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ List *params_list = NIL;
+
+ appendStringInfoString(buf, "DELETE FROM ");
+ deparseRelation(buf, rel);
+ if (remote_conds)
+ appendWhereClause(buf, root, baserel, remote_conds,
+ true, ¶ms_list);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* Add a RETURNING clause, if needed, to an INSERT/UPDATE/DELETE.
*/
static void
*** a/contrib/postgres_fdw/expected/postgres_fdw.out
--- b/contrib/postgres_fdw/expected/postgres_fdw.out
***************
*** 992,998 **** INSERT INTO ft2 (c1,c2,c3)
--- 992,1019 ----
(3 rows)
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ QUERY PLAN
+ ----------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ -> Foreign Scan on public.ft2
+ Output: c1, c2, NULL::integer, c3, c4, c5, c6, c7, c8, ctid
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 300), c3 = (c3 || '_update3'::text) WHERE ((("C 1" % 10) = 3))
+ (4 rows)
+
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Output: c1, c2, c3, c4, c5, c6, c7, c8
+ -> Foreign Scan on public.ft2
+ Output: c1, c2, NULL::integer, c3, c4, c5, c6, c7, c8, ctid
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 400), c3 = (c3 || '_update7'::text) WHERE ((("C 1" % 10) = 7)) RETURNING "C 1", c2, c3, c4, c5, c6, c7, c8
+ (5 rows)
+
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
c1 | c2 | c3 | c4 | c5 | c6 | c7 | c8
------+-----+--------------------+------------------------------+--------------------------+----+------------+-----
***************
*** 1102,1108 **** UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
--- 1123,1129 ----
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
***************
*** 1123,1138 **** UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
! QUERY PLAN
! ----------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
- Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1 RETURNING "C 1", c4
-> Foreign Scan on public.ft2
Output: ctid
! Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) FOR UPDATE
! (6 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
--- 1144,1171 ----
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c7 = DEFAULT WHERE c1 % 10 = 5 AND date(c4) = '1970-01-17'::date; -- can't be pushed down
+ QUERY PLAN
+ -----------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c7 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.ft2
+ Output: c1, c2, NULL::integer, c3, c4, c5, c6, 'ft2 '::character(10), c8, ctid
+ Filter: (date(ft2.c4) = '01-17-1970'::date)
+ Remote SQL: SELECT "C 1", c2, c3, c4, c5, c6, c8, ctid FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) FOR UPDATE
+ (6 rows)
+
+ UPDATE ft2 SET c7 = DEFAULT WHERE c1 % 10 = 5 AND date(c4) = '1970-01-17'::date;
+ EXPLAIN (verbose, costs off)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
! QUERY PLAN
! --------------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
-> Foreign Scan on public.ft2
Output: ctid
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) RETURNING "C 1", c4
! (5 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
***************
*** 1243,1249 **** DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
--- 1276,1282 ----
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
***************
*** 2227,2233 **** CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 2260,2266 ----
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 2386,2392 **** savepoint s3;
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 2419,2425 ----
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (-2) WHERE ((c2 = 42)) AND (("C 1" = 10))
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
***************
*** 86,92 **** typedef struct PgFdwRelationInfo
* planner to executor. Currently we store:
*
* 1) SELECT statement text to be sent to the remote server
! * 2) Integer list of attribute numbers retrieved by the SELECT
*
* These items are indexed with the enum FdwScanPrivateIndex, so an item
* can be fetched with list_nth(). For example, to get the SELECT statement:
--- 86,97 ----
* planner to executor. Currently we store:
*
* 1) SELECT statement text to be sent to the remote server
! * 2) List of restriction clauses that can be executed remotely
! * 3) Integer list of attribute numbers retrieved by the SELECT
! * 4) UPDATE/DELETE statement text to be sent to the remote server
! * 5) Boolean flag showing if we set the command es_processed
! * 6) Boolean flag showing if the remote query has a RETURNING clause
! * 7) Integer list of attribute numbers retrieved by RETURNING, if any
*
* These items are indexed with the enum FdwScanPrivateIndex, so an item
* can be fetched with list_nth(). For example, to get the SELECT statement:
***************
*** 96,105 **** enum FdwScanPrivateIndex
{
/* SQL statement to execute remotely (as a String node) */
FdwScanPrivateSelectSql,
! /* Integer list of attribute numbers retrieved by the SELECT */
! FdwScanPrivateRetrievedAttrs
};
/*
* Similarly, this enum describes what's kept in the fdw_private list for
* a ModifyTable node referencing a postgres_fdw foreign table. We store:
--- 101,123 ----
{
/* SQL statement to execute remotely (as a String node) */
FdwScanPrivateSelectSql,
! /* List of restriction clauses that can be executed remotely */
! FdwScanPrivateRemoteConds,
! /* Integer list of attribute numbers retrieved by SELECT */
! FdwScanPrivateRetrievedAttrsBySelect,
! /* UPDATE/DELETE statement to execute remotely (as a String node) */
! FdwScanPrivateUpdateSql,
! /* set-processed flag (as an integer Value node) */
! FdwScanPrivateSetProcessed,
! /* has-returning flag (as an integer Value node) */
! FdwScanPrivateHasReturning,
! /* Integer list of attribute numbers retrieved by RETURNING */
! FdwScanPrivateRetrievedAttrsByReturning
};
+ #define SelectFdwScanPrivateLength 3
+ #define UpdateFdwScanPrivateLength 7
+
/*
* Similarly, this enum describes what's kept in the fdw_private list for
* a ModifyTable node referencing a postgres_fdw foreign table. We store:
***************
*** 131,138 **** typedef struct PgFdwScanState
AttInMetadata *attinmeta; /* attribute datatype conversion metadata */
/* extracted fdw_private data */
! char *query; /* text of SELECT command */
List *retrieved_attrs; /* list of retrieved attribute numbers */
/* for remote query execution */
PGconn *conn; /* connection for the scan */
--- 149,158 ----
AttInMetadata *attinmeta; /* attribute datatype conversion metadata */
/* extracted fdw_private data */
! char *query; /* text of SELECT or UPDATE/DELETE command */
List *retrieved_attrs; /* list of retrieved attribute numbers */
+ bool set_processed; /* do we set the command es_processed? */
+ bool has_returning; /* is there a RETURNING clause? */
/* for remote query execution */
PGconn *conn; /* connection for the scan */
***************
*** 152,157 **** typedef struct PgFdwScanState
--- 172,182 ----
int fetch_ct_2; /* Min(# of fetches done, 2) */
bool eof_reached; /* true if last fetch reached EOF */
+ /* for update pushdown */
+ bool update_is_pushed_down; /* is UPDATE/DELETE pushed down? */
+ PGresult *result; /* result of an UPDATE/DELETE query */
+ TupleTableSlot *rslot; /* slot containing the result tuple */
+
/* working memory contexts */
MemoryContext batch_cxt; /* context holding current batch of tuples */
MemoryContext temp_cxt; /* context for per-tuple temporary data */
***************
*** 180,185 **** typedef struct PgFdwModifyState
--- 205,214 ----
int p_nums; /* number of parameters to transmit */
FmgrInfo *p_flinfo; /* output conversion functions for them */
+ /* for update pushdown */
+ bool update_is_pushed_down; /* is UPDATE/DELETE pushed down? */
+ PgFdwScanState *fsstate; /* execution state of a foreign scan */
+
/* working memory context */
MemoryContext temp_cxt; /* context for per-tuple temporary data */
} PgFdwModifyState;
***************
*** 308,319 **** static bool ec_member_matches_foreign(PlannerInfo *root, RelOptInfo *rel,
static void create_cursor(ForeignScanState *node);
static void fetch_more_data(ForeignScanState *node);
static void close_cursor(PGconn *conn, unsigned int cursor_number);
static void prepare_foreign_modify(PgFdwModifyState *fmstate);
static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
ItemPointer tupleid,
TupleTableSlot *slot);
! static void store_returning_result(PgFdwModifyState *fmstate,
! TupleTableSlot *slot, PGresult *res);
static int postgresAcquireSampleRowsFunc(Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
--- 337,365 ----
static void create_cursor(ForeignScanState *node);
static void fetch_more_data(ForeignScanState *node);
static void close_cursor(PGconn *conn, unsigned int cursor_number);
+ static bool update_is_pushdown_safe(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs);
+ static List *push_update_down(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs);
static void prepare_foreign_modify(PgFdwModifyState *fmstate);
static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
ItemPointer tupleid,
TupleTableSlot *slot);
! static void store_returning_result(TupleTableSlot *slot,
! PGresult *res,
! int row,
! Relation rel,
! AttInMetadata *attinmeta,
! List *retrieved_attrs,
! MemoryContext temp_context);
static int postgresAcquireSampleRowsFunc(Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
***************
*** 852,858 **** postgresGetForeignPlan(PlannerInfo *root,
* Build the fdw_private list that will be available to the executor.
* Items in the list must match enum FdwScanPrivateIndex, above.
*/
! fdw_private = list_make2(makeString(sql.data),
retrieved_attrs);
/*
--- 898,905 ----
* Build the fdw_private list that will be available to the executor.
* Items in the list must match enum FdwScanPrivateIndex, above.
*/
! fdw_private = list_make3(makeString(sql.data),
! remote_conds,
retrieved_attrs);
/*
***************
*** 914,935 **** postgresBeginForeignScan(ForeignScanState *node, int eflags)
server = GetForeignServer(table->serverid);
user = GetUserMapping(userid, server->serverid);
/*
* Get connection to the foreign server. Connection manager will
* establish new connection if necessary.
*/
fsstate->conn = GetConnection(server, user, false);
- /* Assign a unique ID for my cursor */
- fsstate->cursor_number = GetCursorNumber(fsstate->conn);
- fsstate->cursor_exists = false;
-
- /* Get private info created by planner functions. */
- fsstate->query = strVal(list_nth(fsplan->fdw_private,
- FdwScanPrivateSelectSql));
- fsstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
- FdwScanPrivateRetrievedAttrs);
-
/* Create contexts for batches of tuples and per-tuple temp workspace. */
fsstate->batch_cxt = AllocSetContextCreate(estate->es_query_cxt,
"postgres_fdw tuple data",
--- 961,993 ----
server = GetForeignServer(table->serverid);
user = GetUserMapping(userid, server->serverid);
+ /* Get private info created by planner functions. */
+ if (list_length(fsplan->fdw_private) == UpdateFdwScanPrivateLength)
+ {
+ fsstate->query = strVal(list_nth(fsplan->fdw_private,
+ FdwScanPrivateUpdateSql));
+ fsstate->set_processed = intVal(list_nth(fsplan->fdw_private,
+ FdwScanPrivateSetProcessed));
+ fsstate->has_returning = intVal(list_nth(fsplan->fdw_private,
+ FdwScanPrivateHasReturning));
+ fsstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
+ FdwScanPrivateRetrievedAttrsByReturning);
+ }
+ else
+ {
+ Assert(list_length(fsplan->fdw_private) == SelectFdwScanPrivateLength);
+ fsstate->query = strVal(list_nth(fsplan->fdw_private,
+ FdwScanPrivateSelectSql));
+ fsstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
+ FdwScanPrivateRetrievedAttrsBySelect);
+ }
+
/*
* Get connection to the foreign server. Connection manager will
* establish new connection if necessary.
*/
fsstate->conn = GetConnection(server, user, false);
/* Create contexts for batches of tuples and per-tuple temp workspace. */
fsstate->batch_cxt = AllocSetContextCreate(estate->es_query_cxt,
"postgres_fdw tuple data",
***************
*** 945,950 **** postgresBeginForeignScan(ForeignScanState *node, int eflags)
--- 1003,1037 ----
/* Get info we'll need for input data conversion. */
fsstate->attinmeta = TupleDescGetAttInMetadata(RelationGetDescr(fsstate->rel));
+ /*
+ * If pushing update down, execute the statement, and check for success.
+ */
+ if (list_length(fsplan->fdw_private) == UpdateFdwScanPrivateLength)
+ {
+ /*
+ * We don't use a PG_TRY block here, so be careful not to throw error
+ * without releasing the PGresult.
+ */
+ fsstate->result = PQexec(fsstate->conn, fsstate->query);
+ if (PQresultStatus(fsstate->result) !=
+ (fsstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
+ pgfdw_report_error(ERROR, fsstate->result, fsstate->conn, true,
+ fsstate->query);
+
+ /* Check number of rows affected. */
+ if (fsstate->has_returning)
+ fsstate->num_tuples = PQntuples(fsstate->result);
+ else
+ fsstate->num_tuples = atoi(PQcmdTuples(fsstate->result));
+
+ fsstate->update_is_pushed_down = true;
+ return;
+ }
+
+ /* Assign a unique ID for my cursor */
+ fsstate->cursor_number = GetCursorNumber(fsstate->conn);
+ fsstate->cursor_exists = false;
+
/* Prepare for output conversion of parameters used in remote query. */
numParams = list_length(fsplan->fdw_exprs);
fsstate->numParams = numParams;
***************
*** 995,1000 **** postgresIterateForeignScan(ForeignScanState *node)
--- 1082,1155 ----
TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
/*
+ * If pushing update down, get the result of UPDATE/DELETE RETURNING.
+ */
+ if (fsstate->update_is_pushed_down)
+ {
+ MemoryContext oldcontext;
+
+ /*
+ * If the update query doesn't have a RETURNING clause, then there is
+ * nothing to do, so we just return an empty slot.
+ */
+ if (!fsstate->has_returning)
+ {
+ /*
+ * Increment the command es_processed if necessary. (ModifyTable
+ * cannot do that by itself in this case.)
+ */
+ if (fsstate->set_processed)
+ {
+ EState *estate = node->ss.ps.state;
+
+ estate->es_processed += fsstate->num_tuples;
+ }
+
+ /*
+ * Increment the tuple count for EXPLAIN ANALYZE if we are running
+ * it. (The command cannot do that by itself in this case.)
+ */
+ if (node->ss.ps.instrument)
+ {
+ Instrumentation *instr = node->ss.ps.instrument;
+
+ instr->ntuples += fsstate->num_tuples;
+ }
+
+ return ExecClearTuple(slot);
+ }
+
+ /* If we didn't get any tuples, must be end of data. */
+ if (fsstate->next_tuple >= fsstate->num_tuples)
+ return ExecClearTuple(slot);
+
+ /* OK, we'll store RETURNING tuples in the batch_cxt. */
+ oldcontext = MemoryContextSwitchTo(fsstate->batch_cxt);
+
+ /* Fetch the next tuple. */
+ store_returning_result(slot,
+ fsstate->result,
+ fsstate->next_tuple,
+ fsstate->rel,
+ fsstate->attinmeta,
+ fsstate->retrieved_attrs,
+ fsstate->temp_cxt);
+ fsstate->rslot = slot;
+ fsstate->next_tuple++;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ /*
+ * Return slot. Note that this is safe because we can avoid applying
+ * ExecQual to the tuple due to no local quals (see the comment for
+ * update_is_pushdown_safe) and because the tuple can be safely
+ * projected by ExecProject (see push_update_down) and would then be
+ * ignored in postgresExecForeignUpdate or postgresExecForeignDelete.
+ */
+ return slot;
+ }
+
+ /*
* If this is the first call after Begin or ReScan, we need to create the
* cursor on the remote side.
*/
***************
*** 1094,1099 **** postgresEndForeignScan(ForeignScanState *node)
--- 1249,1261 ----
if (fsstate == NULL)
return;
+ /* if pushing update down, nothing to do other than cleanup */
+ if (fsstate->update_is_pushed_down)
+ {
+ if (fsstate->result)
+ PQclear(fsstate->result);
+ }
+
/* Close the cursor if open, to prevent accumulation of cursors */
if (fsstate->cursor_exists)
close_cursor(fsstate->conn, fsstate->cursor_number);
***************
*** 1167,1174 **** postgresPlanForeignModify(PlannerInfo *root,
List *returningList = NIL;
List *retrieved_attrs = NIL;
- initStringInfo(&sql);
-
/*
* Core code already has some lock on each rel being planned, so we can
* use NoLock here.
--- 1329,1334 ----
***************
*** 1210,1215 **** postgresPlanForeignModify(PlannerInfo *root,
--- 1370,1399 ----
}
/*
+ * For UPDATE/DELETE, if there are no local conditions or joins needed (see
+ * update_is_pushdown_safe for more details), we push the command down.
+ */
+ if (operation == CMD_UPDATE || operation == CMD_DELETE)
+ {
+ /* Check whether it's safe to push the command down. */
+ if (update_is_pushdown_safe(root, plan,
+ resultRelation,
+ subplan_index,
+ rel, targetAttrs))
+ {
+ List *fdw_private;
+
+ /* OK, modify plan so as to push the command down. */
+ fdw_private = push_update_down(root, plan,
+ resultRelation,
+ subplan_index,
+ rel, targetAttrs);
+ heap_close(rel, NoLock);
+ return fdw_private;
+ }
+ }
+
+ /*
* Extract the relevant RETURNING list if any.
*/
if (plan->returningLists)
***************
*** 1218,1223 **** postgresPlanForeignModify(PlannerInfo *root,
--- 1402,1408 ----
/*
* Construct the SQL command string.
*/
+ initStringInfo(&sql);
switch (operation)
{
case CMD_INSERT:
***************
*** 1288,1293 **** postgresBeginForeignModify(ModifyTableState *mtstate,
--- 1473,1514 ----
fmstate = (PgFdwModifyState *) palloc0(sizeof(PgFdwModifyState));
fmstate->rel = rel;
+ /* Deconstruct fdw_private data. */
+ fmstate->query = strVal(list_nth(fdw_private,
+ FdwModifyPrivateUpdateSql));
+ fmstate->target_attrs = (List *) list_nth(fdw_private,
+ FdwModifyPrivateTargetAttnums);
+ fmstate->has_returning = intVal(list_nth(fdw_private,
+ FdwModifyPrivateHasReturning));
+ fmstate->retrieved_attrs = (List *) list_nth(fdw_private,
+ FdwModifyPrivateRetrievedAttrs);
+
+ /*
+ * if query is NULL, we are in update pushdown case.
+ */
+ if (fmstate->query == NULL)
+ {
+ PlanState *node = mtstate->mt_plans[subplan_index];
+ PgFdwScanState *fsstate;
+
+ Assert(fmstate->target_attrs == NIL);
+ Assert(fmstate->has_returning == false);
+ Assert(fmstate->retrieved_attrs == NIL);
+
+ Assert(nodeTag(node) == T_ForeignScanState);
+ fsstate = ((ForeignScanState *) node)->fdw_state;
+ Assert(fsstate->update_is_pushed_down);
+
+ fmstate->update_is_pushed_down = true;
+ if (fsstate->has_returning)
+ {
+ fmstate->has_returning = true;
+ fmstate->fsstate = fsstate;
+ }
+ resultRelInfo->ri_FdwState = fmstate;
+ return;
+ }
+
/*
* Identify which user to do the remote access as. This should match what
* ExecCheckRTEPerms() does.
***************
*** 1304,1319 **** postgresBeginForeignModify(ModifyTableState *mtstate,
fmstate->conn = GetConnection(server, user, true);
fmstate->p_name = NULL; /* prepared statement not made yet */
- /* Deconstruct fdw_private data. */
- fmstate->query = strVal(list_nth(fdw_private,
- FdwModifyPrivateUpdateSql));
- fmstate->target_attrs = (List *) list_nth(fdw_private,
- FdwModifyPrivateTargetAttnums);
- fmstate->has_returning = intVal(list_nth(fdw_private,
- FdwModifyPrivateHasReturning));
- fmstate->retrieved_attrs = (List *) list_nth(fdw_private,
- FdwModifyPrivateRetrievedAttrs);
-
/* Create context for per-tuple temp workspace. */
fmstate->temp_cxt = AllocSetContextCreate(estate->es_query_cxt,
"postgres_fdw temporary data",
--- 1525,1530 ----
***************
*** 1411,1417 **** postgresExecForeignInsert(EState *estate,
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(fmstate, slot, res);
}
else
n_rows = atoi(PQcmdTuples(res));
--- 1622,1632 ----
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(slot, res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
}
else
n_rows = atoi(PQcmdTuples(res));
***************
*** 1442,1447 **** postgresExecForeignUpdate(EState *estate,
--- 1657,1670 ----
PGresult *res;
int n_rows;
+ /* Just return slot created by the ForeignScan, if pushing update down */
+ if (fmstate->update_is_pushed_down)
+ {
+ Assert(fmstate->has_returning);
+ Assert(fmstate->fsstate->rslot);
+ return fmstate->fsstate->rslot;
+ }
+
/* Set up the prepared statement on the remote server, if we didn't yet */
if (!fmstate->p_name)
prepare_foreign_modify(fmstate);
***************
*** 1481,1487 **** postgresExecForeignUpdate(EState *estate,
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(fmstate, slot, res);
}
else
n_rows = atoi(PQcmdTuples(res));
--- 1704,1714 ----
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(slot, res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
}
else
n_rows = atoi(PQcmdTuples(res));
***************
*** 1512,1517 **** postgresExecForeignDelete(EState *estate,
--- 1739,1752 ----
PGresult *res;
int n_rows;
+ /* Just return slot created by the ForeignScan, if pushing update down */
+ if (fmstate->update_is_pushed_down)
+ {
+ Assert(fmstate->has_returning);
+ Assert(fmstate->fsstate->rslot);
+ return fmstate->fsstate->rslot;
+ }
+
/* Set up the prepared statement on the remote server, if we didn't yet */
if (!fmstate->p_name)
prepare_foreign_modify(fmstate);
***************
*** 1551,1557 **** postgresExecForeignDelete(EState *estate,
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(fmstate, slot, res);
}
else
n_rows = atoi(PQcmdTuples(res));
--- 1786,1796 ----
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(slot, res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
}
else
n_rows = atoi(PQcmdTuples(res));
***************
*** 1579,1584 **** postgresEndForeignModify(EState *estate,
--- 1818,1827 ----
if (fmstate == NULL)
return;
+ /* If pushing update down, nothing to do */
+ if (fmstate->update_is_pushed_down)
+ return;
+
/* If we created a prepared statement, destroy it */
if (fmstate->p_name)
{
***************
*** 1661,1667 **** postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
if (es->verbose)
{
fdw_private = ((ForeignScan *) node->ss.ps.plan)->fdw_private;
! sql = strVal(list_nth(fdw_private, FdwScanPrivateSelectSql));
ExplainPropertyText("Remote SQL", sql, es);
}
}
--- 1904,1918 ----
if (es->verbose)
{
fdw_private = ((ForeignScan *) node->ss.ps.plan)->fdw_private;
! if (list_length(fdw_private) == UpdateFdwScanPrivateLength)
! {
! sql = strVal(list_nth(fdw_private, FdwScanPrivateUpdateSql));
! }
! else
! {
! Assert(list_length(fdw_private) == SelectFdwScanPrivateLength);
! sql = strVal(list_nth(fdw_private, FdwScanPrivateSelectSql));
! }
ExplainPropertyText("Remote SQL", sql, es);
}
}
***************
*** 1682,1688 **** postgresExplainForeignModify(ModifyTableState *mtstate,
char *sql = strVal(list_nth(fdw_private,
FdwModifyPrivateUpdateSql));
! ExplainPropertyText("Remote SQL", sql, es);
}
}
--- 1933,1940 ----
char *sql = strVal(list_nth(fdw_private,
FdwModifyPrivateUpdateSql));
! if (sql != NULL)
! ExplainPropertyText("Remote SQL", sql, es);
}
}
***************
*** 1911,1916 **** ec_member_matches_foreign(PlannerInfo *root, RelOptInfo *rel,
--- 2163,2377 ----
}
/*
+ * Check whether it's safe to push the UPDATE/DELETE command down.
+ *
+ * Conditions checked here:
+ *
+ * 1. If the target relation has any BEFORE/AFTER row-level local triggers, we
+ * must not push the command down, since that breaks execution of the triggers.
+ *
+ * 2. If there are any local joins needed, we mustn't push the command down,
+ * because that breaks execution of the joins.
+ *
+ * 3. If there are any quals that can't be evaluated remotely, we mustn't push
+ * the command down, because that breaks evaluation of the quals.
+ *
+ * 4. In UPDATE, if it is unsafe to evaluate any expressions to assign to the
+ * target columns on the remote server, we must not push the command down.
+ */
+ static bool
+ update_is_pushdown_safe(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[resultRelation];
+ Plan *subplan = (Plan *) list_nth(plan->plans, subplan_index);
+ ListCell *lc;
+
+ /* Check point 1 */
+ if (rel->trigdesc &&
+ (rel->trigdesc->trig_update_after_row ||
+ rel->trigdesc->trig_update_before_row))
+ return false;
+
+ /* Check point 2 */
+ if (nodeTag(subplan) != T_ForeignScan)
+ return false;
+
+ /* Check point 3 */
+ if (subplan->qual != NIL)
+ return false;
+
+ /* Check point 4 */
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(subplan->targetlist,
+ attnum);
+
+ if (!is_foreign_expr(root, baserel, (Expr *) tle->expr))
+ return false;
+ }
+
+ return true;
+ }
+
+ /*
+ * Modify a plan so as to push the update command down.
+ */
+ static List *
+ push_update_down(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs)
+ {
+ CmdType operation = plan->operation;
+ bool canSetTag = plan->canSetTag;
+ Plan *subplan = (Plan *) list_nth(plan->plans, subplan_index);
+ ForeignScan *fscan = (ForeignScan *) subplan;
+ StringInfoData sql;
+ List *remote_conds;
+ List *returningList = NIL;
+ List *retrieved_attrs = NIL;
+ List *new_tlist = NIL;
+ List *fdw_private;
+
+ Assert(operation == CMD_UPDATE || operation == CMD_DELETE);
+
+ initStringInfo(&sql);
+
+ /*
+ * Extract the baserestrictinfo clauses that can be evaluated remotely.
+ */
+ remote_conds = (List *) list_nth(fscan->fdw_private,
+ FdwScanPrivateRemoteConds);
+
+ /*
+ * Extract the relevant RETURNING list if any.
+ */
+ if (plan->returningLists)
+ returningList = (List *) list_nth(plan->returningLists, subplan_index);
+
+ /*
+ * Construct the SQL command string.
+ */
+ if (operation == CMD_UPDATE)
+ {
+ List *targetlist = subplan->targetlist;
+
+ deparseDirectUpdateSql(&sql, root, resultRelation, rel,
+ remote_conds,
+ targetlist,
+ targetAttrs,
+ returningList,
+ &retrieved_attrs);
+ }
+ else
+ {
+ Assert(operation == CMD_DELETE);
+
+ deparseDirectDeleteSql(&sql, root, resultRelation, rel,
+ remote_conds,
+ returningList,
+ &retrieved_attrs);
+ }
+
+ /*
+ * Update the fdw_private list that will be available to the executor.
+ * Items in the list must match enum FdwScanPrivateIndex, above.
+ */
+ fscan->fdw_private = lappend(fscan->fdw_private, makeString(sql.data));
+ fscan->fdw_private = lappend(fscan->fdw_private, makeInteger(canSetTag));
+ fscan->fdw_private = lappend(fscan->fdw_private,
+ makeInteger((retrieved_attrs != NIL)));
+ fscan->fdw_private = lappend(fscan->fdw_private, retrieved_attrs);
+
+ /*
+ * Rrewrite the targetlist for an UPDATE command for safety of ExecProject.
+ * Note we ignore and do not reference result tuples in update pushdown case.
+ */
+ if (operation == CMD_UPDATE)
+ {
+ ListCell *lc;
+ int attrno = 1;
+ int numattrs = RelationGetNumberOfAttributes(rel);
+
+ foreach(lc, subplan->targetlist)
+ {
+ TargetEntry *tle = (TargetEntry *) lfirst(lc);
+
+ if (tle->resjunk)
+ {
+ new_tlist = lappend(new_tlist, tle);
+ continue;
+ }
+
+ if (attrno > numattrs)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("table row type and query-specified row type do not match"),
+ errdetail("Query has too many columns.")));
+
+ if (!list_member_int(targetAttrs, attrno))
+ new_tlist = lappend(new_tlist, tle);
+ else
+ {
+ Form_pg_attribute attr;
+ Oid atttype;
+ int32 atttypmod;
+ Oid attcollation;
+ Node *new_expr;
+ TargetEntry *new_tle;
+
+ attr = rel->rd_att->attrs[attrno - 1];
+
+ Assert(!attr->attisdropped);
+ atttype = attr->atttypid;
+ atttypmod = attr->atttypmod;
+ attcollation = attr->attcollation;
+
+ new_expr = (Node *) makeVar(resultRelation,
+ attrno,
+ atttype,
+ atttypmod,
+ attcollation,
+ 0);
+
+ new_tle = makeTargetEntry((Expr *) new_expr,
+ attrno,
+ pstrdup(NameStr(attr->attname)),
+ false);
+
+ new_tlist = lappend(new_tlist, new_tle);
+ }
+
+ attrno++;
+ }
+
+ if (attrno != numattrs + 1)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("table row type and query-specified row type do not match"),
+ errdetail("Query has too few columns.")));
+
+ subplan->targetlist = new_tlist;
+ }
+
+ /*
+ * Build the fdw_private list that will be available to the executor.
+ * Items in the list must match enum FdwModifyPrivateIndex, above.
+ */
+ fdw_private = list_make4(makeString(NULL), NIL, makeInteger(false), NIL);
+
+ return fdw_private;
+ }
+
+ /*
* Create cursor for node's query with current parameter values.
*/
static void
***************
*** 2258,2275 **** convert_prep_stmt_params(PgFdwModifyState *fmstate,
* have PG_TRY blocks to ensure this happens.
*/
static void
! store_returning_result(PgFdwModifyState *fmstate,
! TupleTableSlot *slot, PGresult *res)
{
PG_TRY();
{
HeapTuple newtup;
! newtup = make_tuple_from_result_row(res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
/* tuple will be deleted when it is cleared from the slot */
ExecStoreTuple(newtup, slot, InvalidBuffer, true);
}
--- 2719,2741 ----
* have PG_TRY blocks to ensure this happens.
*/
static void
! store_returning_result(TupleTableSlot *slot,
! PGresult *res,
! int row,
! Relation rel,
! AttInMetadata *attinmeta,
! List *retrieved_attrs,
! MemoryContext temp_context)
{
PG_TRY();
{
HeapTuple newtup;
! newtup = make_tuple_from_result_row(res, row,
! rel,
! attinmeta,
! retrieved_attrs,
! temp_context);
/* tuple will be deleted when it is cleared from the slot */
ExecStoreTuple(newtup, slot, InvalidBuffer, true);
}
*** a/contrib/postgres_fdw/postgres_fdw.h
--- b/contrib/postgres_fdw/postgres_fdw.h
***************
*** 66,75 **** extern void deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 66,87 ----
Index rtindex, Relation rel,
List *targetAttrs, List *returningList,
List **retrieved_attrs);
+ extern void deparseDirectUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List *targetlist,
+ List *targetAttrs,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseDeleteSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *returningList,
List **retrieved_attrs);
+ extern void deparseDirectDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseAnalyzeSizeSql(StringInfo buf, Relation rel);
extern void deparseAnalyzeSql(StringInfo buf, Relation rel,
List **retrieved_attrs);
*** a/contrib/postgres_fdw/sql/postgres_fdw.sql
--- b/contrib/postgres_fdw/sql/postgres_fdw.sql
***************
*** 314,331 **** INSERT INTO ft2 (c1,c2,c3) SELECT c1+1000,c2+100, c3 || c3 FROM ft2 LIMIT 20;
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
--- 314,338 ----
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c7 = DEFAULT WHERE c1 % 10 = 5 AND date(c4) = '1970-01-17'::date; -- can't be pushed down
+ UPDATE ft2 SET c7 = DEFAULT WHERE c1 % 10 = 5 AND date(c4) = '1970-01-17'::date;
+ EXPLAIN (verbose, costs off)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
*** a/doc/src/sgml/postgres-fdw.sgml
--- b/doc/src/sgml/postgres-fdw.sgml
***************
*** 414,419 ****
--- 414,428 ----
<literal>WHERE</> clauses are not sent to the remote server unless they use
only built-in data types, operators, and functions. Operators and
functions in the clauses must be <literal>IMMUTABLE</> as well.
+ For an <command>UPDATE</> or <command>DELETE</> query,
+ <filename>postgres_fdw</> attempts to optimize the query execution by
+ sending the whole query to the remote server if there are no query
+ <literal>WHERE</> clauses that cannot be sent to the remote server,
+ no local joins for the query, or no <literal>BEFORE</> or <literal>AFTER</>
+ row-level local triggers on the target table. In <command>UPDATE</>,
+ expressions to assign to target columns must use only built-in data types,
+ <literal>IMMUTABLE</> operators, and <literal>IMMUTABLE</> functions,
+ to reduce the risk of misexecution of the query.
</para>
<para>
Hi Hanada-san,
Thank you for the answer.
(2014/08/04 19:36), Shigeru Hanada wrote:
2014-07-25 16:30 GMT+09:00 Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>:
(2014/07/24 18:30), Shigeru Hanada wrote:
I'm not sure that I understand your question correctly, but the reason for
that is because foreign tables cannot have INSTEAD OF triggers.
Now I see the reason, but then I worry (though it unlikely happens) a
case that new trigger type might be added in future. The code says
that "only BEFORE and AFTER triggers are unsafe for direct update",
but it would be more safe to code that "any trigger other than event
trigger is unsafe for direct update".
Yeah, I've revised the comment for that in the updated version of the
patch I sent in just now. Could you check it?
We found that this patch speeds up DELETE case remarkably, as you
describe above, but we saw only less than 2x speed on UPDATE cases.
Do you have any numbers of UPDATE cases?
Hmm, performance gain on UPDATE cases seems similar to our results,
except planning times. In your environment the patch reduces planning
time too, but we got longer planning times with your patch (in only
once in six trial, we got shorter planning time than average of
patched version). Could you try multiple times on your environment?
No. Is the overhead so large that it cannot be ignored?
I think that the precise effect of this optimization for DELETE/UPDATE would
depend on eg, data, queries (inc. w/ or w/o RETRUNING clauses) and
server/network performance. Could you tell me these information about the
UPDATE evaluation?
I tried on a CentOS 6.5 on VMware on a Note PC with Core i3 1.17GHz,
2.0GB memory and single HDD, so the performance is poor.The SQLs used for performance test are quite simple, update 10
thousands rows at a time, and repeat it for different section of the
table for six times. The definition of foreign table ft is same as
the one in your case.EXPLAIN ANALYZE VERBOSE UPDATe ft SET data = 'abcdefg' WHERE id >= 0
AND id < 10000;
EXPLAIN ANALYZE VERBOSE UPDATe ft SET data = 'abcdefg' WHERE id >=
10000 AND id < 20000;
EXPLAIN ANALYZE VERBOSE UPDATe ft SET data = 'abcdefg' WHERE id >=
20000 AND id < 30000;
EXPLAIN ANALYZE VERBOSE UPDATe ft SET data = 'abcdefg' WHERE id >=
30000 AND id < 40000;
EXPLAIN ANALYZE VERBOSE UPDATe ft SET data = 'abcdefg' WHERE id >=
40000 AND id < 50000;
EXPLAIN ANALYZE VERBOSE UPDATe ft SET data = 'abcdefg' WHERE id >=
50000 AND id < 60000;
OK I also will evaluate the performance under the same workloads.
Some more random thoughts:
* Naming of new behavior
You named this optimization "Direct Update", but I'm not sure that
this is intuitive enough to express this behavior. I would like to
hear opinions of native speakers.
Update push-down seems nice with according to others.
The name has been changed in the updated version.
Thanks,
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hi Fujita-san,
Issues addressed by Eitoku-san were fixed properly, but he found a bug
and a possible enhancement in the v2 patch.
* push-down check misses delete triggers
update_is_pushdown_safe() seems to have a bug that it misses the
existence of row-level delete trigger. DELETE statement executed
against a foreign table which has row-level delete trigger is pushed
down to remote, and consequently no row-level delete trigger is fired.
* further optimization
Is there any chance to consider further optimization by passing the
operation type (UPDATE|DELETE) of undergoing statement to
update_is_pushdown_safe()? It seems safe to push down UPDATE
statement when the target foreign table has no update trigger even it
has a delete trigger (of course the opposite combination would be also
fine).
* Documentation
The requirement of pushing down UPDATE/DELETE statements would not be
easy to understand for non-expert users, so it seems that there is a
room to enhance documentation. An idea is to define which expression
is safe to send to remote first (it might need to mention the
difference of semantics), and refer the definition from the place
describing the requirement of pushing-down for SELECT, UPDATE and
DELETE.
2014-08-04 20:30 GMT+09:00 Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>:
(2014/07/30 17:22), Etsuro Fujita wrote:
(2014/07/29 0:58), Robert Haas wrote:
On Fri, Jul 25, 2014 at 3:39 AM, Albe Laurenz
<laurenz.albe@wien.gv.at> wrote:Shigeru Hanada wrote:
* Naming of new behavior
You named this optimization "Direct Update", but I'm not sure that
this is intuitive enough to express this behavior. I would like to
hear opinions of native speakers.How about "batch foreign update" or "batch foreign modification"?
(Disclaimer: I'm not a native speaker either.)I think direct update sounds pretty good. "Batch" does not sound as
good to me, since it doesn't clearly describe what makes this patch
special as opposed to some other grouping of updates that happens to
produce a speedup.I agree with Robert on that point.
Another term that might be used is "update pushdown", since we are
pushing the whole update to the remote server instead of having the
local server participate. Without looking at the patch, I don't have
a strong opinion on whether that's better than "direct update" in this
context."Update Pushdown" is fine with me.
If there are no objections of others, I'll change the name from "Direct
Update" to "Update Pushdown".Done. (I've left deparseDirectUpdateSql/deparseDirectDeleteSql as-is,
though.)Other changes:
* Address the comments from Eitoku-san.
* Add regression tests.
* Fix a bug, which fails to show the actual row counts in EXPLAIN ANALYZE
for UPDATE/DELETE without a RETURNING clause.
* Rebase to HEAD.Please find attached an updated version of the patch.
Thanks,
Best regards,
Etsuro Fujita
--
Shigeru HANADA
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
(2014/08/12 18:34), Shigeru Hanada wrote:
Issues addressed by Eitoku-san were fixed properly, but he found a bug
and a possible enhancement in the v2 patch.
Thank you for the review, Hanada-san and Eitoku-san!
* push-down check misses delete triggers
update_is_pushdown_safe() seems to have a bug that it misses the
existence of row-level delete trigger. DELETE statement executed
against a foreign table which has row-level delete trigger is pushed
down to remote, and consequently no row-level delete trigger is fired.
Ah, I noticed that the current code for that is not correct. Will fix.
* further optimization
Is there any chance to consider further optimization by passing the
operation type (UPDATE|DELETE) of undergoing statement to
update_is_pushdown_safe()? It seems safe to push down UPDATE
statement when the target foreign table has no update trigger even it
has a delete trigger (of course the opposite combination would be also
fine).
Good idea! Will improve that too.
* Documentation
The requirement of pushing down UPDATE/DELETE statements would not be
easy to understand for non-expert users, so it seems that there is a
room to enhance documentation. An idea is to define which expression
is safe to send to remote first (it might need to mention the
difference of semantics), and refer the definition from the place
describing the requirement of pushing-down for SELECT, UPDATE and
DELETE.
Yeah, I also think that it would not necessarily easy for the users to
understand which expression is safe to send. So I agree with that
enhancement, but ISTM that it would be better to do that as a separate
patch.
Thanks,
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Etsuro Fujita wrote:
Done. (I've left deparseDirectUpdateSql/deparseDirectDeleteSql as-is,
though.)Other changes:
* Address the comments from Eitoku-san.
* Add regression tests.
* Fix a bug, which fails to show the actual row counts in EXPLAIN
ANALYZE for UPDATE/DELETE without a RETURNING clause.
* Rebase to HEAD.Please find attached an updated version of the patch.
Here is my review:
The patch Applies fine, Builds without warning and passes make Check,
so the ABC of patch reviewing is fine.
I played with it, and apart from Hanada's comments I have found the following:
test=> EXPLAIN (ANALYZE, VERBOSE) UPDATE rtest SET val=NULL WHERE id > 3;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------
Update on laurenz.rtest (cost=100.00..14134.40 rows=299970 width=10) (actual time=0.005..0.005 rows=0 loops=1)
-> Foreign Scan on laurenz.rtest (cost=100.00..14134.40 rows=299970 width=10) (actual time=0.002..0.002 rows=299997 loops=1)
Output: id, val, ctid
Remote SQL: UPDATE laurenz.test SET val = NULL::text WHERE ((id > 3))
Planning time: 0.179 ms
Execution time: 3706.919 ms
(6 rows)
Time: 3708.272 ms
The "actual time" readings are surprising.
Shouldn't these similar to the actual execution time, since most of the time is spent
in the foreign scan node?
Reading the code, I noticed that the pushed down UPDATE or DELETE statement is executed
during postgresBeginForeignScan rather than during postgresIterateForeignScan.
It probably does not matter, but is there a reason to do it different from the normal scan?
It is not expected that postgresReScanForeignScan is called when the UPDATE/DELETE
is pushed down, right? Maybe it would make sense to add an assertion for that.
I ran a simple performance test and found that performance is improved as expected;
updating 100000 rows took 1000 rather than 8000 ms, and DELETING the same amount
took 200 instead of 6500 ms.
Yours,
Laurenz Albe
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
(2014/08/25 21:58), Albe Laurenz wrote:
Here is my review:
Thank you for the review!
I played with it, and apart from Hanada's comments I have found the following:
test=> EXPLAIN (ANALYZE, VERBOSE) UPDATE rtest SET val=NULL WHERE id > 3;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------
Update on laurenz.rtest (cost=100.00..14134.40 rows=299970 width=10) (actual time=0.005..0.005 rows=0 loops=1)
-> Foreign Scan on laurenz.rtest (cost=100.00..14134.40 rows=299970 width=10) (actual time=0.002..0.002 rows=299997 loops=1)
Output: id, val, ctid
Remote SQL: UPDATE laurenz.test SET val = NULL::text WHERE ((id > 3))
Planning time: 0.179 ms
Execution time: 3706.919 ms
(6 rows)Time: 3708.272 ms
The "actual time" readings are surprising.
Shouldn't these similar to the actual execution time, since most of the time is spent
in the foreign scan node?
I was also thinkng that this is confusing to the users. I think this is
because the patch executes the UPDATE/DELETE statement during
postgresBeginForeignScan, not postgresIterateForeignScan, as you
mentioned below:
Reading the code, I noticed that the pushed down UPDATE or DELETE statement is executed
during postgresBeginForeignScan rather than during postgresIterateForeignScan.
It probably does not matter, but is there a reason to do it different from the normal scan?
I'll modify the patch so as to execute the statement during
postgresIterateForeignScan.
It is not expected that postgresReScanForeignScan is called when the UPDATE/DELETE
is pushed down, right? Maybe it would make sense to add an assertion for that.
IIUC, that is right. As ModifyTable doesn't support rescan currently,
postgresReScanForeignScan needn't to be called in the update pushdown
case. The assertion is a good idea. I'll add it.
Thanks,
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Aug 25, 2014 at 8:58 AM, Albe Laurenz <laurenz.albe@wien.gv.at> wrote:
Reading the code, I noticed that the pushed down UPDATE or DELETE statement is executed
during postgresBeginForeignScan rather than during postgresIterateForeignScan.
It probably does not matter, but is there a reason to do it different from the normal scan?
Hmm, I'm worried that may be an API contract violation. ISTM that we
might initialize nodes that we never read from - they can show up in
the EXPLAIN-plan as (never executed) - and things that aren't executed
shouldn't do work, especially work that permanently modifies data.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Robert Haas <robertmhaas@gmail.com> writes:
On Mon, Aug 25, 2014 at 8:58 AM, Albe Laurenz <laurenz.albe@wien.gv.at> wrote:
Reading the code, I noticed that the pushed down UPDATE or DELETE statement is executed
during postgresBeginForeignScan rather than during postgresIterateForeignScan.
It probably does not matter, but is there a reason to do it different from the normal scan?
Hmm, I'm worried that may be an API contract violation.
Indeed it is. You could get away with it if you check the
EXEC_FLAG_EXPLAIN_ONLY flag before doing anything with visible
side-effects, but it's still pretty ugly.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
I wrote:
Robert Haas <robertmhaas@gmail.com> writes:
Hmm, I'm worried that may be an API contract violation.
Indeed it is. You could get away with it if you check the
EXEC_FLAG_EXPLAIN_ONLY flag before doing anything with visible
side-effects, but it's still pretty ugly.
Actually, there's another problem there. What of UPDATE or DELETE with a
LIMIT clause, which is something that seems to be coming down the pike:
https://commitfest.postgresql.org/action/patch_view?id=1550
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
(2014/08/27 22:56), Robert Haas wrote:
On Mon, Aug 25, 2014 at 8:58 AM, Albe Laurenz <laurenz.albe@wien.gv.at> wrote:
Reading the code, I noticed that the pushed down UPDATE or DELETE statement is executed
during postgresBeginForeignScan rather than during postgresIterateForeignScan.
It probably does not matter, but is there a reason to do it different from the normal scan?Hmm, I'm worried that may be an API contract violation.
Will fix.
Thanks,
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
(2014/08/27 23:05), Tom Lane wrote:
I wrote:
Robert Haas <robertmhaas@gmail.com> writes:
Hmm, I'm worried that may be an API contract violation.
Actually, there's another problem there. What of UPDATE or DELETE with a
LIMIT clause, which is something that seems to be coming down the pike:
https://commitfest.postgresql.org/action/patch_view?id=1550
I'd like to try to extend the functionality so as to push
UPDATE/DELETE-with-LIMIT down into the remote server if it's safe. But
I don't yet know if it's possible, because I started looking into the
UPDATE/DELETE-with-LIMIT patch just now ...
Thanks,
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
(2014/08/13 12:40), Etsuro Fujita wrote:
(2014/08/12 18:34), Shigeru Hanada wrote:
Issues addressed by Eitoku-san were fixed properly, but he found a bug
and a possible enhancement in the v2 patch.
* push-down check misses delete triggers
update_is_pushdown_safe() seems to have a bug that it misses the
existence of row-level delete trigger. DELETE statement executed
against a foreign table which has row-level delete trigger is pushed
down to remote, and consequently no row-level delete trigger is fired.Ah, I noticed that the current code for that is not correct. Will fix.
Done.
* further optimization
Is there any chance to consider further optimization by passing the
operation type (UPDATE|DELETE) of undergoing statement to
update_is_pushdown_safe()? It seems safe to push down UPDATE
statement when the target foreign table has no update trigger even it
has a delete trigger (of course the opposite combination would be also
fine).Good idea! Will improve that too.
Done.
* Documentation
The requirement of pushing down UPDATE/DELETE statements would not be
easy to understand for non-expert users, so it seems that there is a
room to enhance documentation. An idea is to define which expression
is safe to send to remote first (it might need to mention the
difference of semantics), and refer the definition from the place
describing the requirement of pushing-down for SELECT, UPDATE and
DELETE.Yeah, I also think that it would not necessarily easy for the users to
understand which expression is safe to send. So I agree with that
enhancement, but ISTM that it would be better to do that as a separate
patch.
As above, I'd like to leave this as another patch.
Please find attached the updated version of the patch.
Thanks,
Best regards,
Etsuro Fujita
Attachments:
postgres_fdw-update-v3.patchtext/x-diff; name=postgres_fdw-update-v3.patchDownload
*** a/contrib/postgres_fdw/deparse.c
--- b/contrib/postgres_fdw/deparse.c
***************
*** 188,197 **** is_foreign_expr(PlannerInfo *root,
if (!foreign_expr_walker((Node *) expr, &glob_cxt, &loc_cxt))
return false;
- /* Expressions examined here should be boolean, ie noncollatable */
- Assert(loc_cxt.collation == InvalidOid);
- Assert(loc_cxt.state == FDW_COLLATE_NONE);
-
/*
* An expression which includes any mutable functions can't be sent over
* because its result is not stable. For example, sending now() remote
--- 188,193 ----
***************
*** 927,932 **** deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 923,981 ----
}
/*
+ * deparse remote UPDATE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparseDirectUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List *targetlist,
+ List *targetAttrs, List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ List *params_list = NIL;
+ deparse_expr_cxt context;
+ bool first;
+ ListCell *lc;
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = NULL;
+
+ appendStringInfoString(buf, "UPDATE ");
+ deparseRelation(buf, rel);
+ appendStringInfoString(buf, " SET ");
+
+ first = true;
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(targetlist, attnum);
+
+ if (!first)
+ appendStringInfoString(buf, ", ");
+ first = false;
+
+ deparseColumnRef(buf, rtindex, attnum, root);
+ appendStringInfo(buf, " = ");
+ deparseExpr((Expr *) tle->expr, &context);
+ }
+ if (remote_conds)
+ appendWhereClause(buf, root, baserel, remote_conds,
+ true, ¶ms_list);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* deparse remote DELETE statement
*
* The statement text is appended to buf, and we also create an integer List
***************
*** 949,954 **** deparseDeleteSql(StringInfo buf, PlannerInfo *root,
--- 998,1030 ----
}
/*
+ * deparse remote DELETE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparseDirectDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ List *params_list = NIL;
+
+ appendStringInfoString(buf, "DELETE FROM ");
+ deparseRelation(buf, rel);
+ if (remote_conds)
+ appendWhereClause(buf, root, baserel, remote_conds,
+ true, ¶ms_list);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* Add a RETURNING clause, if needed, to an INSERT/UPDATE/DELETE.
*/
static void
*** a/contrib/postgres_fdw/expected/postgres_fdw.out
--- b/contrib/postgres_fdw/expected/postgres_fdw.out
***************
*** 998,1004 **** INSERT INTO ft2 (c1,c2,c3)
--- 998,1025 ----
(3 rows)
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ -> Foreign Scan on public.ft2
+ Output: c1, c2, NULL::integer, c3, c4, c5, c6, c7, c8, ctid
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 300), c3 = (c3 || '_update3'::text) WHERE ((("C 1" % 10) = 3))
+ (4 rows)
+
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Output: c1, c2, c3, c4, c5, c6, c7, c8
+ -> Foreign Scan on public.ft2
+ Output: c1, c2, NULL::integer, c3, c4, c5, c6, c7, c8, ctid
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 400), c3 = (c3 || '_update7'::text) WHERE ((("C 1" % 10) = 7)) RETURNING "C 1", c2, c3, c4, c5, c6, c7, c8
+ (5 rows)
+
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
c1 | c2 | c3 | c4 | c5 | c6 | c7 | c8
------+-----+--------------------+------------------------------+--------------------------+----+------------+-----
***************
*** 1108,1114 **** UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
--- 1129,1135 ----
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
***************
*** 1129,1144 **** UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
! QUERY PLAN
! ----------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
- Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1 RETURNING "C 1", c4
-> Foreign Scan on public.ft2
Output: ctid
! Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) FOR UPDATE
! (6 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
--- 1150,1164 ----
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
! QUERY PLAN
! --------------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
-> Foreign Scan on public.ft2
Output: ctid
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) RETURNING "C 1", c4
! (5 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
***************
*** 1249,1255 **** DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
--- 1269,1275 ----
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
***************
*** 2092,2097 **** SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
--- 2112,2130 ----
1104 | 204 | ddd |
(819 rows)
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c7 = DEFAULT WHERE c1 % 10 = 0 AND date(c4) = '1970-01-01'::date; -- can't be pushed down
+ QUERY PLAN
+ -----------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c7 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.ft2
+ Output: c1, c2, NULL::integer, c3, c4, c5, c6, 'ft2 '::character(10), c8, ctid
+ Filter: (date(ft2.c4) = '01-01-1970'::date)
+ Remote SQL: SELECT "C 1", c2, c3, c4, c5, c6, c8, ctid FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 0)) FOR UPDATE
+ (6 rows)
+
+ UPDATE ft2 SET c7 = DEFAULT WHERE c1 % 10 = 0 AND date(c4) = '1970-01-01'::date;
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
BEGIN
***************
*** 2233,2239 **** CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 2266,2272 ----
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 2392,2398 **** savepoint s3;
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 2425,2431 ----
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (-2) WHERE ((c2 = 42)) AND (("C 1" = 10))
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 2840,2845 **** NOTICE: NEW: (13,"test triggered !")
--- 2873,3082 ----
(0,27)
(1 row)
+ -- cleanup
+ DROP TRIGGER trig_row_before ON rem1;
+ DROP TRIGGER trig_row_after ON rem1;
+ -- Test update-pushdown functionality
+ -- Test with statement-level triggers
+ CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: f1, f2, ctid
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (4 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: ctid
+ Remote SQL: DELETE FROM public.loc1
+ (4 rows)
+
+ DROP TRIGGER trig_stmt_before ON rem1;
+ CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: f1, f2, ctid
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (4 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: ctid
+ Remote SQL: DELETE FROM public.loc1
+ (4 rows)
+
+ DROP TRIGGER trig_stmt_after ON rem1;
+ -- Test with row-level ON INSERT triggers
+ CREATE TRIGGER trig_row_before_insert
+ BEFORE INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: f1, f2, ctid
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (4 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: ctid
+ Remote SQL: DELETE FROM public.loc1
+ (4 rows)
+
+ DROP TRIGGER trig_row_before_insert ON rem1;
+ CREATE TRIGGER trig_row_after_insert
+ AFTER INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: f1, f2, ctid
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (4 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: ctid
+ Remote SQL: DELETE FROM public.loc1
+ (4 rows)
+
+ DROP TRIGGER trig_row_after_insert ON rem1;
+ -- Test with row-level ON UPDATE triggers
+ CREATE TRIGGER trig_row_before_update
+ BEFORE UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: ctid
+ Remote SQL: DELETE FROM public.loc1
+ (4 rows)
+
+ DROP TRIGGER trig_row_before_update ON rem1;
+ CREATE TRIGGER trig_row_after_update
+ AFTER UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+ -------------------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: ctid
+ Remote SQL: DELETE FROM public.loc1
+ (4 rows)
+
+ DROP TRIGGER trig_row_after_update ON rem1;
+ -- Test with row-level ON DELETE triggers
+ CREATE TRIGGER trig_row_before_delete
+ BEFORE DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: f1, f2, ctid
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (4 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ DROP TRIGGER trig_row_before_delete ON rem1;
+ CREATE TRIGGER trig_row_after_delete
+ AFTER DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: f1, f2, ctid
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (4 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+ ------------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ DROP TRIGGER trig_row_after_delete ON rem1;
-- ===================================================================
-- test IMPORT FOREIGN SCHEMA
-- ===================================================================
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
***************
*** 86,92 **** typedef struct PgFdwRelationInfo
* planner to executor. Currently we store:
*
* 1) SELECT statement text to be sent to the remote server
! * 2) Integer list of attribute numbers retrieved by the SELECT
*
* These items are indexed with the enum FdwScanPrivateIndex, so an item
* can be fetched with list_nth(). For example, to get the SELECT statement:
--- 86,97 ----
* planner to executor. Currently we store:
*
* 1) SELECT statement text to be sent to the remote server
! * 2) List of restriction clauses that can be executed remotely
! * 3) Integer list of attribute numbers retrieved by the SELECT
! * 4) UPDATE/DELETE statement text to be sent to the remote server
! * 5) Boolean flag showing if we set the command es_processed
! * 6) Boolean flag showing if the remote query has a RETURNING clause
! * 7) Integer list of attribute numbers retrieved by RETURNING, if any
*
* These items are indexed with the enum FdwScanPrivateIndex, so an item
* can be fetched with list_nth(). For example, to get the SELECT statement:
***************
*** 96,105 **** enum FdwScanPrivateIndex
{
/* SQL statement to execute remotely (as a String node) */
FdwScanPrivateSelectSql,
! /* Integer list of attribute numbers retrieved by the SELECT */
! FdwScanPrivateRetrievedAttrs
};
/*
* Similarly, this enum describes what's kept in the fdw_private list for
* a ModifyTable node referencing a postgres_fdw foreign table. We store:
--- 101,123 ----
{
/* SQL statement to execute remotely (as a String node) */
FdwScanPrivateSelectSql,
! /* List of restriction clauses that can be executed remotely */
! FdwScanPrivateRemoteConds,
! /* Integer list of attribute numbers retrieved by SELECT */
! FdwScanPrivateRetrievedAttrsBySelect,
! /* UPDATE/DELETE statement to execute remotely (as a String node) */
! FdwScanPrivateUpdateSql,
! /* set-processed flag (as an integer Value node) */
! FdwScanPrivateSetProcessed,
! /* has-returning flag (as an integer Value node) */
! FdwScanPrivateHasReturning,
! /* Integer list of attribute numbers retrieved by RETURNING */
! FdwScanPrivateRetrievedAttrsByReturning
};
+ #define SelectFdwScanPrivateLength 3
+ #define UpdateFdwScanPrivateLength 7
+
/*
* Similarly, this enum describes what's kept in the fdw_private list for
* a ModifyTable node referencing a postgres_fdw foreign table. We store:
***************
*** 131,138 **** typedef struct PgFdwScanState
AttInMetadata *attinmeta; /* attribute datatype conversion metadata */
/* extracted fdw_private data */
! char *query; /* text of SELECT command */
List *retrieved_attrs; /* list of retrieved attribute numbers */
/* for remote query execution */
PGconn *conn; /* connection for the scan */
--- 149,158 ----
AttInMetadata *attinmeta; /* attribute datatype conversion metadata */
/* extracted fdw_private data */
! char *query; /* text of SELECT or UPDATE/DELETE command */
List *retrieved_attrs; /* list of retrieved attribute numbers */
+ bool set_processed; /* do we set the command es_processed? */
+ bool has_returning; /* is there a RETURNING clause? */
/* for remote query execution */
PGconn *conn; /* connection for the scan */
***************
*** 152,157 **** typedef struct PgFdwScanState
--- 172,182 ----
int fetch_ct_2; /* Min(# of fetches done, 2) */
bool eof_reached; /* true if last fetch reached EOF */
+ /* for update pushdown */
+ bool update_is_pushed_down; /* is UPDATE/DELETE pushed down? */
+ PGresult *result; /* result of an UPDATE/DELETE query */
+ TupleTableSlot *rslot; /* slot containing the result tuple */
+
/* working memory contexts */
MemoryContext batch_cxt; /* context holding current batch of tuples */
MemoryContext temp_cxt; /* context for per-tuple temporary data */
***************
*** 180,185 **** typedef struct PgFdwModifyState
--- 205,214 ----
int p_nums; /* number of parameters to transmit */
FmgrInfo *p_flinfo; /* output conversion functions for them */
+ /* for update pushdown */
+ bool update_is_pushed_down; /* is UPDATE/DELETE pushed down? */
+ PgFdwScanState *fsstate; /* execution state of a foreign scan */
+
/* working memory context */
MemoryContext temp_cxt; /* context for per-tuple temporary data */
} PgFdwModifyState;
***************
*** 308,319 **** static bool ec_member_matches_foreign(PlannerInfo *root, RelOptInfo *rel,
static void create_cursor(ForeignScanState *node);
static void fetch_more_data(ForeignScanState *node);
static void close_cursor(PGconn *conn, unsigned int cursor_number);
static void prepare_foreign_modify(PgFdwModifyState *fmstate);
static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
ItemPointer tupleid,
TupleTableSlot *slot);
! static void store_returning_result(PgFdwModifyState *fmstate,
! TupleTableSlot *slot, PGresult *res);
static int postgresAcquireSampleRowsFunc(Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
--- 337,365 ----
static void create_cursor(ForeignScanState *node);
static void fetch_more_data(ForeignScanState *node);
static void close_cursor(PGconn *conn, unsigned int cursor_number);
+ static bool update_is_pushdown_safe(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs);
+ static List *push_update_down(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs);
static void prepare_foreign_modify(PgFdwModifyState *fmstate);
static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
ItemPointer tupleid,
TupleTableSlot *slot);
! static void store_returning_result(TupleTableSlot *slot,
! PGresult *res,
! int row,
! Relation rel,
! AttInMetadata *attinmeta,
! List *retrieved_attrs,
! MemoryContext temp_context);
static int postgresAcquireSampleRowsFunc(Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
***************
*** 852,858 **** postgresGetForeignPlan(PlannerInfo *root,
* Build the fdw_private list that will be available to the executor.
* Items in the list must match enum FdwScanPrivateIndex, above.
*/
! fdw_private = list_make2(makeString(sql.data),
retrieved_attrs);
/*
--- 898,905 ----
* Build the fdw_private list that will be available to the executor.
* Items in the list must match enum FdwScanPrivateIndex, above.
*/
! fdw_private = list_make3(makeString(sql.data),
! remote_conds,
retrieved_attrs);
/*
***************
*** 914,935 **** postgresBeginForeignScan(ForeignScanState *node, int eflags)
server = GetForeignServer(table->serverid);
user = GetUserMapping(userid, server->serverid);
/*
* Get connection to the foreign server. Connection manager will
* establish new connection if necessary.
*/
fsstate->conn = GetConnection(server, user, false);
- /* Assign a unique ID for my cursor */
- fsstate->cursor_number = GetCursorNumber(fsstate->conn);
- fsstate->cursor_exists = false;
-
- /* Get private info created by planner functions. */
- fsstate->query = strVal(list_nth(fsplan->fdw_private,
- FdwScanPrivateSelectSql));
- fsstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
- FdwScanPrivateRetrievedAttrs);
-
/* Create contexts for batches of tuples and per-tuple temp workspace. */
fsstate->batch_cxt = AllocSetContextCreate(estate->es_query_cxt,
"postgres_fdw tuple data",
--- 961,994 ----
server = GetForeignServer(table->serverid);
user = GetUserMapping(userid, server->serverid);
+ /* Get private info created by planner functions. */
+ if (list_length(fsplan->fdw_private) == UpdateFdwScanPrivateLength)
+ {
+ fsstate->query = strVal(list_nth(fsplan->fdw_private,
+ FdwScanPrivateUpdateSql));
+ fsstate->set_processed = intVal(list_nth(fsplan->fdw_private,
+ FdwScanPrivateSetProcessed));
+ fsstate->has_returning = intVal(list_nth(fsplan->fdw_private,
+ FdwScanPrivateHasReturning));
+ fsstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
+ FdwScanPrivateRetrievedAttrsByReturning);
+ }
+ else
+ {
+ Assert(list_length(fsplan->fdw_private) == SelectFdwScanPrivateLength);
+
+ fsstate->query = strVal(list_nth(fsplan->fdw_private,
+ FdwScanPrivateSelectSql));
+ fsstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
+ FdwScanPrivateRetrievedAttrsBySelect);
+ }
+
/*
* Get connection to the foreign server. Connection manager will
* establish new connection if necessary.
*/
fsstate->conn = GetConnection(server, user, false);
/* Create contexts for batches of tuples and per-tuple temp workspace. */
fsstate->batch_cxt = AllocSetContextCreate(estate->es_query_cxt,
"postgres_fdw tuple data",
***************
*** 945,950 **** postgresBeginForeignScan(ForeignScanState *node, int eflags)
--- 1004,1023 ----
/* Get info we'll need for input data conversion. */
fsstate->attinmeta = TupleDescGetAttInMetadata(RelationGetDescr(fsstate->rel));
+ /*
+ * If pushing update down, we've got no more to do.
+ */
+ if (list_length(fsplan->fdw_private) == UpdateFdwScanPrivateLength)
+ {
+ fsstate->num_tuples = -1; /* -1 means not set yet */
+ fsstate->update_is_pushed_down = true;
+ return;
+ }
+
+ /* Assign a unique ID for my cursor */
+ fsstate->cursor_number = GetCursorNumber(fsstate->conn);
+ fsstate->cursor_exists = false;
+
/* Prepare for output conversion of parameters used in remote query. */
numParams = list_length(fsplan->fdw_exprs);
fsstate->numParams = numParams;
***************
*** 995,1000 **** postgresIterateForeignScan(ForeignScanState *node)
--- 1068,1164 ----
TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
/*
+ * If pushing update down, get the result of UPDATE/DELETE RETURNING.
+ */
+ if (fsstate->update_is_pushed_down)
+ {
+ MemoryContext oldcontext;
+
+ /*
+ * If this is the first call after Begin, we need to execute the statement,
+ * and check for success.
+ */
+ if (fsstate->num_tuples == -1)
+ {
+ /*
+ * We don't use a PG_TRY block here, so be careful not to throw error
+ * without releasing the PGresult.
+ */
+ fsstate->result = PQexec(fsstate->conn, fsstate->query);
+ if (PQresultStatus(fsstate->result) !=
+ (fsstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
+ pgfdw_report_error(ERROR, fsstate->result, fsstate->conn, true,
+ fsstate->query);
+
+ /* Check number of rows affected. */
+ if (fsstate->has_returning)
+ fsstate->num_tuples = PQntuples(fsstate->result);
+ else
+ fsstate->num_tuples = atoi(PQcmdTuples(fsstate->result));
+ }
+
+ /*
+ * If the update query doesn't have a RETURNING clause, then there is
+ * nothing to do, so we just return an empty slot.
+ */
+ if (!fsstate->has_returning)
+ {
+ /*
+ * Increment the command es_processed count if necessary.
+ * (ModifyTable cannot do that by itself in this case.)
+ */
+ if (fsstate->set_processed)
+ {
+ EState *estate = node->ss.ps.state;
+
+ estate->es_processed += fsstate->num_tuples;
+ }
+
+ /*
+ * Increment the tuple count for EXPLAIN ANALYZE if necessary.
+ * (EXPLAIN ANALYZE cannot do that by itself in this case.)
+ */
+ if (node->ss.ps.instrument)
+ {
+ Instrumentation *instr = node->ss.ps.instrument;
+
+ instr->ntuples += fsstate->num_tuples;
+ }
+
+ return ExecClearTuple(slot);
+ }
+
+ /* If we didn't get any tuples, must be end of data. */
+ if (fsstate->next_tuple >= fsstate->num_tuples)
+ return ExecClearTuple(slot);
+
+ /* OK, we'll store RETURNING tuples in the batch_cxt. */
+ oldcontext = MemoryContextSwitchTo(fsstate->batch_cxt);
+
+ /* Fetch the next tuple. */
+ store_returning_result(slot,
+ fsstate->result,
+ fsstate->next_tuple,
+ fsstate->rel,
+ fsstate->attinmeta,
+ fsstate->retrieved_attrs,
+ fsstate->temp_cxt);
+ fsstate->rslot = slot;
+ fsstate->next_tuple++;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ /*
+ * Return slot. Note that this is safe because we can avoid applying
+ * ExecQual to the tuple due to no local quals (see the comment for
+ * update_is_pushdown_safe) and because the tuple can be safely
+ * projected by ExecProject (see push_update_down) and would then be
+ * ignored in postgresExecForeignUpdate or postgresExecForeignDelete.
+ */
+ return slot;
+ }
+
+ /*
* If this is the first call after Begin or ReScan, we need to create the
* cursor on the remote side.
*/
***************
*** 1036,1041 **** postgresReScanForeignScan(ForeignScanState *node)
--- 1200,1208 ----
char sql[64];
PGresult *res;
+ /* This shouldn't be called in update pushdown case. */
+ Assert(fsstate->update_is_pushed_down == false);
+
/* If we haven't created the cursor yet, nothing to do. */
if (!fsstate->cursor_exists)
return;
***************
*** 1094,1099 **** postgresEndForeignScan(ForeignScanState *node)
--- 1261,1273 ----
if (fsstate == NULL)
return;
+ /* if pushing update down, nothing to do other than cleanup */
+ if (fsstate->update_is_pushed_down)
+ {
+ if (fsstate->result)
+ PQclear(fsstate->result);
+ }
+
/* Close the cursor if open, to prevent accumulation of cursors */
if (fsstate->cursor_exists)
close_cursor(fsstate->conn, fsstate->cursor_number);
***************
*** 1167,1174 **** postgresPlanForeignModify(PlannerInfo *root,
List *returningList = NIL;
List *retrieved_attrs = NIL;
- initStringInfo(&sql);
-
/*
* Core code already has some lock on each rel being planned, so we can
* use NoLock here.
--- 1341,1346 ----
***************
*** 1210,1215 **** postgresPlanForeignModify(PlannerInfo *root,
--- 1382,1411 ----
}
/*
+ * For UPDATE/DELETE, if there are no local conditions or joins needed (see
+ * update_is_pushdown_safe for more details), we push the command down.
+ */
+ if (operation == CMD_UPDATE || operation == CMD_DELETE)
+ {
+ /* Check whether it's safe to push the command down. */
+ if (update_is_pushdown_safe(root, plan,
+ resultRelation,
+ subplan_index,
+ rel, targetAttrs))
+ {
+ List *fdw_private;
+
+ /* OK, modify plan so as to push the command down. */
+ fdw_private = push_update_down(root, plan,
+ resultRelation,
+ subplan_index,
+ rel, targetAttrs);
+ heap_close(rel, NoLock);
+ return fdw_private;
+ }
+ }
+
+ /*
* Extract the relevant RETURNING list if any.
*/
if (plan->returningLists)
***************
*** 1218,1223 **** postgresPlanForeignModify(PlannerInfo *root,
--- 1414,1420 ----
/*
* Construct the SQL command string.
*/
+ initStringInfo(&sql);
switch (operation)
{
case CMD_INSERT:
***************
*** 1288,1293 **** postgresBeginForeignModify(ModifyTableState *mtstate,
--- 1485,1526 ----
fmstate = (PgFdwModifyState *) palloc0(sizeof(PgFdwModifyState));
fmstate->rel = rel;
+ /* Deconstruct fdw_private data. */
+ fmstate->query = strVal(list_nth(fdw_private,
+ FdwModifyPrivateUpdateSql));
+ fmstate->target_attrs = (List *) list_nth(fdw_private,
+ FdwModifyPrivateTargetAttnums);
+ fmstate->has_returning = intVal(list_nth(fdw_private,
+ FdwModifyPrivateHasReturning));
+ fmstate->retrieved_attrs = (List *) list_nth(fdw_private,
+ FdwModifyPrivateRetrievedAttrs);
+
+ /*
+ * if query is NULL, we are in update pushdown case.
+ */
+ if (fmstate->query == NULL)
+ {
+ PlanState *node = mtstate->mt_plans[subplan_index];
+ PgFdwScanState *fsstate;
+
+ Assert(fmstate->target_attrs == NIL);
+ Assert(fmstate->has_returning == false);
+ Assert(fmstate->retrieved_attrs == NIL);
+
+ Assert(nodeTag(node) == T_ForeignScanState);
+ fsstate = (PgFdwScanState *) ((ForeignScanState *) node)->fdw_state;
+ Assert(fsstate->update_is_pushed_down);
+
+ fmstate->update_is_pushed_down = true;
+ if (fsstate->has_returning)
+ {
+ fmstate->has_returning = true;
+ fmstate->fsstate = fsstate;
+ }
+ resultRelInfo->ri_FdwState = fmstate;
+ return;
+ }
+
/*
* Identify which user to do the remote access as. This should match what
* ExecCheckRTEPerms() does.
***************
*** 1304,1319 **** postgresBeginForeignModify(ModifyTableState *mtstate,
fmstate->conn = GetConnection(server, user, true);
fmstate->p_name = NULL; /* prepared statement not made yet */
- /* Deconstruct fdw_private data. */
- fmstate->query = strVal(list_nth(fdw_private,
- FdwModifyPrivateUpdateSql));
- fmstate->target_attrs = (List *) list_nth(fdw_private,
- FdwModifyPrivateTargetAttnums);
- fmstate->has_returning = intVal(list_nth(fdw_private,
- FdwModifyPrivateHasReturning));
- fmstate->retrieved_attrs = (List *) list_nth(fdw_private,
- FdwModifyPrivateRetrievedAttrs);
-
/* Create context for per-tuple temp workspace. */
fmstate->temp_cxt = AllocSetContextCreate(estate->es_query_cxt,
"postgres_fdw temporary data",
--- 1537,1542 ----
***************
*** 1411,1417 **** postgresExecForeignInsert(EState *estate,
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(fmstate, slot, res);
}
else
n_rows = atoi(PQcmdTuples(res));
--- 1634,1644 ----
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(slot, res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
}
else
n_rows = atoi(PQcmdTuples(res));
***************
*** 1442,1447 **** postgresExecForeignUpdate(EState *estate,
--- 1669,1682 ----
PGresult *res;
int n_rows;
+ /* Just return slot created by the ForeignScan, if pushing update down */
+ if (fmstate->update_is_pushed_down)
+ {
+ Assert(fmstate->has_returning);
+ Assert(fmstate->fsstate->rslot);
+ return fmstate->fsstate->rslot;
+ }
+
/* Set up the prepared statement on the remote server, if we didn't yet */
if (!fmstate->p_name)
prepare_foreign_modify(fmstate);
***************
*** 1481,1487 **** postgresExecForeignUpdate(EState *estate,
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(fmstate, slot, res);
}
else
n_rows = atoi(PQcmdTuples(res));
--- 1716,1726 ----
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(slot, res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
}
else
n_rows = atoi(PQcmdTuples(res));
***************
*** 1512,1517 **** postgresExecForeignDelete(EState *estate,
--- 1751,1764 ----
PGresult *res;
int n_rows;
+ /* Just return slot created by the ForeignScan, if pushing update down */
+ if (fmstate->update_is_pushed_down)
+ {
+ Assert(fmstate->has_returning);
+ Assert(fmstate->fsstate->rslot);
+ return fmstate->fsstate->rslot;
+ }
+
/* Set up the prepared statement on the remote server, if we didn't yet */
if (!fmstate->p_name)
prepare_foreign_modify(fmstate);
***************
*** 1551,1557 **** postgresExecForeignDelete(EState *estate,
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(fmstate, slot, res);
}
else
n_rows = atoi(PQcmdTuples(res));
--- 1798,1808 ----
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(slot, res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
}
else
n_rows = atoi(PQcmdTuples(res));
***************
*** 1579,1584 **** postgresEndForeignModify(EState *estate,
--- 1830,1839 ----
if (fmstate == NULL)
return;
+ /* If pushing update down, nothing to do */
+ if (fmstate->update_is_pushed_down)
+ return;
+
/* If we created a prepared statement, destroy it */
if (fmstate->p_name)
{
***************
*** 1661,1667 **** postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
if (es->verbose)
{
fdw_private = ((ForeignScan *) node->ss.ps.plan)->fdw_private;
! sql = strVal(list_nth(fdw_private, FdwScanPrivateSelectSql));
ExplainPropertyText("Remote SQL", sql, es);
}
}
--- 1916,1930 ----
if (es->verbose)
{
fdw_private = ((ForeignScan *) node->ss.ps.plan)->fdw_private;
! if (list_length(fdw_private) == UpdateFdwScanPrivateLength)
! {
! sql = strVal(list_nth(fdw_private, FdwScanPrivateUpdateSql));
! }
! else
! {
! Assert(list_length(fdw_private) == SelectFdwScanPrivateLength);
! sql = strVal(list_nth(fdw_private, FdwScanPrivateSelectSql));
! }
ExplainPropertyText("Remote SQL", sql, es);
}
}
***************
*** 1682,1688 **** postgresExplainForeignModify(ModifyTableState *mtstate,
char *sql = strVal(list_nth(fdw_private,
FdwModifyPrivateUpdateSql));
! ExplainPropertyText("Remote SQL", sql, es);
}
}
--- 1945,1952 ----
char *sql = strVal(list_nth(fdw_private,
FdwModifyPrivateUpdateSql));
! if (sql != NULL)
! ExplainPropertyText("Remote SQL", sql, es);
}
}
***************
*** 1911,1916 **** ec_member_matches_foreign(PlannerInfo *root, RelOptInfo *rel,
--- 2175,2394 ----
}
/*
+ * Check whether it's safe to push the UPDATE/DELETE command down.
+ *
+ * Conditions checked here:
+ *
+ * 1. If the target relation has any row-level local BEFORE/AFTER triggers, we
+ * must not push the command down, since that breaks execution of the triggers.
+ *
+ * 2. If there are any local joins needed, we mustn't push the command down,
+ * because that breaks execution of the joins.
+ *
+ * 3. If there are any quals that can't be evaluated remotely, we mustn't push
+ * the command down, because that breaks evaluation of the quals.
+ *
+ * 4. In UPDATE, if it is unsafe to evaluate any expressions to assign to the
+ * target columns on the remote server, we must not push the command down.
+ */
+ static bool
+ update_is_pushdown_safe(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs)
+ {
+ CmdType operation = plan->operation;
+ RelOptInfo *baserel = root->simple_rel_array[resultRelation];
+ Plan *subplan = (Plan *) list_nth(plan->plans, subplan_index);
+ ListCell *lc;
+
+ /* Check point 1 */
+ if (rel->trigdesc &&
+ ((operation == CMD_UPDATE &&
+ (rel->trigdesc->trig_update_after_row ||
+ rel->trigdesc->trig_update_before_row)) ||
+ (operation == CMD_DELETE &&
+ (rel->trigdesc->trig_delete_after_row ||
+ rel->trigdesc->trig_delete_before_row))))
+ return false;
+
+ /* Check point 2 */
+ if (nodeTag(subplan) != T_ForeignScan)
+ return false;
+
+ /* Check point 3 */
+ if (subplan->qual != NIL)
+ return false;
+
+ /* Check point 4 */
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(subplan->targetlist,
+ attnum);
+
+ if (!is_foreign_expr(root, baserel, (Expr *) tle->expr))
+ return false;
+ }
+
+ return true;
+ }
+
+ /*
+ * Modify a plan so as to push the update command down.
+ */
+ static List *
+ push_update_down(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs)
+ {
+ CmdType operation = plan->operation;
+ bool canSetTag = plan->canSetTag;
+ Plan *subplan = (Plan *) list_nth(plan->plans, subplan_index);
+ ForeignScan *fscan = (ForeignScan *) subplan;
+ StringInfoData sql;
+ List *remote_conds;
+ List *returningList = NIL;
+ List *retrieved_attrs = NIL;
+ List *new_tlist = NIL;
+ List *fdw_private;
+
+ Assert(operation == CMD_UPDATE || operation == CMD_DELETE);
+
+ initStringInfo(&sql);
+
+ /*
+ * Extract the baserestrictinfo clauses that can be evaluated remotely.
+ */
+ remote_conds = (List *) list_nth(fscan->fdw_private,
+ FdwScanPrivateRemoteConds);
+
+ /*
+ * Extract the relevant RETURNING list if any.
+ */
+ if (plan->returningLists)
+ returningList = (List *) list_nth(plan->returningLists, subplan_index);
+
+ /*
+ * Construct the SQL command string.
+ */
+ if (operation == CMD_UPDATE)
+ {
+ List *targetlist = subplan->targetlist;
+
+ deparseDirectUpdateSql(&sql, root, resultRelation, rel,
+ remote_conds,
+ targetlist,
+ targetAttrs,
+ returningList,
+ &retrieved_attrs);
+ }
+ else
+ {
+ Assert(operation == CMD_DELETE);
+
+ deparseDirectDeleteSql(&sql, root, resultRelation, rel,
+ remote_conds,
+ returningList,
+ &retrieved_attrs);
+ }
+
+ /*
+ * Update the fdw_private list that will be available to the executor.
+ * Items in the list must match enum FdwScanPrivateIndex, above.
+ */
+ fscan->fdw_private = lappend(fscan->fdw_private, makeString(sql.data));
+ fscan->fdw_private = lappend(fscan->fdw_private, makeInteger(canSetTag));
+ fscan->fdw_private = lappend(fscan->fdw_private,
+ makeInteger((retrieved_attrs != NIL)));
+ fscan->fdw_private = lappend(fscan->fdw_private, retrieved_attrs);
+
+ /*
+ * Rrewrite the targetlist for an UPDATE command for safety of ExecProject.
+ * Note we ignore and do not reference result tuples in update pushdown case.
+ */
+ if (operation == CMD_UPDATE)
+ {
+ ListCell *lc;
+ int attrno = 1;
+ int numattrs = RelationGetNumberOfAttributes(rel);
+
+ foreach(lc, subplan->targetlist)
+ {
+ TargetEntry *tle = (TargetEntry *) lfirst(lc);
+
+ if (tle->resjunk)
+ {
+ new_tlist = lappend(new_tlist, tle);
+ continue;
+ }
+
+ if (attrno > numattrs)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("table row type and query-specified row type do not match"),
+ errdetail("Query has too many columns.")));
+
+ if (!list_member_int(targetAttrs, attrno))
+ new_tlist = lappend(new_tlist, tle);
+ else
+ {
+ Form_pg_attribute attr;
+ Oid atttype;
+ int32 atttypmod;
+ Oid attcollation;
+ Node *new_expr;
+ TargetEntry *new_tle;
+
+ attr = rel->rd_att->attrs[attrno - 1];
+
+ Assert(!attr->attisdropped);
+ atttype = attr->atttypid;
+ atttypmod = attr->atttypmod;
+ attcollation = attr->attcollation;
+
+ new_expr = (Node *) makeVar(resultRelation,
+ attrno,
+ atttype,
+ atttypmod,
+ attcollation,
+ 0);
+
+ new_tle = makeTargetEntry((Expr *) new_expr,
+ attrno,
+ pstrdup(NameStr(attr->attname)),
+ false);
+
+ new_tlist = lappend(new_tlist, new_tle);
+ }
+
+ attrno++;
+ }
+
+ if (attrno != numattrs + 1)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("table row type and query-specified row type do not match"),
+ errdetail("Query has too few columns.")));
+
+ subplan->targetlist = new_tlist;
+ }
+
+ /*
+ * Build the fdw_private list that will be available to the executor.
+ * Items in the list must match enum FdwModifyPrivateIndex, above.
+ */
+ fdw_private = list_make4(makeString(NULL), NIL, makeInteger(false), NIL);
+
+ return fdw_private;
+ }
+
+ /*
* Create cursor for node's query with current parameter values.
*/
static void
***************
*** 2258,2275 **** convert_prep_stmt_params(PgFdwModifyState *fmstate,
* have PG_TRY blocks to ensure this happens.
*/
static void
! store_returning_result(PgFdwModifyState *fmstate,
! TupleTableSlot *slot, PGresult *res)
{
PG_TRY();
{
HeapTuple newtup;
! newtup = make_tuple_from_result_row(res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
/* tuple will be deleted when it is cleared from the slot */
ExecStoreTuple(newtup, slot, InvalidBuffer, true);
}
--- 2736,2758 ----
* have PG_TRY blocks to ensure this happens.
*/
static void
! store_returning_result(TupleTableSlot *slot,
! PGresult *res,
! int row,
! Relation rel,
! AttInMetadata *attinmeta,
! List *retrieved_attrs,
! MemoryContext temp_context)
{
PG_TRY();
{
HeapTuple newtup;
! newtup = make_tuple_from_result_row(res, row,
! rel,
! attinmeta,
! retrieved_attrs,
! temp_context);
/* tuple will be deleted when it is cleared from the slot */
ExecStoreTuple(newtup, slot, InvalidBuffer, true);
}
*** a/contrib/postgres_fdw/postgres_fdw.h
--- b/contrib/postgres_fdw/postgres_fdw.h
***************
*** 66,75 **** extern void deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 66,87 ----
Index rtindex, Relation rel,
List *targetAttrs, List *returningList,
List **retrieved_attrs);
+ extern void deparseDirectUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List *targetlist,
+ List *targetAttrs,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseDeleteSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *returningList,
List **retrieved_attrs);
+ extern void deparseDirectDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseAnalyzeSizeSql(StringInfo buf, Relation rel);
extern void deparseAnalyzeSql(StringInfo buf, Relation rel,
List **retrieved_attrs);
*** a/contrib/postgres_fdw/sql/postgres_fdw.sql
--- b/contrib/postgres_fdw/sql/postgres_fdw.sql
***************
*** 320,339 **** INSERT INTO ft2 (c1,c2,c3) SELECT c1+1000,c2+100, c3 || c3 FROM ft2 LIMIT 20;
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
--- 320,346 ----
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c7 = DEFAULT WHERE c1 % 10 = 0 AND date(c4) = '1970-01-01'::date; -- can't be pushed down
+ UPDATE ft2 SET c7 = DEFAULT WHERE c1 % 10 = 0 AND date(c4) = '1970-01-01'::date;
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
***************
*** 616,621 **** UPDATE rem1 SET f2 = 'testo';
--- 623,711 ----
-- Test returning a system attribute
INSERT INTO rem1(f2) VALUES ('test') RETURNING ctid;
+ -- cleanup
+ DROP TRIGGER trig_row_before ON rem1;
+ DROP TRIGGER trig_row_after ON rem1;
+
+
+ -- Test update-pushdown functionality
+
+ -- Test with statement-level triggers
+ CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_stmt_before ON rem1;
+
+ CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_stmt_after ON rem1;
+
+ -- Test with row-level ON INSERT triggers
+ CREATE TRIGGER trig_row_before_insert
+ BEFORE INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_before_insert ON rem1;
+
+ CREATE TRIGGER trig_row_after_insert
+ AFTER INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_after_insert ON rem1;
+
+ -- Test with row-level ON UPDATE triggers
+ CREATE TRIGGER trig_row_before_update
+ BEFORE UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_before_update ON rem1;
+
+ CREATE TRIGGER trig_row_after_update
+ AFTER UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_after_update ON rem1;
+
+ -- Test with row-level ON DELETE triggers
+ CREATE TRIGGER trig_row_before_delete
+ BEFORE DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ DROP TRIGGER trig_row_before_delete ON rem1;
+
+ CREATE TRIGGER trig_row_after_delete
+ AFTER DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ DROP TRIGGER trig_row_after_delete ON rem1;
+
-- ===================================================================
-- test IMPORT FOREIGN SCHEMA
-- ===================================================================
*** a/doc/src/sgml/postgres-fdw.sgml
--- b/doc/src/sgml/postgres-fdw.sgml
***************
*** 414,419 ****
--- 414,428 ----
<literal>WHERE</> clauses are not sent to the remote server unless they use
only built-in data types, operators, and functions. Operators and
functions in the clauses must be <literal>IMMUTABLE</> as well.
+ For an <command>UPDATE</> or <command>DELETE</> query,
+ <filename>postgres_fdw</> attempts to optimize the query execution by
+ sending the whole query to the remote server if there are no query
+ <literal>WHERE</> clauses that cannot be sent to the remote server,
+ no local joins for the query, or no row-level local <literal>BEFORE</> or
+ <literal>AFTER</> triggers on the target table. In <command>UPDATE</>,
+ expressions to assign to target columns must use only built-in data types,
+ <literal>IMMUTABLE</> operators, and <literal>IMMUTABLE</> functions,
+ to reduce the risk of misexecution of the query.
</para>
<para>
(2014/08/26 12:20), Etsuro Fujita wrote:
(2014/08/25 21:58), Albe Laurenz wrote:
I played with it, and apart from Hanada's comments I have found the
following:test=> EXPLAIN (ANALYZE, VERBOSE) UPDATE rtest SET val=NULL WHERE id > 3;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------Update on laurenz.rtest (cost=100.00..14134.40 rows=299970
width=10) (actual time=0.005..0.005 rows=0 loops=1)
-> Foreign Scan on laurenz.rtest (cost=100.00..14134.40
rows=299970 width=10) (actual time=0.002..0.002 rows=299997 loops=1)
Output: id, val, ctid
Remote SQL: UPDATE laurenz.test SET val = NULL::text WHERE
((id > 3))
Planning time: 0.179 ms
Execution time: 3706.919 ms
(6 rows)Time: 3708.272 ms
The "actual time" readings are surprising.
Shouldn't these similar to the actual execution time, since most of
the time is spent
in the foreign scan node?I was also thinkng that this is confusing to the users. I think this is
because the patch executes the UPDATE/DELETE statement during
postgresBeginForeignScan, not postgresIterateForeignScan, as you
mentioned below:Reading the code, I noticed that the pushed down UPDATE or DELETE
statement is executed
during postgresBeginForeignScan rather than during
postgresIterateForeignScan.
I'll modify the patch so as to execute the statement during
postgresIterateForeignScan.
Done.
It is not expected that postgresReScanForeignScan is called when the
UPDATE/DELETE
is pushed down, right? Maybe it would make sense to add an assertion
for that.IIUC, that is right. As ModifyTable doesn't support rescan currently,
postgresReScanForeignScan needn't to be called in the update pushdown
case. The assertion is a good idea. I'll add it.
Done.
You can find the updated version of the patch at
/messages/by-id/53FFFA50.6020007@lab.ntt.co.jp
Thanks,
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
I confirmed performance gain accomplished by this patch.
This patch makes update queries ~50x faster, and even hit-miss update
is 3x faster than original. Of course benefit is only for queries
whose qualifiers are enough simple so that they can be pushied down
fully, but this improvement is remarkable.
This patch avoids 1) SELECT for determining target rows, and 2)
repeated per-row UPDATE/DELETE in particular situation, so I assumed
that the gain is larger for bulk update, and it's true indeed, but in
fact even hit-miss update (0 row affected) become faster enough. This
would come from the omission of SELECT preceding repeated
UPDATE/DELETE.
I was little worried about overhead in planning phase, but fluctuation
was less than 1ms, so it's negligible.
Measurement Result
==================
Note: numbers below are "execution time" of EXPLAIN ANALYZE, and
average of five runs
--------------+-------------+-----------+--------
rows affected | original | patched | gain
--------------+-------------+-----------+--------
0 | 4.841 | 1.548 | 3.13x
1 | 6.944 | 1.793 | 3.87x
100 | 174.420 | 5.167 | 33.76x
10,000 | 8,215.551 | 163.832 | 50.15x
100,000 | 78,135.905 | 1,595.739 | 48.97x
200,000 | 179,784.928 | 4,305.856 | 41.75x
--------------+-------------+-----------+--------
Measurement procedure
=====================
[Local side]
1) Create foreign table which refers pgbench_accounts on the remote side
[Remote side]
2) pgbench -i -s 100
3) Execute ANALYZE
4) Restart PostgreSQL to clear shared buffers
[Local side]
5) Execute ANALYZE against foreign table
6) Execute UPDATE SQL against foreign table once for warm the cache
7) Execute UPDATE SQL against foreign table five times
Test SQL for 10000-rows cas is below, only aid condition is changed
according to measurement variation.
EXPLAIN ANALYZE VERBOSE UPDATE ft_pgbench_accounts SET bid=bid+1,
abalance=abalance+1, filler='update test' WHERE aid<=10000;
2014-08-29 12:59 GMT+09:00 Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>:
(2014/08/26 12:20), Etsuro Fujita wrote:
(2014/08/25 21:58), Albe Laurenz wrote:
I played with it, and apart from Hanada's comments I have found the
following:test=> EXPLAIN (ANALYZE, VERBOSE) UPDATE rtest SET val=NULL WHERE id > 3;
QUERY PLAN----------------------------------------------------------------------------------------------------------------------------------
Update on laurenz.rtest (cost=100.00..14134.40 rows=299970
width=10) (actual time=0.005..0.005 rows=0 loops=1)
-> Foreign Scan on laurenz.rtest (cost=100.00..14134.40
rows=299970 width=10) (actual time=0.002..0.002 rows=299997 loops=1)
Output: id, val, ctid
Remote SQL: UPDATE laurenz.test SET val = NULL::text WHERE
((id > 3))
Planning time: 0.179 ms
Execution time: 3706.919 ms
(6 rows)Time: 3708.272 ms
The "actual time" readings are surprising.
Shouldn't these similar to the actual execution time, since most of
the time is spent
in the foreign scan node?I was also thinkng that this is confusing to the users. I think this is
because the patch executes the UPDATE/DELETE statement during
postgresBeginForeignScan, not postgresIterateForeignScan, as you
mentioned below:Reading the code, I noticed that the pushed down UPDATE or DELETE
statement is executed
during postgresBeginForeignScan rather than during
postgresIterateForeignScan.I'll modify the patch so as to execute the statement during
postgresIterateForeignScan.Done.
It is not expected that postgresReScanForeignScan is called when the
UPDATE/DELETE
is pushed down, right? Maybe it would make sense to add an assertion
for that.IIUC, that is right. As ModifyTable doesn't support rescan currently,
postgresReScanForeignScan needn't to be called in the update pushdown
case. The assertion is a good idea. I'll add it.Done.
You can find the updated version of the patch at
/messages/by-id/53FFFA50.6020007@lab.ntt.co.jp
Thanks,
Best regards,
Etsuro Fujita
--
Shigeru HANADA
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Etsuro Fujita wrote:
Please find attached the updated version of the patch.
I gave it a spin and could not find any undesirable behaviour, and the
output of EXPLAIN ANALYZE looks like I'd expect.
I noticed that you use the list length of fdw_private to check if
the UPDATE or DELETE is pushed down to the remote server or not.
While this works fine, I wonder if it wouldn't be better to have some
explicit flag in fdw_private for that purpose. Future modifications that
change the list length might easily overlook that it is used for this
purpose, thereby breaking the code.
Other than that it looks alright to me.
Yours,
Laurenz Albe
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
I wrote:
I gave it a spin and could not find any undesirable behaviour, and the
output of EXPLAIN ANALYZE looks like I'd expect.I noticed that you use the list length of fdw_private to check if
the UPDATE or DELETE is pushed down to the remote server or not.While this works fine, I wonder if it wouldn't be better to have some
explicit flag in fdw_private for that purpose. Future modifications that
change the list length might easily overlook that it is used for this
purpose, thereby breaking the code.Other than that it looks alright to me.
Maybe I should have mentioned that I have set the patch to "Waiting for Author"
because I'd like to hear your opinion on that, but I'm prepared to set it
to "Ready for Committer" soon.
Yours,
Laurenz Albe
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
(2014/09/08 16:18), Albe Laurenz wrote:
I wrote:
I gave it a spin and could not find any undesirable behaviour, and the
output of EXPLAIN ANALYZE looks like I'd expect.
Thank you for the review!
I noticed that you use the list length of fdw_private to check if
the UPDATE or DELETE is pushed down to the remote server or not.While this works fine, I wonder if it wouldn't be better to have some
explicit flag in fdw_private for that purpose. Future modifications that
change the list length might easily overlook that it is used for this
purpose, thereby breaking the code.
Other than that it looks alright to me.
Maybe I should have mentioned that I have set the patch to "Waiting for Author"
because I'd like to hear your opinion on that, but I'm prepared to set it
to "Ready for Committer" soon.
I agree with you on that point. So, I've updated the patch to have the
explicit flag, as you proposed. Attached is the updated version of the
patch. In this version, I've also revised code and its comments a bit.
Sorry for the delay.
Best regards,
Etsuro Fujita
Attachments:
postgres_fdw-update-v4.patchtext/x-patch; name=postgres_fdw-update-v4.patchDownload
*** a/contrib/postgres_fdw/deparse.c
--- b/contrib/postgres_fdw/deparse.c
***************
*** 188,197 **** is_foreign_expr(PlannerInfo *root,
if (!foreign_expr_walker((Node *) expr, &glob_cxt, &loc_cxt))
return false;
- /* Expressions examined here should be boolean, ie noncollatable */
- Assert(loc_cxt.collation == InvalidOid);
- Assert(loc_cxt.state == FDW_COLLATE_NONE);
-
/*
* An expression which includes any mutable functions can't be sent over
* because its result is not stable. For example, sending now() remote
--- 188,193 ----
***************
*** 927,932 **** deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 923,982 ----
}
/*
+ * deparse remote UPDATE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparseDirectUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List *targetlist,
+ List *targetAttrs,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ List *params_list = NIL;
+ deparse_expr_cxt context;
+ bool first;
+ ListCell *lc;
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = ¶ms_list;
+
+ appendStringInfoString(buf, "UPDATE ");
+ deparseRelation(buf, rel);
+ appendStringInfoString(buf, " SET ");
+
+ first = true;
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(targetlist, attnum);
+
+ if (!first)
+ appendStringInfoString(buf, ", ");
+ first = false;
+
+ deparseColumnRef(buf, rtindex, attnum, root);
+ appendStringInfo(buf, " = ");
+ deparseExpr((Expr *) tle->expr, &context);
+ }
+ if (remote_conds)
+ appendWhereClause(buf, root, baserel, remote_conds,
+ true, ¶ms_list);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* deparse remote DELETE statement
*
* The statement text is appended to buf, and we also create an integer List
***************
*** 949,954 **** deparseDeleteSql(StringInfo buf, PlannerInfo *root,
--- 999,1031 ----
}
/*
+ * deparse remote DELETE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparseDirectDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ List *params_list = NIL;
+
+ appendStringInfoString(buf, "DELETE FROM ");
+ deparseRelation(buf, rel);
+ if (remote_conds)
+ appendWhereClause(buf, root, baserel, remote_conds,
+ true, ¶ms_list);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* Add a RETURNING clause, if needed, to an INSERT/UPDATE/DELETE.
*/
static void
*** a/contrib/postgres_fdw/expected/postgres_fdw.out
--- b/contrib/postgres_fdw/expected/postgres_fdw.out
***************
*** 998,1004 **** INSERT INTO ft2 (c1,c2,c3)
--- 998,1025 ----
(3 rows)
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ -> Foreign Scan on public.ft2
+ Output: c1, c2, NULL::integer, c3, c4, c5, c6, c7, c8, ctid
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 300), c3 = (c3 || '_update3'::text) WHERE ((("C 1" % 10) = 3))
+ (4 rows)
+
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Output: c1, c2, c3, c4, c5, c6, c7, c8
+ -> Foreign Scan on public.ft2
+ Output: c1, c2, NULL::integer, c3, c4, c5, c6, c7, c8, ctid
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 400), c3 = (c3 || '_update7'::text) WHERE ((("C 1" % 10) = 7)) RETURNING "C 1", c2, c3, c4, c5, c6, c7, c8
+ (5 rows)
+
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
c1 | c2 | c3 | c4 | c5 | c6 | c7 | c8
------+-----+--------------------+------------------------------+--------------------------+----+------------+-----
***************
*** 1108,1114 **** UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
--- 1129,1135 ----
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
***************
*** 1129,1144 **** UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
! QUERY PLAN
! ----------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
- Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1 RETURNING "C 1", c4
-> Foreign Scan on public.ft2
Output: ctid
! Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) FOR UPDATE
! (6 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
--- 1150,1164 ----
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
! QUERY PLAN
! --------------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
-> Foreign Scan on public.ft2
Output: ctid
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) RETURNING "C 1", c4
! (5 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
***************
*** 1249,1255 **** DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
--- 1269,1275 ----
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
***************
*** 2092,2097 **** SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
--- 2112,2130 ----
1104 | 204 | ddd |
(819 rows)
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c7 = DEFAULT WHERE c1 % 10 = 0 AND date(c4) = '1970-01-01'::date; -- can't be pushed down
+ QUERY PLAN
+ -----------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c7 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.ft2
+ Output: c1, c2, NULL::integer, c3, c4, c5, c6, 'ft2 '::character(10), c8, ctid
+ Filter: (date(ft2.c4) = '01-01-1970'::date)
+ Remote SQL: SELECT "C 1", c2, c3, c4, c5, c6, c8, ctid FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 0)) FOR UPDATE
+ (6 rows)
+
+ UPDATE ft2 SET c7 = DEFAULT WHERE c1 % 10 = 0 AND date(c4) = '1970-01-01'::date;
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
BEGIN
***************
*** 2233,2239 **** CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 2266,2272 ----
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 2392,2398 **** savepoint s3;
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 2425,2431 ----
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (-2) WHERE ((c2 = 42)) AND (("C 1" = 10))
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 2840,2845 **** NOTICE: NEW: (13,"test triggered !")
--- 2873,3083 ----
(0,27)
(1 row)
+ -- cleanup
+ DROP TRIGGER trig_row_before ON rem1;
+ DROP TRIGGER trig_row_after ON rem1;
+ DROP TRIGGER trig_local_before ON loc1;
+ -- Test update-pushdown functionality
+ -- Test with statement-level triggers
+ CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: f1, f2, ctid
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (4 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: ctid
+ Remote SQL: DELETE FROM public.loc1
+ (4 rows)
+
+ DROP TRIGGER trig_stmt_before ON rem1;
+ CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: f1, f2, ctid
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (4 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: ctid
+ Remote SQL: DELETE FROM public.loc1
+ (4 rows)
+
+ DROP TRIGGER trig_stmt_after ON rem1;
+ -- Test with row-level ON INSERT triggers
+ CREATE TRIGGER trig_row_before_insert
+ BEFORE INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: f1, f2, ctid
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (4 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: ctid
+ Remote SQL: DELETE FROM public.loc1
+ (4 rows)
+
+ DROP TRIGGER trig_row_before_insert ON rem1;
+ CREATE TRIGGER trig_row_after_insert
+ AFTER INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: f1, f2, ctid
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (4 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: ctid
+ Remote SQL: DELETE FROM public.loc1
+ (4 rows)
+
+ DROP TRIGGER trig_row_after_insert ON rem1;
+ -- Test with row-level ON UPDATE triggers
+ CREATE TRIGGER trig_row_before_update
+ BEFORE UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: ctid
+ Remote SQL: DELETE FROM public.loc1
+ (4 rows)
+
+ DROP TRIGGER trig_row_before_update ON rem1;
+ CREATE TRIGGER trig_row_after_update
+ AFTER UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+ -------------------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: ctid
+ Remote SQL: DELETE FROM public.loc1
+ (4 rows)
+
+ DROP TRIGGER trig_row_after_update ON rem1;
+ -- Test with row-level ON DELETE triggers
+ CREATE TRIGGER trig_row_before_delete
+ BEFORE DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: f1, f2, ctid
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (4 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ DROP TRIGGER trig_row_before_delete ON rem1;
+ CREATE TRIGGER trig_row_after_delete
+ AFTER DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Scan on public.rem1
+ Output: f1, f2, ctid
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (4 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+ ------------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ DROP TRIGGER trig_row_after_delete ON rem1;
-- ===================================================================
-- test IMPORT FOREIGN SCHEMA
-- ===================================================================
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
***************
*** 86,92 **** typedef struct PgFdwRelationInfo
* planner to executor. Currently we store:
*
* 1) SELECT statement text to be sent to the remote server
! * 2) Integer list of attribute numbers retrieved by the SELECT
*
* These items are indexed with the enum FdwScanPrivateIndex, so an item
* can be fetched with list_nth(). For example, to get the SELECT statement:
--- 86,98 ----
* planner to executor. Currently we store:
*
* 1) SELECT statement text to be sent to the remote server
! * 2) List of restriction clauses that can be executed remotely
! * 3) Integer list of attribute numbers retrieved by the SELECT
! * 4) Boolean flag showing if an UPDATE/DELETE is pushed down
! * 5) UPDATE/DELETE statement text to be sent to the remote server
! * 6) Boolean flag showing if we set the command es_processed
! * 7) Boolean flag showing if the remote query has a RETURNING clause
! * 8) Integer list of attribute numbers retrieved by RETURNING, if any
*
* These items are indexed with the enum FdwScanPrivateIndex, so an item
* can be fetched with list_nth(). For example, to get the SELECT statement:
***************
*** 96,103 **** enum FdwScanPrivateIndex
{
/* SQL statement to execute remotely (as a String node) */
FdwScanPrivateSelectSql,
! /* Integer list of attribute numbers retrieved by the SELECT */
! FdwScanPrivateRetrievedAttrs
};
/*
--- 102,121 ----
{
/* SQL statement to execute remotely (as a String node) */
FdwScanPrivateSelectSql,
! /* List of restriction clauses that can be executed remotely */
! FdwScanPrivateRemoteConds,
! /* Integer list of attribute numbers retrieved by SELECT */
! FdwScanPrivateRetrievedAttrsBySelect,
! /* UPDATE/DELETE-pushdown flag (as an integer Value node) */
! FdwScanPrivateUpdatePushdown,
! /* UPDATE/DELETE statement to execute remotely (as a String node) */
! FdwScanPrivateUpdateSql,
! /* set-processed flag (as an integer Value node) */
! FdwScanPrivateSetProcessed,
! /* has-returning flag (as an integer Value node) */
! FdwScanPrivateHasReturning,
! /* Integer list of attribute numbers retrieved by RETURNING */
! FdwScanPrivateRetrievedAttrsByReturning
};
/*
***************
*** 131,138 **** typedef struct PgFdwScanState
AttInMetadata *attinmeta; /* attribute datatype conversion metadata */
/* extracted fdw_private data */
! char *query; /* text of SELECT command */
List *retrieved_attrs; /* list of retrieved attribute numbers */
/* for remote query execution */
PGconn *conn; /* connection for the scan */
--- 149,158 ----
AttInMetadata *attinmeta; /* attribute datatype conversion metadata */
/* extracted fdw_private data */
! char *query; /* text of SELECT or UPDATE/DELETE command */
List *retrieved_attrs; /* list of retrieved attribute numbers */
+ bool set_processed; /* do we set the command es_processed? */
+ bool has_returning; /* is there a RETURNING clause? */
/* for remote query execution */
PGconn *conn; /* connection for the scan */
***************
*** 152,157 **** typedef struct PgFdwScanState
--- 172,182 ----
int fetch_ct_2; /* Min(# of fetches done, 2) */
bool eof_reached; /* true if last fetch reached EOF */
+ /* for update pushdown */
+ bool update_pushdown; /* is an UPDATE/DELETE pushed down? */
+ PGresult *result; /* result of an UPDATE/DELETE query */
+ TupleTableSlot *rslot; /* slot containing the result tuple */
+
/* working memory contexts */
MemoryContext batch_cxt; /* context holding current batch of tuples */
MemoryContext temp_cxt; /* context for per-tuple temporary data */
***************
*** 180,185 **** typedef struct PgFdwModifyState
--- 205,214 ----
int p_nums; /* number of parameters to transmit */
FmgrInfo *p_flinfo; /* output conversion functions for them */
+ /* for update pushdown */
+ bool update_pushdown; /* is an UPDATE/DELETE pushed down? */
+ PgFdwScanState *fsstate; /* execution state of a foreign scan */
+
/* working memory context */
MemoryContext temp_cxt; /* context for per-tuple temporary data */
} PgFdwModifyState;
***************
*** 308,319 **** static bool ec_member_matches_foreign(PlannerInfo *root, RelOptInfo *rel,
static void create_cursor(ForeignScanState *node);
static void fetch_more_data(ForeignScanState *node);
static void close_cursor(PGconn *conn, unsigned int cursor_number);
static void prepare_foreign_modify(PgFdwModifyState *fmstate);
static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
ItemPointer tupleid,
TupleTableSlot *slot);
! static void store_returning_result(PgFdwModifyState *fmstate,
! TupleTableSlot *slot, PGresult *res);
static int postgresAcquireSampleRowsFunc(Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
--- 337,367 ----
static void create_cursor(ForeignScanState *node);
static void fetch_more_data(ForeignScanState *node);
static void close_cursor(PGconn *conn, unsigned int cursor_number);
+ static bool update_is_pushdown_safe(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs);
+ static List *push_update_down(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs);
+ static List *rewrite_targetlist(Index resultRelation, Relation rel,
+ List *targetlist, List *targetAttrs);
static void prepare_foreign_modify(PgFdwModifyState *fmstate);
static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
ItemPointer tupleid,
TupleTableSlot *slot);
! static void store_returning_result(TupleTableSlot *slot,
! PGresult *res,
! int row,
! Relation rel,
! AttInMetadata *attinmeta,
! List *retrieved_attrs,
! MemoryContext temp_context);
static int postgresAcquireSampleRowsFunc(Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
***************
*** 852,859 **** postgresGetForeignPlan(PlannerInfo *root,
* Build the fdw_private list that will be available to the executor.
* Items in the list must match enum FdwScanPrivateIndex, above.
*/
! fdw_private = list_make2(makeString(sql.data),
! retrieved_attrs);
/*
* Create the ForeignScan node from target list, local filtering
--- 900,909 ----
* Build the fdw_private list that will be available to the executor.
* Items in the list must match enum FdwScanPrivateIndex, above.
*/
! fdw_private = list_make4(makeString(sql.data),
! remote_conds,
! retrieved_attrs,
! makeInteger(false));
/*
* Create the ForeignScan node from target list, local filtering
***************
*** 914,935 **** postgresBeginForeignScan(ForeignScanState *node, int eflags)
server = GetForeignServer(table->serverid);
user = GetUserMapping(userid, server->serverid);
/*
* Get connection to the foreign server. Connection manager will
* establish new connection if necessary.
*/
fsstate->conn = GetConnection(server, user, false);
- /* Assign a unique ID for my cursor */
- fsstate->cursor_number = GetCursorNumber(fsstate->conn);
- fsstate->cursor_exists = false;
-
- /* Get private info created by planner functions. */
- fsstate->query = strVal(list_nth(fsplan->fdw_private,
- FdwScanPrivateSelectSql));
- fsstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
- FdwScanPrivateRetrievedAttrs);
-
/* Create contexts for batches of tuples and per-tuple temp workspace. */
fsstate->batch_cxt = AllocSetContextCreate(estate->es_query_cxt,
"postgres_fdw tuple data",
--- 964,997 ----
server = GetForeignServer(table->serverid);
user = GetUserMapping(userid, server->serverid);
+ /* Get private info created by planner functions. */
+ fsstate->update_pushdown = intVal(list_nth(fsplan->fdw_private,
+ FdwScanPrivateUpdatePushdown));
+ if (fsstate->update_pushdown)
+ {
+ fsstate->query = strVal(list_nth(fsplan->fdw_private,
+ FdwScanPrivateUpdateSql));
+ fsstate->set_processed = intVal(list_nth(fsplan->fdw_private,
+ FdwScanPrivateSetProcessed));
+ fsstate->has_returning = intVal(list_nth(fsplan->fdw_private,
+ FdwScanPrivateHasReturning));
+ fsstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
+ FdwScanPrivateRetrievedAttrsByReturning);
+ }
+ else
+ {
+ fsstate->query = strVal(list_nth(fsplan->fdw_private,
+ FdwScanPrivateSelectSql));
+ fsstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
+ FdwScanPrivateRetrievedAttrsBySelect);
+ }
+
/*
* Get connection to the foreign server. Connection manager will
* establish new connection if necessary.
*/
fsstate->conn = GetConnection(server, user, false);
/* Create contexts for batches of tuples and per-tuple temp workspace. */
fsstate->batch_cxt = AllocSetContextCreate(estate->es_query_cxt,
"postgres_fdw tuple data",
***************
*** 945,950 **** postgresBeginForeignScan(ForeignScanState *node, int eflags)
--- 1007,1025 ----
/* Get info we'll need for input data conversion. */
fsstate->attinmeta = TupleDescGetAttInMetadata(RelationGetDescr(fsstate->rel));
+ /*
+ * If pushing update down, we've got no more to do.
+ */
+ if (fsstate->update_pushdown)
+ {
+ fsstate->num_tuples = -1; /* -1 means not set yet */
+ return;
+ }
+
+ /* Assign a unique ID for my cursor */
+ fsstate->cursor_number = GetCursorNumber(fsstate->conn);
+ fsstate->cursor_exists = false;
+
/* Prepare for output conversion of parameters used in remote query. */
numParams = list_length(fsplan->fdw_exprs);
fsstate->numParams = numParams;
***************
*** 995,1000 **** postgresIterateForeignScan(ForeignScanState *node)
--- 1070,1166 ----
TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
/*
+ * Get the result of an UPDATE/DELETE RETURNING, if pushing the command down.
+ */
+ if (fsstate->update_pushdown)
+ {
+ MemoryContext oldcontext;
+
+ /*
+ * If this is the first call after Begin, we need to execute the statement,
+ * and check for success.
+ */
+ if (fsstate->num_tuples == -1)
+ {
+ /*
+ * We don't use a PG_TRY block here, so be careful not to throw error
+ * without releasing the PGresult.
+ */
+ fsstate->result = PQexec(fsstate->conn, fsstate->query);
+ if (PQresultStatus(fsstate->result) !=
+ (fsstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
+ pgfdw_report_error(ERROR, fsstate->result, fsstate->conn, true,
+ fsstate->query);
+
+ /* Check number of rows affected. */
+ if (fsstate->has_returning)
+ fsstate->num_tuples = PQntuples(fsstate->result);
+ else
+ fsstate->num_tuples = atoi(PQcmdTuples(fsstate->result));
+ }
+
+ /*
+ * If the update query doesn't have a RETURNING clause, then there is
+ * nothing to do, so we just return an empty slot.
+ */
+ if (!fsstate->has_returning)
+ {
+ /*
+ * Increment the command es_processed count if necessary.
+ * (Note: ModifyTable cannot do that by itself in this case.)
+ */
+ if (fsstate->set_processed)
+ {
+ EState *estate = node->ss.ps.state;
+
+ estate->es_processed += fsstate->num_tuples;
+ }
+
+ /*
+ * Increment the tuple count for EXPLAIN ANALYZE if necessary.
+ * (Note: EXPLAIN ANALYZE cannot do that by itself in this case.)
+ */
+ if (node->ss.ps.instrument)
+ {
+ Instrumentation *instr = node->ss.ps.instrument;
+
+ instr->ntuples += fsstate->num_tuples;
+ }
+
+ return ExecClearTuple(slot);
+ }
+
+ /* If we didn't get any tuples, must be end of data. */
+ if (fsstate->next_tuple >= fsstate->num_tuples)
+ return ExecClearTuple(slot);
+
+ /* OK, we'll store RETURNING tuples in the batch_cxt. */
+ oldcontext = MemoryContextSwitchTo(fsstate->batch_cxt);
+
+ /* Fetch the next tuple. */
+ store_returning_result(slot,
+ fsstate->result,
+ fsstate->next_tuple,
+ fsstate->rel,
+ fsstate->attinmeta,
+ fsstate->retrieved_attrs,
+ fsstate->temp_cxt);
+ fsstate->rslot = slot;
+ fsstate->next_tuple++;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ /*
+ * Return slot. Note that this is safe because we can avoid applying
+ * ExecQual to the tuple due to no local quals (see the comment for
+ * update_is_pushdown_safe) and because the tuple can be safely
+ * projected by ExecProject (see push_update_down) and would then be
+ * ignored by postgresExecForeignUpdate or postgresExecForeignDelete.
+ */
+ return slot;
+ }
+
+ /*
* If this is the first call after Begin or ReScan, we need to create the
* cursor on the remote side.
*/
***************
*** 1036,1041 **** postgresReScanForeignScan(ForeignScanState *node)
--- 1202,1210 ----
char sql[64];
PGresult *res;
+ /* This shouldn't be called in update pushdown case. */
+ Assert(fsstate->update_pushdown == false);
+
/* If we haven't created the cursor yet, nothing to do. */
if (!fsstate->cursor_exists)
return;
***************
*** 1094,1099 **** postgresEndForeignScan(ForeignScanState *node)
--- 1263,1275 ----
if (fsstate == NULL)
return;
+ /* if pushing update down, nothing to do other than cleanup */
+ if (fsstate->update_pushdown)
+ {
+ if (fsstate->result)
+ PQclear(fsstate->result);
+ }
+
/* Close the cursor if open, to prevent accumulation of cursors */
if (fsstate->cursor_exists)
close_cursor(fsstate->conn, fsstate->cursor_number);
***************
*** 1145,1157 **** postgresAddForeignUpdateTargets(Query *parsetree,
/*
* postgresPlanForeignModify
* Plan an insert/update/delete operation on a foreign table
- *
- * Note: currently, the plan tree generated for UPDATE/DELETE will always
- * include a ForeignScan that retrieves ctids (using SELECT FOR UPDATE)
- * and then the ModifyTable node will have to execute individual remote
- * UPDATE/DELETE commands. If there are no local conditions or joins
- * needed, it'd be better to let the scan node do UPDATE/DELETE RETURNING
- * and then do nothing at ModifyTable. Room for future optimization ...
*/
static List *
postgresPlanForeignModify(PlannerInfo *root,
--- 1321,1326 ----
***************
*** 1167,1174 **** postgresPlanForeignModify(PlannerInfo *root,
List *returningList = NIL;
List *retrieved_attrs = NIL;
- initStringInfo(&sql);
-
/*
* Core code already has some lock on each rel being planned, so we can
* use NoLock here.
--- 1336,1341 ----
***************
*** 1210,1215 **** postgresPlanForeignModify(PlannerInfo *root,
--- 1377,1405 ----
}
/*
+ * For UPDATE/DELETE, try to push the command down into the remote.
+ */
+ if (operation == CMD_UPDATE || operation == CMD_DELETE)
+ {
+ /* Check to see whether it's safe to push the command down. */
+ if (update_is_pushdown_safe(root, plan,
+ resultRelation,
+ subplan_index,
+ rel, targetAttrs))
+ {
+ List *fdw_private;
+
+ /* OK, modify plan so as to push the command down. */
+ fdw_private = push_update_down(root, plan,
+ resultRelation,
+ subplan_index,
+ rel, targetAttrs);
+ heap_close(rel, NoLock);
+ return fdw_private;
+ }
+ }
+
+ /*
* Extract the relevant RETURNING list if any.
*/
if (plan->returningLists)
***************
*** 1218,1223 **** postgresPlanForeignModify(PlannerInfo *root,
--- 1408,1414 ----
/*
* Construct the SQL command string.
*/
+ initStringInfo(&sql);
switch (operation)
{
case CMD_INSERT:
***************
*** 1288,1293 **** postgresBeginForeignModify(ModifyTableState *mtstate,
--- 1479,1520 ----
fmstate = (PgFdwModifyState *) palloc0(sizeof(PgFdwModifyState));
fmstate->rel = rel;
+ /* Deconstruct fdw_private data. */
+ fmstate->query = strVal(list_nth(fdw_private,
+ FdwModifyPrivateUpdateSql));
+ fmstate->target_attrs = (List *) list_nth(fdw_private,
+ FdwModifyPrivateTargetAttnums);
+ fmstate->has_returning = intVal(list_nth(fdw_private,
+ FdwModifyPrivateHasReturning));
+ fmstate->retrieved_attrs = (List *) list_nth(fdw_private,
+ FdwModifyPrivateRetrievedAttrs);
+
+ /*
+ * if query is NULL, we are in update pushdown case.
+ */
+ if (fmstate->query == NULL)
+ {
+ PlanState *node = mtstate->mt_plans[subplan_index];
+ PgFdwScanState *fsstate;
+
+ Assert(fmstate->target_attrs == NIL);
+ Assert(fmstate->has_returning == false);
+ Assert(fmstate->retrieved_attrs == NIL);
+
+ Assert(nodeTag(node) == T_ForeignScanState);
+ fsstate = (PgFdwScanState *) ((ForeignScanState *) node)->fdw_state;
+ Assert(fsstate->update_pushdown);
+
+ fmstate->update_pushdown = true;
+ if (fsstate->has_returning)
+ {
+ fmstate->has_returning = true;
+ fmstate->fsstate = fsstate;
+ }
+ resultRelInfo->ri_FdwState = fmstate;
+ return;
+ }
+
/*
* Identify which user to do the remote access as. This should match what
* ExecCheckRTEPerms() does.
***************
*** 1304,1319 **** postgresBeginForeignModify(ModifyTableState *mtstate,
fmstate->conn = GetConnection(server, user, true);
fmstate->p_name = NULL; /* prepared statement not made yet */
- /* Deconstruct fdw_private data. */
- fmstate->query = strVal(list_nth(fdw_private,
- FdwModifyPrivateUpdateSql));
- fmstate->target_attrs = (List *) list_nth(fdw_private,
- FdwModifyPrivateTargetAttnums);
- fmstate->has_returning = intVal(list_nth(fdw_private,
- FdwModifyPrivateHasReturning));
- fmstate->retrieved_attrs = (List *) list_nth(fdw_private,
- FdwModifyPrivateRetrievedAttrs);
-
/* Create context for per-tuple temp workspace. */
fmstate->temp_cxt = AllocSetContextCreate(estate->es_query_cxt,
"postgres_fdw temporary data",
--- 1531,1536 ----
***************
*** 1411,1417 **** postgresExecForeignInsert(EState *estate,
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(fmstate, slot, res);
}
else
n_rows = atoi(PQcmdTuples(res));
--- 1628,1638 ----
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(slot, res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
}
else
n_rows = atoi(PQcmdTuples(res));
***************
*** 1442,1447 **** postgresExecForeignUpdate(EState *estate,
--- 1663,1676 ----
PGresult *res;
int n_rows;
+ /* Just return the slot of the ForeignScan node, if pushing update down */
+ if (fmstate->update_pushdown)
+ {
+ Assert(fmstate->has_returning);
+ Assert(fmstate->fsstate->rslot);
+ return fmstate->fsstate->rslot;
+ }
+
/* Set up the prepared statement on the remote server, if we didn't yet */
if (!fmstate->p_name)
prepare_foreign_modify(fmstate);
***************
*** 1481,1487 **** postgresExecForeignUpdate(EState *estate,
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(fmstate, slot, res);
}
else
n_rows = atoi(PQcmdTuples(res));
--- 1710,1720 ----
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(slot, res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
}
else
n_rows = atoi(PQcmdTuples(res));
***************
*** 1512,1517 **** postgresExecForeignDelete(EState *estate,
--- 1745,1758 ----
PGresult *res;
int n_rows;
+ /* Just return the slot of the ForeignScan node, if pushing update down */
+ if (fmstate->update_pushdown)
+ {
+ Assert(fmstate->has_returning);
+ Assert(fmstate->fsstate->rslot);
+ return fmstate->fsstate->rslot;
+ }
+
/* Set up the prepared statement on the remote server, if we didn't yet */
if (!fmstate->p_name)
prepare_foreign_modify(fmstate);
***************
*** 1551,1557 **** postgresExecForeignDelete(EState *estate,
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(fmstate, slot, res);
}
else
n_rows = atoi(PQcmdTuples(res));
--- 1792,1802 ----
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(slot, res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
}
else
n_rows = atoi(PQcmdTuples(res));
***************
*** 1579,1584 **** postgresEndForeignModify(EState *estate,
--- 1824,1833 ----
if (fmstate == NULL)
return;
+ /* If pushing update down, nothing to do */
+ if (fmstate->update_pushdown)
+ return;
+
/* If we created a prepared statement, destroy it */
if (fmstate->p_name)
{
***************
*** 1657,1667 **** postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
{
List *fdw_private;
char *sql;
if (es->verbose)
{
fdw_private = ((ForeignScan *) node->ss.ps.plan)->fdw_private;
! sql = strVal(list_nth(fdw_private, FdwScanPrivateSelectSql));
ExplainPropertyText("Remote SQL", sql, es);
}
}
--- 1906,1922 ----
{
List *fdw_private;
char *sql;
+ bool update_pushdown;
if (es->verbose)
{
fdw_private = ((ForeignScan *) node->ss.ps.plan)->fdw_private;
! update_pushdown = intVal(list_nth(fdw_private,
! FdwScanPrivateUpdatePushdown));
! if (update_pushdown)
! sql = strVal(list_nth(fdw_private, FdwScanPrivateUpdateSql));
! else
! sql = strVal(list_nth(fdw_private, FdwScanPrivateSelectSql));
ExplainPropertyText("Remote SQL", sql, es);
}
}
***************
*** 1682,1688 **** postgresExplainForeignModify(ModifyTableState *mtstate,
char *sql = strVal(list_nth(fdw_private,
FdwModifyPrivateUpdateSql));
! ExplainPropertyText("Remote SQL", sql, es);
}
}
--- 1937,1944 ----
char *sql = strVal(list_nth(fdw_private,
FdwModifyPrivateUpdateSql));
! if (sql != NULL)
! ExplainPropertyText("Remote SQL", sql, es);
}
}
***************
*** 1911,1916 **** ec_member_matches_foreign(PlannerInfo *root, RelOptInfo *rel,
--- 2167,2402 ----
}
/*
+ * Check to see whether it's safe to push an UPDATE/DELETE command down.
+ *
+ * Conditions checked here:
+ *
+ * 1. If the target relation has any row-level local BEFORE/AFTER triggers, we
+ * must not push the command down, since that breaks execution of the triggers.
+ *
+ * 2. If there are any local joins needed, we mustn't push the command down,
+ * because that breaks execution of the joins.
+ *
+ * 3. If there are any quals that can't be evaluated remotely, we mustn't push
+ * the command down, because that breaks evaluation of the quals.
+ *
+ * 4. In UPDATE, if it is unsafe to evaluate any expressions to assign to the
+ * target columns on the remote server, we must not push the command down.
+ */
+ static bool
+ update_is_pushdown_safe(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs)
+ {
+ CmdType operation = plan->operation;
+ RelOptInfo *baserel = root->simple_rel_array[resultRelation];
+ Plan *subplan = (Plan *) list_nth(plan->plans, subplan_index);
+ ListCell *lc;
+
+ /* Check point 1 */
+ if (rel->trigdesc &&
+ ((operation == CMD_UPDATE &&
+ (rel->trigdesc->trig_update_after_row ||
+ rel->trigdesc->trig_update_before_row)) ||
+ (operation == CMD_DELETE &&
+ (rel->trigdesc->trig_delete_after_row ||
+ rel->trigdesc->trig_delete_before_row))))
+ return false;
+
+ /* Check point 2 */
+ if (nodeTag(subplan) != T_ForeignScan)
+ return false;
+
+ /* Check point 3 */
+ if (subplan->qual != NIL)
+ return false;
+
+ /* Check point 4 */
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(subplan->targetlist,
+ attnum);
+
+ if (!is_foreign_expr(root, baserel, (Expr *) tle->expr))
+ return false;
+ }
+
+ return true;
+ }
+
+ /*
+ * Modify a plan so as to push an UPDATE/DELETE command down.
+ */
+ static List *
+ push_update_down(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs)
+ {
+ CmdType operation = plan->operation;
+ bool canSetTag = plan->canSetTag;
+ Plan *subplan = (Plan *) list_nth(plan->plans, subplan_index);
+ ForeignScan *fscan = (ForeignScan *) subplan;
+ StringInfoData sql;
+ List *remote_conds;
+ List *returningList = NIL;
+ List *retrieved_attrs = NIL;
+ List *fdw_private;
+ Value *v;
+
+ Assert(operation == CMD_UPDATE || operation == CMD_DELETE);
+
+ initStringInfo(&sql);
+
+ /*
+ * Extract the baserestrictinfo clauses that can be evaluated remotely.
+ */
+ remote_conds = (List *) list_nth(fscan->fdw_private,
+ FdwScanPrivateRemoteConds);
+
+ /*
+ * Extract the relevant RETURNING list if any.
+ */
+ if (plan->returningLists)
+ returningList = (List *) list_nth(plan->returningLists, subplan_index);
+
+ /*
+ * Construct the SQL command string.
+ */
+ if (operation == CMD_UPDATE)
+ {
+ List *targetlist = subplan->targetlist;
+
+ deparseDirectUpdateSql(&sql, root, resultRelation, rel,
+ remote_conds,
+ targetlist,
+ targetAttrs,
+ returningList,
+ &retrieved_attrs);
+
+ /*
+ * Rrewrite targetlist for safety of ExecProject.
+ */
+ subplan->targetlist = rewrite_targetlist(resultRelation,
+ rel,
+ targetlist,
+ targetAttrs);
+ }
+ else
+ {
+ Assert(operation == CMD_DELETE);
+
+ deparseDirectDeleteSql(&sql, root, resultRelation, rel,
+ remote_conds,
+ returningList,
+ &retrieved_attrs);
+ }
+
+ /*
+ * Update the fdw_private list that will be available to the executor.
+ * Items in the list must match enum FdwScanPrivateIndex, above.
+ */
+ fscan->fdw_private = lappend(fscan->fdw_private, makeString(sql.data));
+ fscan->fdw_private = lappend(fscan->fdw_private, makeInteger(canSetTag));
+ fscan->fdw_private = lappend(fscan->fdw_private,
+ makeInteger((retrieved_attrs != NIL)));
+ fscan->fdw_private = lappend(fscan->fdw_private, retrieved_attrs);
+
+ /*
+ * Also set the UPDATE/DELETE-pushdown flag in the fdw_private list.
+ */
+ v = (Value *) list_nth(fscan->fdw_private, FdwScanPrivateUpdatePushdown);
+ v->val.ival = true;
+
+ /*
+ * Build the fdw_private list that will be available to the executor.
+ * Items in the list must match enum FdwModifyPrivateIndex, above.
+ */
+ fdw_private = list_make4(makeString(NULL), NIL, makeInteger(false), NIL);
+
+ return fdw_private;
+ }
+
+ /*
+ * Rrewrite the targetlist of an UPDATE.
+ *
+ * This is called for safety of ExecProject if pushing the command down.
+ */
+ static List *
+ rewrite_targetlist(Index resultRelation, Relation rel,
+ List *targetlist, List *targetAttrs)
+ {
+ List *new_tlist = NIL;
+ int numattrs = RelationGetNumberOfAttributes(rel);
+ int attrno = 1;
+ ListCell *lc;
+
+ foreach(lc, targetlist)
+ {
+ TargetEntry *tle = (TargetEntry *) lfirst(lc);
+
+ if (tle->resjunk)
+ {
+ new_tlist = lappend(new_tlist, tle);
+ continue;
+ }
+
+ if (attrno > numattrs)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("table row type and query-specified row type do not match"),
+ errdetail("Query has too many columns.")));
+
+ if (!list_member_int(targetAttrs, attrno))
+ new_tlist = lappend(new_tlist, tle);
+ else
+ {
+ Form_pg_attribute attr = rel->rd_att->attrs[attrno - 1];
+ Oid atttype;
+ int32 atttypmod;
+ Oid attcollation;
+ Node *new_expr;
+ TargetEntry *new_tle;
+
+ Assert(!attr->attisdropped);
+ atttype = attr->atttypid;
+ atttypmod = attr->atttypmod;
+ attcollation = attr->attcollation;
+
+ new_expr = (Node *) makeVar(resultRelation,
+ attrno,
+ atttype,
+ atttypmod,
+ attcollation,
+ 0);
+
+ new_tle = makeTargetEntry((Expr *) new_expr,
+ attrno,
+ pstrdup(NameStr(attr->attname)),
+ false);
+
+ new_tlist = lappend(new_tlist, new_tle);
+ }
+
+ attrno++;
+ }
+
+ if (attrno != numattrs + 1)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("table row type and query-specified row type do not match"),
+ errdetail("Query has too few columns.")));
+
+ return new_tlist;
+ }
+
+ /*
* Create cursor for node's query with current parameter values.
*/
static void
***************
*** 2258,2275 **** convert_prep_stmt_params(PgFdwModifyState *fmstate,
* have PG_TRY blocks to ensure this happens.
*/
static void
! store_returning_result(PgFdwModifyState *fmstate,
! TupleTableSlot *slot, PGresult *res)
{
PG_TRY();
{
HeapTuple newtup;
! newtup = make_tuple_from_result_row(res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
/* tuple will be deleted when it is cleared from the slot */
ExecStoreTuple(newtup, slot, InvalidBuffer, true);
}
--- 2744,2766 ----
* have PG_TRY blocks to ensure this happens.
*/
static void
! store_returning_result(TupleTableSlot *slot,
! PGresult *res,
! int row,
! Relation rel,
! AttInMetadata *attinmeta,
! List *retrieved_attrs,
! MemoryContext temp_context)
{
PG_TRY();
{
HeapTuple newtup;
! newtup = make_tuple_from_result_row(res, row,
! rel,
! attinmeta,
! retrieved_attrs,
! temp_context);
/* tuple will be deleted when it is cleared from the slot */
ExecStoreTuple(newtup, slot, InvalidBuffer, true);
}
*** a/contrib/postgres_fdw/postgres_fdw.h
--- b/contrib/postgres_fdw/postgres_fdw.h
***************
*** 66,75 **** extern void deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 66,87 ----
Index rtindex, Relation rel,
List *targetAttrs, List *returningList,
List **retrieved_attrs);
+ extern void deparseDirectUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List *targetlist,
+ List *targetAttrs,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseDeleteSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *returningList,
List **retrieved_attrs);
+ extern void deparseDirectDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseAnalyzeSizeSql(StringInfo buf, Relation rel);
extern void deparseAnalyzeSql(StringInfo buf, Relation rel,
List **retrieved_attrs);
*** a/contrib/postgres_fdw/sql/postgres_fdw.sql
--- b/contrib/postgres_fdw/sql/postgres_fdw.sql
***************
*** 320,339 **** INSERT INTO ft2 (c1,c2,c3) SELECT c1+1000,c2+100, c3 || c3 FROM ft2 LIMIT 20;
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
--- 320,346 ----
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c7 = DEFAULT WHERE c1 % 10 = 0 AND date(c4) = '1970-01-01'::date; -- can't be pushed down
+ UPDATE ft2 SET c7 = DEFAULT WHERE c1 % 10 = 0 AND date(c4) = '1970-01-01'::date;
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
***************
*** 616,621 **** UPDATE rem1 SET f2 = 'testo';
--- 623,712 ----
-- Test returning a system attribute
INSERT INTO rem1(f2) VALUES ('test') RETURNING ctid;
+ -- cleanup
+ DROP TRIGGER trig_row_before ON rem1;
+ DROP TRIGGER trig_row_after ON rem1;
+ DROP TRIGGER trig_local_before ON loc1;
+
+
+ -- Test update-pushdown functionality
+
+ -- Test with statement-level triggers
+ CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_stmt_before ON rem1;
+
+ CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_stmt_after ON rem1;
+
+ -- Test with row-level ON INSERT triggers
+ CREATE TRIGGER trig_row_before_insert
+ BEFORE INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_before_insert ON rem1;
+
+ CREATE TRIGGER trig_row_after_insert
+ AFTER INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_after_insert ON rem1;
+
+ -- Test with row-level ON UPDATE triggers
+ CREATE TRIGGER trig_row_before_update
+ BEFORE UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_before_update ON rem1;
+
+ CREATE TRIGGER trig_row_after_update
+ AFTER UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_after_update ON rem1;
+
+ -- Test with row-level ON DELETE triggers
+ CREATE TRIGGER trig_row_before_delete
+ BEFORE DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ DROP TRIGGER trig_row_before_delete ON rem1;
+
+ CREATE TRIGGER trig_row_after_delete
+ AFTER DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ DROP TRIGGER trig_row_after_delete ON rem1;
+
-- ===================================================================
-- test IMPORT FOREIGN SCHEMA
-- ===================================================================
*** a/doc/src/sgml/postgres-fdw.sgml
--- b/doc/src/sgml/postgres-fdw.sgml
***************
*** 414,419 ****
--- 414,428 ----
<literal>WHERE</> clauses are not sent to the remote server unless they use
only built-in data types, operators, and functions. Operators and
functions in the clauses must be <literal>IMMUTABLE</> as well.
+ For an <command>UPDATE</> or <command>DELETE</> query,
+ <filename>postgres_fdw</> attempts to optimize the query execution by
+ sending the whole query to the remote server if there are no query
+ <literal>WHERE</> clauses that cannot be sent to the remote server,
+ no local joins for the query, or no row-level local <literal>BEFORE</> or
+ <literal>AFTER</> triggers on the target table. In <command>UPDATE</>,
+ expressions to assign to target columns must use only built-in data types,
+ <literal>IMMUTABLE</> operators, and <literal>IMMUTABLE</> functions,
+ to reduce the risk of misexecution of the query.
</para>
<para>
Etsuro Fujita wrote:
I agree with you on that point. So, I've updated the patch to have the
explicit flag, as you proposed. Attached is the updated version of the
patch. In this version, I've also revised code and its comments a bit.
Thank you, I have set the patch to "Ready for Committer".
Yours,
Laurenz Albe
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
* Albe Laurenz (laurenz.albe@wien.gv.at) wrote:
Etsuro Fujita wrote:
I agree with you on that point. So, I've updated the patch to have the
explicit flag, as you proposed. Attached is the updated version of the
patch. In this version, I've also revised code and its comments a bit.Thank you, I have set the patch to "Ready for Committer".
I had a few minutes, so I started looking at this patch and I definitely
like where it's going. One concern which was brought up that I didn't
see completely addressed was around UPDATE/DELETE with LIMIT, which
seems to be making progress towards getting in. Presumably, we'd simply
disallow this optimization in that case, but we'll need a way to
identify that case..
I have to admit that, while I applaud the effort made to have this
change only be to postgres_fdw, I'm not sure that having the
update/delete happening during the Scan phase and then essentially
no-op'ing the ExecForeignUpdate/ExecForeignDelete is entirely in-line
with the defined API.
I would have thought we'd add a capability check function that says "can
this Modify be completely pushed down" and then have a way for that to
happen in ExecForeignUpdate/ExecForeignDelete. That means changes to
the existing API, of course, and if people feel that's unnecessary then
I'd suggest that we need to at least document that we're willing to
support these bulk operations happening on the remote siude during the
pre-Modify Scan and not during the ExecForeignUpdate/ExecForeignDelete.
Thanks,
Stephen
Stephen Frost <sfrost@snowman.net> writes:
I have to admit that, while I applaud the effort made to have this
change only be to postgres_fdw, I'm not sure that having the
update/delete happening during the Scan phase and then essentially
no-op'ing the ExecForeignUpdate/ExecForeignDelete is entirely in-line
with the defined API.
Yeah, I've started looking at this patch and that seemed like not
necessarily such a wise choice. I think it'd be better if the generated
plan tree had a different structure in this case. The existing approach
is an impressive hack but it's hard to call it anything but a hack.
I'm not sure offhand what the new plan tree ought to look like. We could
just generate a ForeignScan node, but that seems like rather a misnomer.
Is it worth inventing a new ForeignUpdate node type? Or maybe it'd still
be ForeignScan but with a flag field saying "hey this is really an update
(or a delete)". The main benefit I can think of right now is that the
EXPLAIN output would be less strange-looking --- but EXPLAIN is hardly
the only thing that ever looks at plan trees, so having an outright
misleading plan structure is likely to burn us down the line.
One advantage of getting the core code involved in the decision about
whether an update/delete can be pushed down is that FDW-independent checks
like whether there are relevant triggers could be implemented in the core
code, rather than having to duplicate them (and later maintain them) in
every FDW that wants to do this. OTOH, maybe the trigger issue is really
the only thing that could be shared, not sure. Stuff like "is there a
LIMIT" probably has to be in the FDW since some FDWs could support pushing
down LIMIT and others not.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Tom, all,
* Tom Lane (tgl@sss.pgh.pa.us) wrote:
Yeah, I've started looking at this patch and that seemed like not
necessarily such a wise choice. I think it'd be better if the generated
plan tree had a different structure in this case. The existing approach
is an impressive hack but it's hard to call it anything but a hack.
Agreed.
I'm not sure offhand what the new plan tree ought to look like. We could
just generate a ForeignScan node, but that seems like rather a misnomer.
Is it worth inventing a new ForeignUpdate node type? Or maybe it'd still
be ForeignScan but with a flag field saying "hey this is really an update
(or a delete)". The main benefit I can think of right now is that the
EXPLAIN output would be less strange-looking --- but EXPLAIN is hardly
the only thing that ever looks at plan trees, so having an outright
misleading plan structure is likely to burn us down the line.
My initial reaction is that a ForeignUpdate node makes the most sense.
That said, I've not invested any time trying to flesh that out, so take
my gut feeling for what it's worth.
One advantage of getting the core code involved in the decision about
whether an update/delete can be pushed down is that FDW-independent checks
like whether there are relevant triggers could be implemented in the core
code, rather than having to duplicate them (and later maintain them) in
every FDW that wants to do this. OTOH, maybe the trigger issue is really
the only thing that could be shared, not sure. Stuff like "is there a
LIMIT" probably has to be in the FDW since some FDWs could support pushing
down LIMIT and others not.
Yeah, agreed. I feel like we'll definitely want to support pushing down
LIMIT and it'll depend on the FDW as to if it can handle that or not
(and how it handles it, if it does). My thinking was actually that we'd
ask the FDW if it supports pushing down the UPDATE or DELETE statement,
as I suspect not all FDWs will want to answer that question the same way
for every case.
Thanks,
Stephen
Tom Lane wrote:
Stephen Frost <sfrost@snowman.net> writes:
I have to admit that, while I applaud the effort made to have this
change only be to postgres_fdw, I'm not sure that having the
update/delete happening during the Scan phase and then essentially
no-op'ing the ExecForeignUpdate/ExecForeignDelete is entirely in-line
with the defined API.Yeah, I've started looking at this patch and that seemed like not
necessarily such a wise choice. I think it'd be better if the generated
plan tree had a different structure in this case. The existing approach
is an impressive hack but it's hard to call it anything but a hack.
I guess that the idea is inspired by this comment on postgres_fdw.c:
* Note: currently, the plan tree generated for UPDATE/DELETE will always
* include a ForeignScan that retrieves ctids (using SELECT FOR UPDATE)
* and then the ModifyTable node will have to execute individual remote
* UPDATE/DELETE commands. If there are no local conditions or joins
* needed, it'd be better to let the scan node do UPDATE/DELETE RETURNING
* and then do nothing at ModifyTable. Room for future optimization ...
I'm not sure offhand what the new plan tree ought to look like. We could
just generate a ForeignScan node, but that seems like rather a misnomer.
Is it worth inventing a new ForeignUpdate node type? Or maybe it'd still
be ForeignScan but with a flag field saying "hey this is really an update
(or a delete)". The main benefit I can think of right now is that the
EXPLAIN output would be less strange-looking --- but EXPLAIN is hardly
the only thing that ever looks at plan trees, so having an outright
misleading plan structure is likely to burn us down the line.
I can understand these qualms.
I wonder if "ForeignUpdate" is such a good name though, since it would
surprise the uninitiate that in the regular (no push-down) case the
actual modification is *not* performed by ForeignUpdate.
So it should rather be a "ForeignModifyingScan", but I personally would
prefer a "has_side_effects" flag on ForeignScan.
One advantage of getting the core code involved in the decision about
whether an update/delete can be pushed down is that FDW-independent checks
like whether there are relevant triggers could be implemented in the core
code, rather than having to duplicate them (and later maintain them) in
every FDW that wants to do this. OTOH, maybe the trigger issue is really
the only thing that could be shared, not sure. Stuff like "is there a
LIMIT" probably has to be in the FDW since some FDWs could support pushing
down LIMIT and others not.
You are right, the gain would probably be limited.
Yours,
Laurenz Albe
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
(2014/09/12 16:03), Albe Laurenz wrote:
Tom Lane wrote:
Stephen Frost <sfrost@snowman.net> writes:
I have to admit that, while I applaud the effort made to have this
change only be to postgres_fdw, I'm not sure that having the
update/delete happening during the Scan phase and then essentially
no-op'ing the ExecForeignUpdate/ExecForeignDelete is entirely in-line
with the defined API.Yeah, I've started looking at this patch and that seemed like not
necessarily such a wise choice. I think it'd be better if the generated
plan tree had a different structure in this case. The existing approach
is an impressive hack but it's hard to call it anything but a hack.
Thank you for the review, Tom and Stephen!
I guess that the idea is inspired by this comment on postgres_fdw.c:
* Note: currently, the plan tree generated for UPDATE/DELETE will always
* include a ForeignScan that retrieves ctids (using SELECT FOR UPDATE)
* and then the ModifyTable node will have to execute individual remote
* UPDATE/DELETE commands. If there are no local conditions or joins
* needed, it'd be better to let the scan node do UPDATE/DELETE RETURNING
* and then do nothing at ModifyTable. Room for future optimization ...
That's right. Thanks, Laurenz!
And in addition to that, I've created the patch with the conscious aim
of not getting the core code involved, because I was thinking this is
just an optimization. But I have to admit I was conscious of that too much.
I'm not sure offhand what the new plan tree ought to look like. We could
just generate a ForeignScan node, but that seems like rather a misnomer.
Is it worth inventing a new ForeignUpdate node type? Or maybe it'd still
be ForeignScan but with a flag field saying "hey this is really an update
(or a delete)". The main benefit I can think of right now is that the
EXPLAIN output would be less strange-looking --- but EXPLAIN is hardly
the only thing that ever looks at plan trees, so having an outright
misleading plan structure is likely to burn us down the line.I can understand these qualms.
I wonder if "ForeignUpdate" is such a good name though, since it would
surprise the uninitiate that in the regular (no push-down) case the
actual modification is *not* performed by ForeignUpdate.
So it should rather be a "ForeignModifyingScan", but I personally would
prefer a "has_side_effects" flag on ForeignScan.
+1 for improving the EXPLAIN output by inventing a plan tree with a
different structure in this case, in general.
One advantage of getting the core code involved in the decision about
whether an update/delete can be pushed down is that FDW-independent checks
like whether there are relevant triggers could be implemented in the core
code, rather than having to duplicate them (and later maintain them) in
every FDW that wants to do this.
Good idea!
Stuff like "is there a
LIMIT" probably has to be in the FDW since some FDWs could support pushing
down LIMIT and others not.
That's what I have in mind. I'll work on that at the next CF.
Thanks,
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Albe Laurenz <laurenz.albe@wien.gv.at> writes:
Tom Lane wrote:
I'm not sure offhand what the new plan tree ought to look like. We could
just generate a ForeignScan node, but that seems like rather a misnomer.
Is it worth inventing a new ForeignUpdate node type? Or maybe it'd still
be ForeignScan but with a flag field saying "hey this is really an update
(or a delete)". The main benefit I can think of right now is that the
EXPLAIN output would be less strange-looking --- but EXPLAIN is hardly
the only thing that ever looks at plan trees, so having an outright
misleading plan structure is likely to burn us down the line.
I can understand these qualms.
I wonder if "ForeignUpdate" is such a good name though, since it would
surprise the uninitiate that in the regular (no push-down) case the
actual modification is *not* performed by ForeignUpdate.
So it should rather be a "ForeignModifyingScan", but I personally would
prefer a "has_side_effects" flag on ForeignScan.
I was envisioning that the EXPLAIN output would look like
Foreign Scan on tab1
Remote SQL: SELECT ...
for the normal case, versus
Foreign Update on tab1
Remote SQL: UPDATE ...
for the pushed-down-update case (and similarly for DELETE). For a
non-optimized update it'd still be a ForeignScan underneath a ModifyTable.
As for the internal representation, I was thinking of adding a CmdType
field to struct ForeignScan, with currently only CMD_SELECT, CMD_UPDATE,
CMD_DELETE as allowed values, though possibly in future we'd think of a
reason to allow CMD_INSERT there. This is more or less isomorphic to your
"has_side_effects" flag, but it allows distinguishing UPDATE from DELETE
which might be useful.
The only thing that's bothering me about this concept is that I'm not
seeing how to scale it up to handling a pushed-down update on a join,
ie, "UPDATE foo ... FROM bar ..." where both foo and bar are remote.
Maybe it's silly to worry about that until join push-down is done;
but in that case I'd vote for postponing this whole patch until we
have join push-down.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
(2014/09/13 0:13), Tom Lane wrote:
Albe Laurenz <laurenz.albe@wien.gv.at> writes:
Tom Lane wrote:
I'm not sure offhand what the new plan tree ought to look like. We could
just generate a ForeignScan node, but that seems like rather a misnomer.
Is it worth inventing a new ForeignUpdate node type? Or maybe it'd still
be ForeignScan but with a flag field saying "hey this is really an update
(or a delete)". The main benefit I can think of right now is that the
EXPLAIN output would be less strange-looking --- but EXPLAIN is hardly
the only thing that ever looks at plan trees, so having an outright
misleading plan structure is likely to burn us down the line.
I was envisioning that the EXPLAIN output would look like
Foreign Scan on tab1
Remote SQL: SELECT ...for the normal case, versus
Foreign Update on tab1
Remote SQL: UPDATE ...for the pushed-down-update case (and similarly for DELETE). For a
non-optimized update it'd still be a ForeignScan underneath a ModifyTable.
As for the internal representation, I was thinking of adding a CmdType
field to struct ForeignScan, with currently only CMD_SELECT, CMD_UPDATE,
CMD_DELETE as allowed values, though possibly in future we'd think of a
reason to allow CMD_INSERT there.
+1
The only thing that's bothering me about this concept is that I'm not
seeing how to scale it up to handling a pushed-down update on a join,
ie, "UPDATE foo ... FROM bar ..." where both foo and bar are remote.
Maybe it's silly to worry about that until join push-down is done;
but in that case I'd vote for postponing this whole patch until we
have join push-down.
OK
Thanks,
PS: I'll help Hanada-san do the work if there is anything I can do.
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2014/09/13 0:13, Tom Lane wrote:
Albe Laurenz <laurenz.albe@wien.gv.at> writes:
Tom Lane wrote:
I'm not sure offhand what the new plan tree ought to look like. We could
just generate a ForeignScan node, but that seems like rather a misnomer.
Is it worth inventing a new ForeignUpdate node type? Or maybe it'd still
be ForeignScan but with a flag field saying "hey this is really an update
(or a delete)". The main benefit I can think of right now is that the
EXPLAIN output would be less strange-looking --- but EXPLAIN is hardly
the only thing that ever looks at plan trees, so having an outright
misleading plan structure is likely to burn us down the line.I can understand these qualms.
I wonder if "ForeignUpdate" is such a good name though, since it would
surprise the uninitiate that in the regular (no push-down) case the
actual modification is *not* performed by ForeignUpdate.
So it should rather be a "ForeignModifyingScan", but I personally would
prefer a "has_side_effects" flag on ForeignScan.I was envisioning that the EXPLAIN output would look like
Foreign Scan on tab1
Remote SQL: SELECT ...for the normal case, versus
Foreign Update on tab1
Remote SQL: UPDATE ...for the pushed-down-update case (and similarly for DELETE). For a
non-optimized update it'd still be a ForeignScan underneath a ModifyTable.As for the internal representation, I was thinking of adding a CmdType
field to struct ForeignScan, with currently only CMD_SELECT, CMD_UPDATE,
CMD_DELETE as allowed values, though possibly in future we'd think of a
reason to allow CMD_INSERT there. This is more or less isomorphic to your
"has_side_effects" flag, but it allows distinguishing UPDATE from DELETE
which might be useful.The only thing that's bothering me about this concept is that I'm not
seeing how to scale it up to handling a pushed-down update on a join,
ie, "UPDATE foo ... FROM bar ..." where both foo and bar are remote.
Maybe it's silly to worry about that until join push-down is done;
but in that case I'd vote for postponing this whole patch until we
have join push-down.
I'll re-add this to the final CF. And I'll update the patch.
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2015/02/16 12:03, Etsuro Fujita wrote:
I'll update the patch.
While updating the patch, I noticed that in the previous patch, there is
a bug in pushing down parameterized UPDATE/DELETE queries; generic plans
for such queries fail with a can't-happen error. I fixed the bug and
tried to add the regression tests that execute the generic plans, but I
couldn't because I can't figure out how to force generic plans. Is
there any way to do that?
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Etsuro Fujita wrote:
While updating the patch, I noticed that in the previous patch, there is
a bug in pushing down parameterized UPDATE/DELETE queries; generic plans
for such queries fail with a can't-happen error. I fixed the bug and
tried to add the regression tests that execute the generic plans, but I
couldn't because I can't figure out how to force generic plans. Is
there any way to do that?
I don't know about a way to force it, but if you run the statement six
times, it will probably switch to a generic plan.
Yours,
Laurenz Albe
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2015/03/04 16:58, Albe Laurenz wrote:
Etsuro Fujita wrote:
While updating the patch, I noticed that in the previous patch, there is
a bug in pushing down parameterized UPDATE/DELETE queries; generic plans
for such queries fail with a can't-happen error. I fixed the bug and
tried to add the regression tests that execute the generic plans, but I
couldn't because I can't figure out how to force generic plans. Is
there any way to do that?I don't know about a way to force it, but if you run the statement six
times, it will probably switch to a generic plan.
Yeah, I was just thinking running the statement six times in the
regression tests ...
Thanks!
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2015/03/04 17:07, Etsuro Fujita wrote:
On 2015/03/04 16:58, Albe Laurenz wrote:
Etsuro Fujita wrote:
While updating the patch, I noticed that in the previous patch, there is
a bug in pushing down parameterized UPDATE/DELETE queries; generic plans
for such queries fail with a can't-happen error. I fixed the bug and
tried to add the regression tests that execute the generic plans, but I
couldn't because I can't figure out how to force generic plans. Is
there any way to do that?I don't know about a way to force it, but if you run the statement six
times, it will probably switch to a generic plan.Yeah, I was just thinking running the statement six times in the
regression tests ...
Here is an updated version. In this version, the bug has been fixed,
but any regression tests for that hasn't been added, because I'm not
sure that the above way is a good idea and don't have any other ideas.
The EXPLAIN output has also been improved as discussed in [1]/messages/by-id/31942.1410534785@sss.pgh.pa.us.
On top of this, I'll try to extend the join push-down patch to support a
pushed-down update on a join, though I'm still digesting the series of
patches.
Comments welcome.
Best regards,
Etsuro Fujita
Attachments:
postgres_fdw-update-v5.patchtext/x-patch; name=postgres_fdw-update-v5.patchDownload
*** a/contrib/postgres_fdw/deparse.c
--- b/contrib/postgres_fdw/deparse.c
***************
*** 189,198 **** is_foreign_expr(PlannerInfo *root,
if (!foreign_expr_walker((Node *) expr, &glob_cxt, &loc_cxt))
return false;
- /* Expressions examined here should be boolean, ie noncollatable */
- Assert(loc_cxt.collation == InvalidOid);
- Assert(loc_cxt.state == FDW_COLLATE_NONE);
-
/*
* An expression which includes any mutable functions can't be sent over
* because its result is not stable. For example, sending now() remote
--- 189,194 ----
***************
*** 788,794 **** deparseTargetList(StringInfo buf,
*
* If params is not NULL, it receives a list of Params and other-relation Vars
* used in the clauses; these values must be transmitted to the remote server
! * as parameter values.
*
* If params is NULL, we're generating the query for EXPLAIN purposes,
* so Params and other-relation Vars should be replaced by dummy values.
--- 784,791 ----
*
* If params is not NULL, it receives a list of Params and other-relation Vars
* used in the clauses; these values must be transmitted to the remote server
! * as parameter values. Caller is responsible for initializing the result list
! * to empty if necessary.
*
* If params is NULL, we're generating the query for EXPLAIN purposes,
* so Params and other-relation Vars should be replaced by dummy values.
***************
*** 805,813 **** appendWhereClause(StringInfo buf,
int nestlevel;
ListCell *lc;
- if (params)
- *params = NIL; /* initialize result list to empty */
-
/* Set up context struct for recursion */
context.root = root;
context.foreignrel = baserel;
--- 802,807 ----
***************
*** 940,945 **** deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 934,996 ----
}
/*
+ * deparse remote UPDATE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ deparse_expr_cxt context;
+ bool first;
+ ListCell *lc;
+
+ if (params_list)
+ *params_list = NIL; /* initialize result list to empty */
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = params_list;
+
+ appendStringInfoString(buf, "UPDATE ");
+ deparseRelation(buf, rel);
+ appendStringInfoString(buf, " SET ");
+
+ first = true;
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(targetlist, attnum);
+
+ if (!first)
+ appendStringInfoString(buf, ", ");
+ first = false;
+
+ deparseColumnRef(buf, rtindex, attnum, root);
+ appendStringInfo(buf, " = ");
+ deparseExpr((Expr *) tle->expr, &context);
+ }
+ if (remote_conds)
+ appendWhereClause(buf, root, baserel, remote_conds,
+ true, params_list);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* deparse remote DELETE statement
*
* The statement text is appended to buf, and we also create an integer List
***************
*** 962,967 **** deparseDeleteSql(StringInfo buf, PlannerInfo *root,
--- 1013,1048 ----
}
/*
+ * deparse remote DELETE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+
+ if (params_list)
+ *params_list = NIL; /* initialize result list to empty */
+
+ appendStringInfoString(buf, "DELETE FROM ");
+ deparseRelation(buf, rel);
+ if (remote_conds)
+ appendWhereClause(buf, root, baserel, remote_conds,
+ true, params_list);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* Add a RETURNING clause, if needed, to an INSERT/UPDATE/DELETE.
*/
static void
*** a/contrib/postgres_fdw/expected/postgres_fdw.out
--- b/contrib/postgres_fdw/expected/postgres_fdw.out
***************
*** 1099,1105 **** INSERT INTO ft2 (c1,c2,c3)
--- 1099,1124 ----
(3 rows)
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 300), c3 = (c3 || '_update3'::text) WHERE ((("C 1" % 10) = 3))
+ (3 rows)
+
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Output: c1, c2, c3, c4, c5, c6, c7, c8
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 400), c3 = (c3 || '_update7'::text) WHERE ((("C 1" % 10) = 7)) RETURNING "C 1", c2, c3, c4, c5, c6, c7, c8
+ (4 rows)
+
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
c1 | c2 | c3 | c4 | c5 | c6 | c7 | c8
------+-----+--------------------+------------------------------+--------------------------+----+------------+-----
***************
*** 1209,1215 **** UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
--- 1228,1234 ----
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
***************
*** 1230,1245 **** UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
! QUERY PLAN
! ----------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1 RETURNING "C 1", c4
! -> Foreign Scan on public.ft2
! Output: ctid
! Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) FOR UPDATE
! (6 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
--- 1249,1262 ----
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
! QUERY PLAN
! --------------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
! -> Foreign Delete on public.ft2
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) RETURNING "C 1", c4
! (4 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
***************
*** 1350,1356 **** DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
--- 1367,1373 ----
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
***************
*** 2193,2198 **** SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
--- 2210,2228 ----
1104 | 204 | ddd |
(819 rows)
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c7 = DEFAULT WHERE c1 % 10 = 0 AND date(c4) = '1970-01-01'::date; -- can't be pushed down
+ QUERY PLAN
+ -----------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c7 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.ft2
+ Output: c1, c2, NULL::integer, c3, c4, c5, c6, 'ft2 '::character(10), c8, ctid
+ Filter: (date(ft2.c4) = '01-01-1970'::date)
+ Remote SQL: SELECT "C 1", c2, c3, c4, c5, c6, c8, ctid FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 0)) FOR UPDATE
+ (6 rows)
+
+ UPDATE ft2 SET c7 = DEFAULT WHERE c1 % 10 = 0 AND date(c4) = '1970-01-01'::date;
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
BEGIN
***************
*** 2334,2340 **** CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 2364,2370 ----
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 2493,2499 **** savepoint s3;
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 2523,2529 ----
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (-2) WHERE ((c2 = 42)) AND (("C 1" = 10))
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 2633,2639 **** CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2positive;
-- But inconsistent check constraints provide inconsistent results
ALTER FOREIGN TABLE ft1 ADD CONSTRAINT ft1_c2negative CHECK (c2 < 0);
--- 2663,2669 ----
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2positive;
-- But inconsistent check constraints provide inconsistent results
ALTER FOREIGN TABLE ft1 ADD CONSTRAINT ft1_c2negative CHECK (c2 < 0);
***************
*** 3026,3031 **** NOTICE: NEW: (13,"test triggered !")
--- 3056,3254 ----
(0,27)
(1 row)
+ -- cleanup
+ DROP TRIGGER trig_row_before ON rem1;
+ DROP TRIGGER trig_row_after ON rem1;
+ DROP TRIGGER trig_local_before ON loc1;
+ -- Test update-pushdown functionality
+ -- Test with statement-level triggers
+ CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_stmt_before ON rem1;
+ CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_stmt_after ON rem1;
+ -- Test with row-level ON INSERT triggers
+ CREATE TRIGGER trig_row_before_insert
+ BEFORE INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_before_insert ON rem1;
+ CREATE TRIGGER trig_row_after_insert
+ AFTER INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_after_insert ON rem1;
+ -- Test with row-level ON UPDATE triggers
+ CREATE TRIGGER trig_row_before_update
+ BEFORE UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_before_update ON rem1;
+ CREATE TRIGGER trig_row_after_update
+ AFTER UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+ -------------------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_after_update ON rem1;
+ -- Test with row-level ON DELETE triggers
+ CREATE TRIGGER trig_row_before_delete
+ BEFORE DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ DROP TRIGGER trig_row_before_delete ON rem1;
+ CREATE TRIGGER trig_row_after_delete
+ AFTER DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+ ------------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ DROP TRIGGER trig_row_after_delete ON rem1;
-- ===================================================================
-- test IMPORT FOREIGN SCHEMA
-- ===================================================================
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
***************
*** 87,93 **** typedef struct PgFdwRelationInfo
* planner to executor. Currently we store:
*
* 1) SELECT statement text to be sent to the remote server
! * 2) Integer list of attribute numbers retrieved by the SELECT
*
* These items are indexed with the enum FdwScanPrivateIndex, so an item
* can be fetched with list_nth(). For example, to get the SELECT statement:
--- 87,99 ----
* planner to executor. Currently we store:
*
* 1) SELECT statement text to be sent to the remote server
! * 2) List of restriction clauses that can be executed remotely
! * 3) Integer list of attribute numbers retrieved by the SELECT
! * 4) Boolean flag showing if an UPDATE/DELETE is pushed down
! * 5) UPDATE/DELETE statement text to be sent to the remote server
! * 6) Boolean flag showing if we set the command es_processed
! * 7) Boolean flag showing if the remote query has a RETURNING clause
! * 8) Integer list of attribute numbers retrieved by RETURNING, if any
*
* These items are indexed with the enum FdwScanPrivateIndex, so an item
* can be fetched with list_nth(). For example, to get the SELECT statement:
***************
*** 97,104 **** enum FdwScanPrivateIndex
{
/* SQL statement to execute remotely (as a String node) */
FdwScanPrivateSelectSql,
! /* Integer list of attribute numbers retrieved by the SELECT */
! FdwScanPrivateRetrievedAttrs
};
/*
--- 103,122 ----
{
/* SQL statement to execute remotely (as a String node) */
FdwScanPrivateSelectSql,
! /* List of restriction clauses that can be executed remotely */
! FdwScanPrivateRemoteConds,
! /* Integer list of attribute numbers retrieved by SELECT */
! FdwScanPrivateRetrievedAttrsBySelect,
! /* UPDATE/DELETE-pushdown flag (as an integer Value node) */
! FdwScanPrivateUpdatePushedDown,
! /* UPDATE/DELETE statement to execute remotely (as a String node) */
! FdwScanPrivateUpdateSql,
! /* set-processed flag (as an integer Value node) */
! FdwScanPrivateSetProcessed,
! /* has-returning flag (as an integer Value node) */
! FdwScanPrivateHasReturning,
! /* Integer list of attribute numbers retrieved by RETURNING */
! FdwScanPrivateRetrievedAttrsByReturning
};
/*
***************
*** 132,139 **** typedef struct PgFdwScanState
AttInMetadata *attinmeta; /* attribute datatype conversion metadata */
/* extracted fdw_private data */
! char *query; /* text of SELECT command */
List *retrieved_attrs; /* list of retrieved attribute numbers */
/* for remote query execution */
PGconn *conn; /* connection for the scan */
--- 150,159 ----
AttInMetadata *attinmeta; /* attribute datatype conversion metadata */
/* extracted fdw_private data */
! char *query; /* text of SELECT or UPDATE/DELETE command */
List *retrieved_attrs; /* list of retrieved attribute numbers */
+ bool set_processed; /* do we set the command es_processed? */
+ bool has_returning; /* is there a RETURNING clause? */
/* for remote query execution */
PGconn *conn; /* connection for the scan */
***************
*** 153,158 **** typedef struct PgFdwScanState
--- 173,183 ----
int fetch_ct_2; /* Min(# of fetches done, 2) */
bool eof_reached; /* true if last fetch reached EOF */
+ /* for update pushdown */
+ bool update_pushed_down; /* is an UPDATE/DELETE pushed down? */
+ PGresult *result; /* result of an UPDATE/DELETE query */
+ TupleTableSlot *rslot; /* slot containing the result tuple */
+
/* working memory contexts */
MemoryContext batch_cxt; /* context holding current batch of tuples */
MemoryContext temp_cxt; /* context for per-tuple temporary data */
***************
*** 181,186 **** typedef struct PgFdwModifyState
--- 206,215 ----
int p_nums; /* number of parameters to transmit */
FmgrInfo *p_flinfo; /* output conversion functions for them */
+ /* for update pushdown */
+ bool update_pushed_down; /* is an UPDATE/DELETE pushed down? */
+ PgFdwScanState *fsstate; /* execution state of a foreign scan */
+
/* working memory context */
MemoryContext temp_cxt; /* context for per-tuple temporary data */
} PgFdwModifyState;
***************
*** 309,320 **** static bool ec_member_matches_foreign(PlannerInfo *root, RelOptInfo *rel,
static void create_cursor(ForeignScanState *node);
static void fetch_more_data(ForeignScanState *node);
static void close_cursor(PGconn *conn, unsigned int cursor_number);
static void prepare_foreign_modify(PgFdwModifyState *fmstate);
static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
ItemPointer tupleid,
TupleTableSlot *slot);
! static void store_returning_result(PgFdwModifyState *fmstate,
! TupleTableSlot *slot, PGresult *res);
static int postgresAcquireSampleRowsFunc(Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
--- 338,370 ----
static void create_cursor(ForeignScanState *node);
static void fetch_more_data(ForeignScanState *node);
static void close_cursor(PGconn *conn, unsigned int cursor_number);
+ static void execute_pushed_down_command(ForeignScanState *node);
+ static TupleTableSlot *get_next_returning_result(ForeignScanState *node);
+ static bool update_is_pushdown_safe(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs);
+ static List *push_update_down(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs);
+ static List *rewrite_targetlist(Index resultRelation, Relation rel,
+ List *targetlist, List *targetAttrs);
static void prepare_foreign_modify(PgFdwModifyState *fmstate);
static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
ItemPointer tupleid,
TupleTableSlot *slot);
! static void store_returning_result(TupleTableSlot *slot,
! PGresult *res,
! int row,
! Relation rel,
! AttInMetadata *attinmeta,
! List *retrieved_attrs,
! MemoryContext temp_context);
static int postgresAcquireSampleRowsFunc(Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
***************
*** 853,860 **** postgresGetForeignPlan(PlannerInfo *root,
* Build the fdw_private list that will be available to the executor.
* Items in the list must match enum FdwScanPrivateIndex, above.
*/
! fdw_private = list_make2(makeString(sql.data),
! retrieved_attrs);
/*
* Create the ForeignScan node from target list, local filtering
--- 903,912 ----
* Build the fdw_private list that will be available to the executor.
* Items in the list must match enum FdwScanPrivateIndex, above.
*/
! fdw_private = list_make4(makeString(sql.data),
! remote_conds,
! retrieved_attrs,
! makeInteger(false));
/*
* Create the ForeignScan node from target list, local filtering
***************
*** 903,908 **** postgresBeginForeignScan(ForeignScanState *node, int eflags)
--- 955,966 ----
node->fdw_state = (void *) fsstate;
/*
+ * Decide whether we push an UPDATE/DELETE command down.
+ */
+ fsstate->update_pushed_down = intVal(list_nth(fsplan->fdw_private,
+ FdwScanPrivateUpdatePushedDown));
+
+ /*
* Identify which user to do the remote access as. This should match what
* ExecCheckRTEPerms() does.
*/
***************
*** 921,935 **** postgresBeginForeignScan(ForeignScanState *node, int eflags)
*/
fsstate->conn = GetConnection(server, user, false);
! /* Assign a unique ID for my cursor */
! fsstate->cursor_number = GetCursorNumber(fsstate->conn);
! fsstate->cursor_exists = false;
! /* Get private info created by planner functions. */
! fsstate->query = strVal(list_nth(fsplan->fdw_private,
! FdwScanPrivateSelectSql));
! fsstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
! FdwScanPrivateRetrievedAttrs);
/* Create contexts for batches of tuples and per-tuple temp workspace. */
fsstate->batch_cxt = AllocSetContextCreate(estate->es_query_cxt,
--- 979,1011 ----
*/
fsstate->conn = GetConnection(server, user, false);
! if (fsstate->update_pushed_down)
! {
! /* Initialize state variable */
! fsstate->num_tuples = -1; /* -1 means not set yet */
! /* Get private info created by planner functions. */
! fsstate->query = strVal(list_nth(fsplan->fdw_private,
! FdwScanPrivateUpdateSql));
! fsstate->set_processed = intVal(list_nth(fsplan->fdw_private,
! FdwScanPrivateSetProcessed));
! fsstate->has_returning = intVal(list_nth(fsplan->fdw_private,
! FdwScanPrivateHasReturning));
! fsstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
! FdwScanPrivateRetrievedAttrsByReturning);
! }
! else
! {
! /* Assign a unique ID for my cursor */
! fsstate->cursor_number = GetCursorNumber(fsstate->conn);
! fsstate->cursor_exists = false;
!
! /* Get private info created by planner functions. */
! fsstate->query = strVal(list_nth(fsplan->fdw_private,
! FdwScanPrivateSelectSql));
! fsstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
! FdwScanPrivateRetrievedAttrsBySelect);
! }
/* Create contexts for batches of tuples and per-tuple temp workspace. */
fsstate->batch_cxt = AllocSetContextCreate(estate->es_query_cxt,
***************
*** 996,1001 **** postgresIterateForeignScan(ForeignScanState *node)
--- 1072,1094 ----
TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
/*
+ * Get the result of an UPDATE/DELETE RETURNING, if pushing the command down.
+ */
+ if (fsstate->update_pushed_down)
+ {
+ /*
+ * If this is the first call after Begin, execute the command.
+ */
+ if (fsstate->num_tuples == -1)
+ execute_pushed_down_command(node);
+
+ /*
+ * Get the next returning result.
+ */
+ return get_next_returning_result(node);
+ }
+
+ /*
* If this is the first call after Begin or ReScan, we need to create the
* cursor on the remote side.
*/
***************
*** 1037,1042 **** postgresReScanForeignScan(ForeignScanState *node)
--- 1130,1138 ----
char sql[64];
PGresult *res;
+ /* This shouldn't be called in update pushdown case. */
+ Assert(fsstate->update_pushed_down == false);
+
/* If we haven't created the cursor yet, nothing to do. */
if (!fsstate->cursor_exists)
return;
***************
*** 1095,1103 **** postgresEndForeignScan(ForeignScanState *node)
if (fsstate == NULL)
return;
! /* Close the cursor if open, to prevent accumulation of cursors */
! if (fsstate->cursor_exists)
! close_cursor(fsstate->conn, fsstate->cursor_number);
/* Release remote connection */
ReleaseConnection(fsstate->conn);
--- 1191,1208 ----
if (fsstate == NULL)
return;
! if (fsstate->update_pushed_down)
! {
! /* Clean up */
! if (fsstate->result)
! PQclear(fsstate->result);
! }
! else
! {
! /* Close the cursor if open, to prevent accumulation of cursors */
! if (fsstate->cursor_exists)
! close_cursor(fsstate->conn, fsstate->cursor_number);
! }
/* Release remote connection */
ReleaseConnection(fsstate->conn);
***************
*** 1146,1158 **** postgresAddForeignUpdateTargets(Query *parsetree,
/*
* postgresPlanForeignModify
* Plan an insert/update/delete operation on a foreign table
- *
- * Note: currently, the plan tree generated for UPDATE/DELETE will always
- * include a ForeignScan that retrieves ctids (using SELECT FOR UPDATE)
- * and then the ModifyTable node will have to execute individual remote
- * UPDATE/DELETE commands. If there are no local conditions or joins
- * needed, it'd be better to let the scan node do UPDATE/DELETE RETURNING
- * and then do nothing at ModifyTable. Room for future optimization ...
*/
static List *
postgresPlanForeignModify(PlannerInfo *root,
--- 1251,1256 ----
***************
*** 1168,1175 **** postgresPlanForeignModify(PlannerInfo *root,
List *returningList = NIL;
List *retrieved_attrs = NIL;
- initStringInfo(&sql);
-
/*
* Core code already has some lock on each rel being planned, so we can
* use NoLock here.
--- 1266,1271 ----
***************
*** 1213,1218 **** postgresPlanForeignModify(PlannerInfo *root,
--- 1309,1337 ----
}
/*
+ * For UPDATE/DELETE, try to push the command down into the remote.
+ */
+ if (operation == CMD_UPDATE || operation == CMD_DELETE)
+ {
+ /* Check to see whether it's safe to push the command down. */
+ if (update_is_pushdown_safe(root, plan,
+ resultRelation,
+ subplan_index,
+ rel, targetAttrs))
+ {
+ List *fdw_private;
+
+ /* OK, modify plan so as to push the command down. */
+ fdw_private = push_update_down(root, plan,
+ resultRelation,
+ subplan_index,
+ rel, targetAttrs);
+ heap_close(rel, NoLock);
+ return fdw_private;
+ }
+ }
+
+ /*
* Extract the relevant RETURNING list if any.
*/
if (plan->returningLists)
***************
*** 1221,1226 **** postgresPlanForeignModify(PlannerInfo *root,
--- 1340,1346 ----
/*
* Construct the SQL command string.
*/
+ initStringInfo(&sql);
switch (operation)
{
case CMD_INSERT:
***************
*** 1291,1296 **** postgresBeginForeignModify(ModifyTableState *mtstate,
--- 1411,1452 ----
fmstate = (PgFdwModifyState *) palloc0(sizeof(PgFdwModifyState));
fmstate->rel = rel;
+ /* Deconstruct fdw_private data. */
+ fmstate->query = strVal(list_nth(fdw_private,
+ FdwModifyPrivateUpdateSql));
+ fmstate->target_attrs = (List *) list_nth(fdw_private,
+ FdwModifyPrivateTargetAttnums);
+ fmstate->has_returning = intVal(list_nth(fdw_private,
+ FdwModifyPrivateHasReturning));
+ fmstate->retrieved_attrs = (List *) list_nth(fdw_private,
+ FdwModifyPrivateRetrievedAttrs);
+
+ /*
+ * if query is NULL, we are in update pushdown case.
+ */
+ if (fmstate->query == NULL)
+ {
+ PlanState *node = mtstate->mt_plans[subplan_index];
+ PgFdwScanState *fsstate;
+
+ Assert(fmstate->target_attrs == NIL);
+ Assert(fmstate->has_returning == false);
+ Assert(fmstate->retrieved_attrs == NIL);
+
+ Assert(nodeTag(node) == T_ForeignScanState);
+ fsstate = (PgFdwScanState *) ((ForeignScanState *) node)->fdw_state;
+ Assert(fsstate->update_pushed_down);
+
+ fmstate->update_pushed_down = true;
+ if (fsstate->has_returning)
+ {
+ fmstate->has_returning = true;
+ fmstate->fsstate = fsstate;
+ }
+ resultRelInfo->ri_FdwState = fmstate;
+ return;
+ }
+
/*
* Identify which user to do the remote access as. This should match what
* ExecCheckRTEPerms() does.
***************
*** 1307,1322 **** postgresBeginForeignModify(ModifyTableState *mtstate,
fmstate->conn = GetConnection(server, user, true);
fmstate->p_name = NULL; /* prepared statement not made yet */
- /* Deconstruct fdw_private data. */
- fmstate->query = strVal(list_nth(fdw_private,
- FdwModifyPrivateUpdateSql));
- fmstate->target_attrs = (List *) list_nth(fdw_private,
- FdwModifyPrivateTargetAttnums);
- fmstate->has_returning = intVal(list_nth(fdw_private,
- FdwModifyPrivateHasReturning));
- fmstate->retrieved_attrs = (List *) list_nth(fdw_private,
- FdwModifyPrivateRetrievedAttrs);
-
/* Create context for per-tuple temp workspace. */
fmstate->temp_cxt = AllocSetContextCreate(estate->es_query_cxt,
"postgres_fdw temporary data",
--- 1463,1468 ----
***************
*** 1414,1420 **** postgresExecForeignInsert(EState *estate,
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(fmstate, slot, res);
}
else
n_rows = atoi(PQcmdTuples(res));
--- 1560,1570 ----
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(slot, res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
}
else
n_rows = atoi(PQcmdTuples(res));
***************
*** 1445,1450 **** postgresExecForeignUpdate(EState *estate,
--- 1595,1608 ----
PGresult *res;
int n_rows;
+ /* Just return the slot of the ForeignScan node, if pushing update down */
+ if (fmstate->update_pushed_down)
+ {
+ Assert(fmstate->has_returning);
+ Assert(fmstate->fsstate->rslot);
+ return fmstate->fsstate->rslot;
+ }
+
/* Set up the prepared statement on the remote server, if we didn't yet */
if (!fmstate->p_name)
prepare_foreign_modify(fmstate);
***************
*** 1484,1490 **** postgresExecForeignUpdate(EState *estate,
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(fmstate, slot, res);
}
else
n_rows = atoi(PQcmdTuples(res));
--- 1642,1652 ----
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(slot, res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
}
else
n_rows = atoi(PQcmdTuples(res));
***************
*** 1515,1520 **** postgresExecForeignDelete(EState *estate,
--- 1677,1690 ----
PGresult *res;
int n_rows;
+ /* Just return the slot of the ForeignScan node, if pushing update down */
+ if (fmstate->update_pushed_down)
+ {
+ Assert(fmstate->has_returning);
+ Assert(fmstate->fsstate->rslot);
+ return fmstate->fsstate->rslot;
+ }
+
/* Set up the prepared statement on the remote server, if we didn't yet */
if (!fmstate->p_name)
prepare_foreign_modify(fmstate);
***************
*** 1554,1560 **** postgresExecForeignDelete(EState *estate,
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(fmstate, slot, res);
}
else
n_rows = atoi(PQcmdTuples(res));
--- 1724,1734 ----
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(slot, res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
}
else
n_rows = atoi(PQcmdTuples(res));
***************
*** 1582,1587 **** postgresEndForeignModify(EState *estate,
--- 1756,1765 ----
if (fmstate == NULL)
return;
+ /* If pushing update down, nothing to do */
+ if (fmstate->update_pushed_down)
+ return;
+
/* If we created a prepared statement, destroy it */
if (fmstate->p_name)
{
***************
*** 1660,1670 **** postgresExplainForeignScan(ForeignScanState *node, ExplainState *es)
{
List *fdw_private;
char *sql;
if (es->verbose)
{
fdw_private = ((ForeignScan *) node->ss.ps.plan)->fdw_private;
! sql = strVal(list_nth(fdw_private, FdwScanPrivateSelectSql));
ExplainPropertyText("Remote SQL", sql, es);
}
}
--- 1838,1854 ----
{
List *fdw_private;
char *sql;
+ bool update_pushed_down;
if (es->verbose)
{
fdw_private = ((ForeignScan *) node->ss.ps.plan)->fdw_private;
! update_pushed_down = intVal(list_nth(fdw_private,
! FdwScanPrivateUpdatePushedDown));
! if (update_pushed_down)
! sql = strVal(list_nth(fdw_private, FdwScanPrivateUpdateSql));
! else
! sql = strVal(list_nth(fdw_private, FdwScanPrivateSelectSql));
ExplainPropertyText("Remote SQL", sql, es);
}
}
***************
*** 1685,1691 **** postgresExplainForeignModify(ModifyTableState *mtstate,
char *sql = strVal(list_nth(fdw_private,
FdwModifyPrivateUpdateSql));
! ExplainPropertyText("Remote SQL", sql, es);
}
}
--- 1869,1876 ----
char *sql = strVal(list_nth(fdw_private,
FdwModifyPrivateUpdateSql));
! if (sql != NULL)
! ExplainPropertyText("Remote SQL", sql, es);
}
}
***************
*** 1914,1919 **** ec_member_matches_foreign(PlannerInfo *root, RelOptInfo *rel,
--- 2099,2344 ----
}
/*
+ * Check to see whether it's safe to push an UPDATE/DELETE command down.
+ *
+ * Conditions checked here:
+ *
+ * 1. If the target relation has any row-level local BEFORE/AFTER triggers, we
+ * must not push the command down, since that breaks execution of the triggers.
+ *
+ * 2. If there are any local joins needed, we mustn't push the command down,
+ * because that breaks execution of the joins.
+ *
+ * 3. If there are any quals that can't be evaluated remotely, we mustn't push
+ * the command down, because that breaks evaluation of the quals.
+ *
+ * 4. In UPDATE, if it is unsafe to evaluate any expressions to assign to the
+ * target columns on the remote server, we must not push the command down.
+ */
+ static bool
+ update_is_pushdown_safe(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs)
+ {
+ CmdType operation = plan->operation;
+ RelOptInfo *baserel = root->simple_rel_array[resultRelation];
+ Plan *subplan = (Plan *) list_nth(plan->plans, subplan_index);
+ ListCell *lc;
+
+ /* Check point 1 */
+ if (rel->trigdesc &&
+ ((operation == CMD_UPDATE &&
+ (rel->trigdesc->trig_update_after_row ||
+ rel->trigdesc->trig_update_before_row)) ||
+ (operation == CMD_DELETE &&
+ (rel->trigdesc->trig_delete_after_row ||
+ rel->trigdesc->trig_delete_before_row))))
+ return false;
+
+ /* Check point 2 */
+ if (nodeTag(subplan) != T_ForeignScan)
+ return false;
+
+ /* Check point 3 */
+ if (subplan->qual != NIL)
+ return false;
+
+ /* Check point 4 */
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(subplan->targetlist,
+ attnum);
+
+ if (!is_foreign_expr(root, baserel, (Expr *) tle->expr))
+ return false;
+ }
+
+ return true;
+ }
+
+ /*
+ * Modify a plan so as to push an UPDATE/DELETE command down.
+ */
+ static List *
+ push_update_down(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs)
+ {
+ CmdType operation = plan->operation;
+ bool canSetTag = plan->canSetTag;
+ Plan *subplan = (Plan *) list_nth(plan->plans, subplan_index);
+ ForeignScan *fscan = (ForeignScan *) subplan;
+ StringInfoData sql;
+ List *remote_conds;
+ List *params_list = NIL;
+ List *returningList = NIL;
+ List *retrieved_attrs = NIL;
+ List *fdw_private;
+ ListCell *lc;
+
+ Assert(operation == CMD_UPDATE || operation == CMD_DELETE);
+
+ initStringInfo(&sql);
+
+ /*
+ * Extract the baserestrictinfo clauses that can be evaluated remotely.
+ */
+ remote_conds = (List *) list_nth(fscan->fdw_private,
+ FdwScanPrivateRemoteConds);
+
+ /*
+ * Extract the relevant RETURNING list if any.
+ */
+ if (plan->returningLists)
+ returningList = (List *) list_nth(plan->returningLists, subplan_index);
+
+ /*
+ * Construct the SQL command string.
+ */
+ if (operation == CMD_UPDATE)
+ {
+ List *targetlist = subplan->targetlist;
+
+ deparsePushedDownUpdateSql(&sql, root, resultRelation, rel,
+ targetlist,
+ targetAttrs,
+ remote_conds,
+ ¶ms_list,
+ returningList,
+ &retrieved_attrs);
+
+ /*
+ * Rrewrite targetlist for safety of ExecProject.
+ */
+ subplan->targetlist = rewrite_targetlist(resultRelation,
+ rel,
+ targetlist,
+ targetAttrs);
+ }
+ else
+ {
+ Assert(operation == CMD_DELETE);
+
+ deparsePushedDownDeleteSql(&sql, root, resultRelation, rel,
+ remote_conds,
+ ¶ms_list,
+ returningList,
+ &retrieved_attrs);
+ }
+
+ /*
+ * Update the operation info.
+ */
+ fscan->operation = operation;
+
+ /*
+ * Update the fdw_exprs list that will be available to the executor.
+ */
+ fscan->fdw_exprs = params_list;
+
+ /*
+ * Update the fdw_private list that will be available to the executor.
+ * Items in the list must match enum FdwScanPrivateIndex, above.
+ */
+ lc = list_nth_cell(fscan->fdw_private, FdwScanPrivateUpdatePushedDown);
+ lfirst(lc) = makeInteger(true);
+
+ fscan->fdw_private = lappend(fscan->fdw_private, makeString(sql.data));
+ fscan->fdw_private = lappend(fscan->fdw_private, makeInteger(canSetTag));
+ fscan->fdw_private = lappend(fscan->fdw_private,
+ makeInteger((retrieved_attrs != NIL)));
+ fscan->fdw_private = lappend(fscan->fdw_private, retrieved_attrs);
+
+ /*
+ * Build the fdw_private list that will be available to the executor.
+ * Items in the list must match enum FdwModifyPrivateIndex, above.
+ */
+ fdw_private = list_make4(makeString(NULL), NIL, makeInteger(false), NIL);
+
+ return fdw_private;
+ }
+
+ /*
+ * Rrewrite the targetlist of an UPDATE.
+ *
+ * This is called for safety of ExecProject if pushing the command down.
+ */
+ static List *
+ rewrite_targetlist(Index resultRelation, Relation rel,
+ List *targetlist, List *targetAttrs)
+ {
+ List *new_tlist = NIL;
+ int numattrs = RelationGetNumberOfAttributes(rel);
+ int attrno = 1;
+ ListCell *lc;
+
+ foreach(lc, targetlist)
+ {
+ TargetEntry *tle = (TargetEntry *) lfirst(lc);
+
+ if (tle->resjunk)
+ {
+ new_tlist = lappend(new_tlist, tle);
+ continue;
+ }
+
+ if (attrno > numattrs)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("table row type and query-specified row type do not match"),
+ errdetail("Query has too many columns.")));
+
+ if (!list_member_int(targetAttrs, attrno))
+ new_tlist = lappend(new_tlist, tle);
+ else
+ {
+ Form_pg_attribute attr = rel->rd_att->attrs[attrno - 1];
+ Oid atttype;
+ int32 atttypmod;
+ Oid attcollation;
+ Node *new_expr;
+ TargetEntry *new_tle;
+
+ Assert(!attr->attisdropped);
+ atttype = attr->atttypid;
+ atttypmod = attr->atttypmod;
+ attcollation = attr->attcollation;
+
+ new_expr = (Node *) makeVar(resultRelation,
+ attrno,
+ atttype,
+ atttypmod,
+ attcollation,
+ 0);
+
+ new_tle = makeTargetEntry((Expr *) new_expr,
+ attrno,
+ pstrdup(NameStr(attr->attname)),
+ false);
+
+ new_tlist = lappend(new_tlist, new_tle);
+ }
+
+ attrno++;
+ }
+
+ if (attrno != numattrs + 1)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("table row type and query-specified row type do not match"),
+ errdetail("Query has too few columns.")));
+
+ return new_tlist;
+ }
+
+ /*
* Create cursor for node's query with current parameter values.
*/
static void
***************
*** 2150,2155 **** close_cursor(PGconn *conn, unsigned int cursor_number)
--- 2575,2729 ----
}
/*
+ * Execute a pushed-down UPDATE/DELETE command.
+ */
+ static void
+ execute_pushed_down_command(ForeignScanState *node)
+ {
+ PgFdwScanState *fsstate = (PgFdwScanState *) node->fdw_state;
+ ExprContext *econtext = node->ss.ps.ps_ExprContext;
+ int numParams = fsstate->numParams;
+ const char **values = fsstate->param_values;
+
+ /*
+ * Construct array of query parameter values in text format. We do the
+ * conversions in the short-lived per-tuple context, so as not to cause a
+ * memory leak over repeated scans.
+ */
+ if (numParams > 0)
+ {
+ int nestlevel;
+ MemoryContext oldcontext;
+ int i;
+ ListCell *lc;
+
+ oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
+
+ nestlevel = set_transmission_modes();
+
+ i = 0;
+ foreach(lc, fsstate->param_exprs)
+ {
+ ExprState *expr_state = (ExprState *) lfirst(lc);
+ Datum expr_value;
+ bool isNull;
+
+ /* Evaluate the parameter expression */
+ expr_value = ExecEvalExpr(expr_state, econtext, &isNull, NULL);
+
+ /*
+ * Get string representation of each parameter value by invoking
+ * type-specific output function, unless the value is null.
+ */
+ if (isNull)
+ values[i] = NULL;
+ else
+ values[i] = OutputFunctionCall(&fsstate->param_flinfo[i],
+ expr_value);
+ i++;
+ }
+
+ reset_transmission_modes(nestlevel);
+
+ MemoryContextSwitchTo(oldcontext);
+ }
+
+ /*
+ * Notice that we pass NULL for paramTypes, thus forcing the remote server
+ * to infer types for all parameters. Since we explicitly cast every
+ * parameter (see deparse.c), the "inference" is trivial and will produce
+ * the desired result. This allows us to avoid assuming that the remote
+ * server has the same OIDs we do for the parameters' types.
+ *
+ * We don't use a PG_TRY block here, so be careful not to throw error
+ * without releasing the PGresult.
+ */
+ fsstate->result = PQexecParams(fsstate->conn, fsstate->query,
+ numParams, NULL, values, NULL, NULL, 0);
+ if (PQresultStatus(fsstate->result) !=
+ (fsstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
+ pgfdw_report_error(ERROR, fsstate->result, fsstate->conn, true,
+ fsstate->query);
+
+ /* Get the number of rows affected. */
+ if (fsstate->has_returning)
+ fsstate->num_tuples = PQntuples(fsstate->result);
+ else
+ fsstate->num_tuples = atoi(PQcmdTuples(fsstate->result));
+ }
+
+ /*
+ * Get the result of an UPDATE/DELETE RETURNING.
+ */
+ static TupleTableSlot *
+ get_next_returning_result(ForeignScanState *node)
+ {
+ PgFdwScanState *fsstate = (PgFdwScanState *) node->fdw_state;
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+ MemoryContext oldcontext;
+
+ /*
+ * If the update query doesn't have a RETURNING clause, then there is
+ * nothing to do, so we just return an empty slot.
+ */
+ if (!fsstate->has_returning)
+ {
+ /*
+ * Increment the command es_processed count if necessary.
+ * (Note: ModifyTable cannot do that by itself in this case.)
+ */
+ if (fsstate->set_processed)
+ {
+ EState *estate = node->ss.ps.state;
+
+ estate->es_processed += fsstate->num_tuples;
+ }
+
+ /*
+ * Increment the tuple count for EXPLAIN ANALYZE if necessary.
+ * (Note: EXPLAIN ANALYZE cannot do that by itself in this case.)
+ */
+ if (node->ss.ps.instrument)
+ {
+ Instrumentation *instr = node->ss.ps.instrument;
+
+ instr->ntuples += fsstate->num_tuples;
+ }
+
+ return ExecClearTuple(slot);
+ }
+
+ /* If we didn't get any tuples, must be end of data. */
+ if (fsstate->next_tuple >= fsstate->num_tuples)
+ return ExecClearTuple(slot);
+
+ /* OK, we'll store RETURNING tuples in the batch_cxt. */
+ oldcontext = MemoryContextSwitchTo(fsstate->batch_cxt);
+
+ /* Fetch the next tuple. */
+ store_returning_result(slot,
+ fsstate->result,
+ fsstate->next_tuple,
+ fsstate->rel,
+ fsstate->attinmeta,
+ fsstate->retrieved_attrs,
+ fsstate->temp_cxt);
+ fsstate->rslot = slot;
+ fsstate->next_tuple++;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ /*
+ * Return slot. Note that this is safe because we can avoid applying
+ * ExecQual to the tuple due to no local quals (see the comment for
+ * update_is_pushdown_safe) and because the tuple can be safely
+ * projected by ExecProject (see push_update_down) and would then be
+ * ignored by postgresExecForeignUpdate or postgresExecForeignDelete.
+ */
+ return slot;
+ }
+
+ /*
* prepare_foreign_modify
* Establish a prepared statement for execution of INSERT/UPDATE/DELETE
*/
***************
*** 2261,2278 **** convert_prep_stmt_params(PgFdwModifyState *fmstate,
* have PG_TRY blocks to ensure this happens.
*/
static void
! store_returning_result(PgFdwModifyState *fmstate,
! TupleTableSlot *slot, PGresult *res)
{
PG_TRY();
{
HeapTuple newtup;
! newtup = make_tuple_from_result_row(res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
/* tuple will be deleted when it is cleared from the slot */
ExecStoreTuple(newtup, slot, InvalidBuffer, true);
}
--- 2835,2857 ----
* have PG_TRY blocks to ensure this happens.
*/
static void
! store_returning_result(TupleTableSlot *slot,
! PGresult *res,
! int row,
! Relation rel,
! AttInMetadata *attinmeta,
! List *retrieved_attrs,
! MemoryContext temp_context)
{
PG_TRY();
{
HeapTuple newtup;
! newtup = make_tuple_from_result_row(res, row,
! rel,
! attinmeta,
! retrieved_attrs,
! temp_context);
/* tuple will be deleted when it is cleared from the slot */
ExecStoreTuple(newtup, slot, InvalidBuffer, true);
}
*** a/contrib/postgres_fdw/postgres_fdw.h
--- b/contrib/postgres_fdw/postgres_fdw.h
***************
*** 66,75 **** extern void deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 66,89 ----
Index rtindex, Relation rel,
List *targetAttrs, List *returningList,
List **retrieved_attrs);
+ extern void deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseDeleteSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *returningList,
List **retrieved_attrs);
+ extern void deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseAnalyzeSizeSql(StringInfo buf, Relation rel);
extern void deparseAnalyzeSql(StringInfo buf, Relation rel,
List **retrieved_attrs);
*** a/contrib/postgres_fdw/sql/postgres_fdw.sql
--- b/contrib/postgres_fdw/sql/postgres_fdw.sql
***************
*** 339,358 **** INSERT INTO ft2 (c1,c2,c3) SELECT c1+1000,c2+100, c3 || c3 FROM ft2 LIMIT 20;
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
--- 339,365 ----
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c7 = DEFAULT WHERE c1 % 10 = 0 AND date(c4) = '1970-01-01'::date; -- can't be pushed down
+ UPDATE ft2 SET c7 = DEFAULT WHERE c1 % 10 = 0 AND date(c4) = '1970-01-01'::date;
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
***************
*** 665,670 **** UPDATE rem1 SET f2 = 'testo';
--- 672,761 ----
-- Test returning a system attribute
INSERT INTO rem1(f2) VALUES ('test') RETURNING ctid;
+ -- cleanup
+ DROP TRIGGER trig_row_before ON rem1;
+ DROP TRIGGER trig_row_after ON rem1;
+ DROP TRIGGER trig_local_before ON loc1;
+
+
+ -- Test update-pushdown functionality
+
+ -- Test with statement-level triggers
+ CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_stmt_before ON rem1;
+
+ CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_stmt_after ON rem1;
+
+ -- Test with row-level ON INSERT triggers
+ CREATE TRIGGER trig_row_before_insert
+ BEFORE INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_before_insert ON rem1;
+
+ CREATE TRIGGER trig_row_after_insert
+ AFTER INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_after_insert ON rem1;
+
+ -- Test with row-level ON UPDATE triggers
+ CREATE TRIGGER trig_row_before_update
+ BEFORE UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_before_update ON rem1;
+
+ CREATE TRIGGER trig_row_after_update
+ AFTER UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_after_update ON rem1;
+
+ -- Test with row-level ON DELETE triggers
+ CREATE TRIGGER trig_row_before_delete
+ BEFORE DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ DROP TRIGGER trig_row_before_delete ON rem1;
+
+ CREATE TRIGGER trig_row_after_delete
+ AFTER DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ DROP TRIGGER trig_row_after_delete ON rem1;
+
-- ===================================================================
-- test IMPORT FOREIGN SCHEMA
-- ===================================================================
*** a/doc/src/sgml/postgres-fdw.sgml
--- b/doc/src/sgml/postgres-fdw.sgml
***************
*** 414,419 ****
--- 414,428 ----
<literal>WHERE</> clauses are not sent to the remote server unless they use
only built-in data types, operators, and functions. Operators and
functions in the clauses must be <literal>IMMUTABLE</> as well.
+ For an <command>UPDATE</> or <command>DELETE</> query,
+ <filename>postgres_fdw</> attempts to optimize the query execution by
+ sending the whole query to the remote server if there are no query
+ <literal>WHERE</> clauses that cannot be sent to the remote server,
+ no local joins for the query, or no row-level local <literal>BEFORE</> or
+ <literal>AFTER</> triggers on the target table. In <command>UPDATE</>,
+ expressions to assign to target columns must use only built-in data types,
+ <literal>IMMUTABLE</> operators, and <literal>IMMUTABLE</> functions,
+ to reduce the risk of misexecution of the query.
</para>
<para>
*** a/src/backend/commands/explain.c
--- b/src/backend/commands/explain.c
***************
*** 947,953 **** ExplainNode(PlanState *planstate, List *ancestors,
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
! pname = sname = "Foreign Scan";
break;
case T_CustomScan:
sname = "Custom Scan";
--- 947,971 ----
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
! sname = "Foreign Scan";
! switch (((ForeignScan *) plan)->operation)
! {
! case CMD_SELECT:
! pname = "Foreign Scan";
! operation = "Select";
! break;
! case CMD_UPDATE:
! pname = "Foreign Update";
! operation = "Update";
! break;
! case CMD_DELETE:
! pname = "Foreign Delete";
! operation = "Delete";
! break;
! default:
! pname = "???";
! break;
! }
break;
case T_CustomScan:
sname = "Custom Scan";
***************
*** 1683,1688 **** show_plan_tlist(PlanState *planstate, List *ancestors, ExplainState *es)
--- 1701,1711 ----
return;
if (IsA(plan, RecursiveUnion))
return;
+ /* Likewise for ForeignScan in case of pushed-down UPDATE/DELETE */
+ if (IsA(plan, ForeignScan) &&
+ (((ForeignScan *) plan)->operation == CMD_UPDATE ||
+ ((ForeignScan *) plan)->operation == CMD_DELETE))
+ return;
/* Set up deparsing context */
context = set_deparse_context_planstate(es->deparse_cxt,
*** a/src/backend/nodes/copyfuncs.c
--- b/src/backend/nodes/copyfuncs.c
***************
*** 592,597 **** _copyForeignScan(const ForeignScan *from)
--- 592,598 ----
/*
* copy remainder of node
*/
+ COPY_SCALAR_FIELD(operation);
COPY_NODE_FIELD(fdw_exprs);
COPY_NODE_FIELD(fdw_private);
COPY_SCALAR_FIELD(fsSystemCol);
*** a/src/backend/nodes/outfuncs.c
--- b/src/backend/nodes/outfuncs.c
***************
*** 558,563 **** _outForeignScan(StringInfo str, const ForeignScan *node)
--- 558,564 ----
_outScanInfo(str, (const Scan *) node);
+ WRITE_ENUM_FIELD(operation, CmdType);
WRITE_NODE_FIELD(fdw_exprs);
WRITE_NODE_FIELD(fdw_private);
WRITE_BOOL_FIELD(fsSystemCol);
*** a/src/backend/optimizer/plan/createplan.c
--- b/src/backend/optimizer/plan/createplan.c
***************
*** 3569,3574 **** make_foreignscan(List *qptlist,
--- 3569,3575 ----
plan->lefttree = NULL;
plan->righttree = NULL;
node->scan.scanrelid = scanrelid;
+ node->operation = CMD_SELECT;
node->fdw_exprs = fdw_exprs;
node->fdw_private = fdw_private;
/* fsSystemCol will be filled in by create_foreignscan_plan */
*** a/src/include/nodes/plannodes.h
--- b/src/include/nodes/plannodes.h
***************
*** 480,485 **** typedef struct WorkTableScan
--- 480,486 ----
typedef struct ForeignScan
{
Scan scan;
+ CmdType operation; /* SELECT, UPDATE, or DELETE */
List *fdw_exprs; /* expressions that FDW may evaluate */
List *fdw_private; /* private data for FDW */
bool fsSystemCol; /* true if any "system column" is needed */
On 2015/03/05 21:08, Etsuro Fujita wrote:
Here is an updated version.
The EXPLAIN output has also been improved as discussed in [1].
I noticed that the EXPLAIN for a pushed-down update (delete) on
inheritance childs doubly displays "Foreign Update" ("Foreign Delete"),
one for ForeignScan and the other for ModifyTable. Here is an example:
postgres=# explain verbose update parent set c1 = c1;
QUERY PLAN
------------------------------------------------------------------------------
Update on public.parent (cost=0.00..364.54 rows=4819 width=10)
Update on public.parent
Foreign Update on public.ft1
Foreign Update on public.ft2
-> Seq Scan on public.parent (cost=0.00..0.00 rows=1 width=10)
Output: parent.c1, parent.ctid
-> Foreign Update on public.ft1 (cost=100.00..182.27 rows=2409
width=10)
Remote SQL: UPDATE public.t1 SET c1 = c1
-> Foreign Update on public.ft2 (cost=100.00..182.27 rows=2409
width=10)
Remote SQL: UPDATE public.t2 SET c1 = c1
(10 rows)
Should we do something? Suggestions are welcome.
Best regards,
Etsuro Fujita
[1]: /messages/by-id/31942.1410534785@sss.pgh.pa.us
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 16-04-2015 PM 07:50, Etsuro Fujita wrote:
The EXPLAIN output has also been improved as discussed in [1].
I noticed that the EXPLAIN for a pushed-down update (delete) on inheritance
childs doubly displays "Foreign Update" ("Foreign Delete"), one for
ForeignScan and the other for ModifyTable. Here is an example:postgres=# explain verbose update parent set c1 = c1;
QUERY PLAN
------------------------------------------------------------------------------
Update on public.parent (cost=0.00..364.54 rows=4819 width=10)
Update on public.parent
Foreign Update on public.ft1
Foreign Update on public.ft2
-> Seq Scan on public.parent (cost=0.00..0.00 rows=1 width=10)
Output: parent.c1, parent.ctid
-> Foreign Update on public.ft1 (cost=100.00..182.27 rows=2409 width=10)
Remote SQL: UPDATE public.t1 SET c1 = c1
-> Foreign Update on public.ft2 (cost=100.00..182.27 rows=2409 width=10)
Remote SQL: UPDATE public.t2 SET c1 = c1
(10 rows)Should we do something? Suggestions are welcome.
From what I see in Tom's commit message[0] for FTI patch, this shouldn't be,
right?
To be specific, there should be "Foreign Scan" there as per the commit. Am I
missing something?
Thanks,
Amit
[1]: http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=cb1ca4d800621dcae67ca6c799006de99fa4f0a5
http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=cb1ca4d800621dcae67ca6c799006de99fa4f0a5
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2015/04/16 19:57, Amit Langote wrote:
On 16-04-2015 PM 07:50, Etsuro Fujita wrote:
The EXPLAIN output has also been improved as discussed in [1].
I noticed that the EXPLAIN for a pushed-down update (delete) on inheritance
childs doubly displays "Foreign Update" ("Foreign Delete"), one for
ForeignScan and the other for ModifyTable. Here is an example:postgres=# explain verbose update parent set c1 = c1;
QUERY PLAN
------------------------------------------------------------------------------
Update on public.parent (cost=0.00..364.54 rows=4819 width=10)
Update on public.parent
Foreign Update on public.ft1
Foreign Update on public.ft2
-> Seq Scan on public.parent (cost=0.00..0.00 rows=1 width=10)
Output: parent.c1, parent.ctid
-> Foreign Update on public.ft1 (cost=100.00..182.27 rows=2409 width=10)
Remote SQL: UPDATE public.t1 SET c1 = c1
-> Foreign Update on public.ft2 (cost=100.00..182.27 rows=2409 width=10)
Remote SQL: UPDATE public.t2 SET c1 = c1
(10 rows)Should we do something? Suggestions are welcome.
From what I see in Tom's commit message[0] for FTI patch, this shouldn't be,
right?
To be specific, there should be "Foreign Scan" there as per the commit. Am I
missing something?
As shown in the below example, this patch doesn't change the EXPLAIN
output for non-pushed-down update (delete) cases, but since we changed
the EXPLAIN output as discussed in [1], the patch doubly displays
"Foreign Update" ("Foreign Delete") for pushed-down update (delet) cases
like the above example.
postgres=# explain verbose update parent set c1 = trunc(random() * 9 +
1)::int;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------
Update on public.parent (cost=0.00..452.06 rows=5461 width=6)
Update on public.parent
Foreign Update on public.ft1
Remote SQL: UPDATE public.t1 SET c1 = $2 WHERE ctid = $1
Foreign Update on public.ft2
Remote SQL: UPDATE public.t2 SET c1 = $2 WHERE ctid = $1
-> Seq Scan on public.parent (cost=0.00..0.01 rows=1 width=6)
Output: (trunc(((random() * '9'::double precision) +
'1'::double precision)))::integer, parent.ctid
-> Foreign Scan on public.ft1 (cost=100.00..226.03 rows=2730 width=6)
Output: (trunc(((random() * '9'::double precision) +
'1'::double precision)))::integer, ft1.ctid
Remote SQL: SELECT ctid FROM public.t1 FOR UPDATE
-> Foreign Scan on public.ft2 (cost=100.00..226.03 rows=2730 width=6)
Output: (trunc(((random() * '9'::double precision) +
'1'::double precision)))::integer, ft2.ctid
Remote SQL: SELECT ctid FROM public.t2 FOR UPDATE
(14 rows)
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Fujita-san,
On 16-04-2015 PM 08:40, Etsuro Fujita wrote:
From what I see in Tom's commit message[0] for FTI patch, this shouldn't be,
right?To be specific, there should be "Foreign Scan" there as per the commit. Am I
missing something?As shown in the below example, this patch doesn't change the EXPLAIN
output for non-pushed-down update (delete) cases, but since we changed
the EXPLAIN output as discussed in [1], the patch doubly displays
"Foreign Update" ("Foreign Delete") for pushed-down update (delet) cases
like the above example.postgres=# explain verbose update parent set c1 = trunc(random() * 9 +
1)::int;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------
Update on public.parent (cost=0.00..452.06 rows=5461 width=6)
Update on public.parent
Foreign Update on public.ft1
Remote SQL: UPDATE public.t1 SET c1 = $2 WHERE ctid = $1
Foreign Update on public.ft2
Remote SQL: UPDATE public.t2 SET c1 = $2 WHERE ctid = $1
-> Seq Scan on public.parent (cost=0.00..0.01 rows=1 width=6)
Output: (trunc(((random() * '9'::double precision) +
'1'::double precision)))::integer, parent.ctid
-> Foreign Scan on public.ft1 (cost=100.00..226.03 rows=2730 width=6)
Output: (trunc(((random() * '9'::double precision) +
'1'::double precision)))::integer, ft1.ctid
Remote SQL: SELECT ctid FROM public.t1 FOR UPDATE
-> Foreign Scan on public.ft2 (cost=100.00..226.03 rows=2730 width=6)
Output: (trunc(((random() * '9'::double precision) +
'1'::double precision)))::integer, ft2.ctid
Remote SQL: SELECT ctid FROM public.t2 FOR UPDATE
(14 rows)
I think I missed the point that you are talking about the result after the
patch for foreign udpate pushdown (which is the topic of this thread) has been
applied. Sorry about the noise.
By the way, one suggestion may be to attach a "(pushed down)" to the
ModifyTable's "Foreign Update". And in that case, there would be no mention of
corresponding scan node in the list below exactly because there would be none.
postgres=# explain verbose update parent set c1 = c1;
QUERY PLAN
------------------------------------------------------------------------------
Update on public.parent (cost=0.00..364.54 rows=4819 width=10)
Update on public.parent
Foreign Update (pushed down) on public.ft1
Foreign Update (pushed down) on public.ft2
-> Seq Scan on public.parent (cost=0.00..0.00 rows=1 width=10)
Output: parent.c1, parent.ctid
Thoughts?
Thanks,
Amit
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2015/04/17 10:23, Amit Langote wrote:
By the way, one suggestion may be to attach a "(pushed down)" to the
ModifyTable's "Foreign Update". And in that case, there would be no mention of
corresponding scan node in the list below exactly because there would be none.postgres=# explain verbose update parent set c1 = c1;
QUERY PLAN
------------------------------------------------------------------------------
Update on public.parent (cost=0.00..364.54 rows=4819 width=10)
Update on public.parent
Foreign Update (pushed down) on public.ft1
Foreign Update (pushed down) on public.ft2
-> Seq Scan on public.parent (cost=0.00..0.00 rows=1 width=10)
Output: parent.c1, parent.ctid
Thanks for the suggestion!
I'm not sure that that is a good idea because (1) that is contrary to
the reality (the update pushdown patch lets the ForeignScan nodes do
UPDATE/DELETE RETURNING and then do nothing at ModifyTable!) and because
(2) that might cause the problem of associating subplans' update
information with subplans' scan information, pointed out by Tom [1]/messages/by-id/22505.1426986174@sss.pgh.pa.us.
Best regards,
Etsuro Fujita
[1]: /messages/by-id/22505.1426986174@sss.pgh.pa.us
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 17-04-2015 PM 12:35, Etsuro Fujita wrote:
On 2015/04/17 10:23, Amit Langote wrote:
By the way, one suggestion may be to attach a "(pushed down)" to the
ModifyTable's "Foreign Update". And in that case, there would be no mention of
corresponding scan node in the list below exactly because there would be none.postgres=# explain verbose update parent set c1 = c1;
QUERY PLAN
------------------------------------------------------------------------------
Update on public.parent (cost=0.00..364.54 rows=4819 width=10)
Update on public.parent
Foreign Update (pushed down) on public.ft1
Foreign Update (pushed down) on public.ft2
-> Seq Scan on public.parent (cost=0.00..0.00 rows=1 width=10)
Output: parent.c1, parent.ctidThanks for the suggestion!
I'm not sure that that is a good idea because (1) that is contrary to
the reality (the update pushdown patch lets the ForeignScan nodes do
UPDATE/DELETE RETURNING and then do nothing at ModifyTable!) and because
Ah, the reality is exactly the reverse then. Thanks for your patience.
(2) that might cause the problem of associating subplans' update
information with subplans' scan information, pointed out by Tom [1].
Having realized how it really works now, my +1 to "Foreign Modifying Scan" for
cases of pushed down update as suggested by Albe Laurenz. I guess it would be
signaled by the proposed ForeignScan.CmdType being CMD_UPDATE / CMP_UPDATE
(/CMD_INSERT).
Thanks,
Amit
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2015/04/17 13:16, Amit Langote wrote:
On 17-04-2015 PM 12:35, Etsuro Fujita wrote:
(2) that might cause the problem of associating subplans' update
information with subplans' scan information, pointed out by Tom [1].
Having realized how it really works now, my +1 to "Foreign Modifying Scan" for
cases of pushed down update as suggested by Albe Laurenz. I guess it would be
signaled by the proposed ForeignScan.CmdType being CMD_UPDATE / CMP_UPDATE
(/CMD_INSERT).
Thanks for the opinion! I think that that is an idea. However, I'd
like to propose to rename "Foreign Update" ("Foreign Delete") of
ModifyTable simply to "Update" ("Delete") not only because (1) that
solves the duplication problem but also because (2) ISTM that is
consistent with the non-inherited updates in both of the
non-pushed-down-update case and the pushed-down-update case. Here are
examples for (2).
* Inherited and non-inherited updates for the non-pushed-down case:
postgres=# explain verbose update parent set c1 = trunc(random() * 9 +
1)::int;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------
Update on public.parent (cost=0.00..452.06 rows=5461 width=6)
Update on public.parent
Update on public.ft1
Remote SQL: UPDATE public.t1 SET c1 = $2 WHERE ctid = $1
Update on public.ft2
Remote SQL: UPDATE public.t2 SET c1 = $2 WHERE ctid = $1
-> Seq Scan on public.parent (cost=0.00..0.01 rows=1 width=6)
Output: (trunc(((random() * '9'::double precision) +
'1'::double precision)))::integer, parent.ctid
-> Foreign Scan on public.ft1 (cost=100.00..226.03 rows=2730 width=6)
Output: (trunc(((random() * '9'::double precision) +
'1'::double precision)))::integer, ft1.ctid
Remote SQL: SELECT ctid FROM public.t1 FOR UPDATE
-> Foreign Scan on public.ft2 (cost=100.00..226.03 rows=2730 width=6)
Output: (trunc(((random() * '9'::double precision) +
'1'::double precision)))::integer, ft2.ctid
Remote SQL: SELECT ctid FROM public.t2 FOR UPDATE
(14 rows)
postgres=# explain verbose update ft1 set c1 = trunc(random() * 9 + 1)::int;
QUERY PLAN
------------------------------------------------------------------------------------------------------
Update on public.ft1 (cost=100.00..226.03 rows=2730 width=6)
Remote SQL: UPDATE public.t1 SET c1 = $2 WHERE ctid = $1
-> Foreign Scan on public.ft1 (cost=100.00..226.03 rows=2730 width=6)
Output: (trunc(((random() * '9'::double precision) +
'1'::double precision)))::integer, ctid
Remote SQL: SELECT ctid FROM public.t1 FOR UPDATE
(5 rows)
* Inherited and non-inherited updates for the pushed-down case:
postgres=# explain verbose update parent set c1 = c1 + 1;
QUERY PLAN
------------------------------------------------------------------------------
Update on public.parent (cost=0.00..376.59 rows=4819 width=10)
Update on public.parent
Update on public.ft1
Update on public.ft2
-> Seq Scan on public.parent (cost=0.00..0.00 rows=1 width=10)
Output: (parent.c1 + 1), parent.ctid
-> Foreign Update on public.ft1 (cost=100.00..188.29 rows=2409
width=10)
Remote SQL: UPDATE public.t1 SET c1 = (c1 + 1)
-> Foreign Update on public.ft2 (cost=100.00..188.29 rows=2409
width=10)
Remote SQL: UPDATE public.t2 SET c1 = (c1 + 1)
(10 rows)
postgres=# explain verbose update ft1 set c1 = c1 + 1;
QUERY PLAN
------------------------------------------------------------------------------
Update on public.ft1 (cost=100.00..188.29 rows=2409 width=10)
-> Foreign Update on public.ft1 (cost=100.00..188.29 rows=2409
width=10)
Remote SQL: UPDATE public.t1 SET c1 = (c1 + 1)
(3 rows)
Comments are welcome.
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hi,
At Mon, 20 Apr 2015 16:40:52 +0900, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp> wrote in <5534AD84.3020501@lab.ntt.co.jp>
On 2015/04/17 13:16, Amit Langote wrote:
On 17-04-2015 PM 12:35, Etsuro Fujita wrote:
(2) that might cause the problem of associating subplans' update
information with subplans' scan information, pointed out by Tom [1].Having realized how it really works now, my +1 to "Foreign Modifying Scan" for
cases of pushed down update as suggested by Albe Laurenz. I guess it would be
signaled by the proposed ForeignScan.CmdType being CMD_UPDATE / CMP_UPDATE
(/CMD_INSERT).Thanks for the opinion! I think that that is an idea. However, I'd
like to propose to rename "Foreign Update" ("Foreign Delete") of
ModifyTable simply to "Update" ("Delete") not only because (1) that
solves the duplication problem but also because (2) ISTM that is
consistent with the non-inherited updates in both of the
non-pushed-down-update case and the pushed-down-update case. Here are
examples for (2).
Update node without "Foreign" that runs "Remote SQL" looks to me
somewhat unusual..
It seems to me that the problem is "Foreign Update"s for
ModifyTable that does nothing eventually. Even though I don't
understand this fully, especially what "Foreign Update" for
ModifyTable does when "Foreign Update" in place of "Foreign Scan"
finished substantial work, I think the ForeignUpdate nodes should
be removed during planning if it is really ineffective, or such
"Foreign Update"s shoud be renamed or provided with some
explaination in explain output if it does anything or unremovable
from some reason.
If removed it looks like,
| =# explain verbose update p set b = b + 1;
| QUERY PLAN
| ------------------------------------------------------------------------------
| Update on public.p (cost=0.00..360.08 rows=4311 width=14)
| Update on public.p
| -> Seq Scan on public.p (cost=0.00..0.00 rows=1 width=14)
| Output: p.a, (p.b + 1), p.ctid
| -> Foreign Update on public.ft1 (cost=100.00..180.04 rows=2155 width=14)
| Remote SQL: UPDATE public.t1 SET b = (b + 1)
| -> Foreign Update on public.ft2 (cost=100.00..180.04 rows=2155 width=14)
| Remote SQL: UPDATE public.t2 SET b = (b + 1)
regards,
* Inherited and non-inherited updates for the non-pushed-down case:
postgres=# explain verbose update parent set c1 = trunc(random() * 9 +
1)::int;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------
Update on public.parent (cost=0.00..452.06 rows=5461 width=6)
Update on public.parent
Update on public.ft1
Remote SQL: UPDATE public.t1 SET c1 = $2 WHERE ctid = $1
Update on public.ft2
Remote SQL: UPDATE public.t2 SET c1 = $2 WHERE ctid = $1
-> Seq Scan on public.parent (cost=0.00..0.01 rows=1 width=6)
Output: (trunc(((random() * '9'::double precision) +
'1'::double precision)))::integer, parent.ctid
-> Foreign Scan on public.ft1 (cost=100.00..226.03 rows=2730 width=6)
Output: (trunc(((random() * '9'::double precision) +
'1'::double precision)))::integer, ft1.ctid
Remote SQL: SELECT ctid FROM public.t1 FOR UPDATE
-> Foreign Scan on public.ft2 (cost=100.00..226.03 rows=2730 width=6)
Output: (trunc(((random() * '9'::double precision) +
'1'::double precision)))::integer, ft2.ctid
Remote SQL: SELECT ctid FROM public.t2 FOR UPDATE
(14 rows)postgres=# explain verbose update ft1 set c1 = trunc(random() * 9 + 1)::int;
QUERY PLAN
------------------------------------------------------------------------------------------------------
Update on public.ft1 (cost=100.00..226.03 rows=2730 width=6)
Remote SQL: UPDATE public.t1 SET c1 = $2 WHERE ctid = $1
-> Foreign Scan on public.ft1 (cost=100.00..226.03 rows=2730 width=6)
Output: (trunc(((random() * '9'::double precision) +
'1'::double precision)))::integer, ctid
Remote SQL: SELECT ctid FROM public.t1 FOR UPDATE
(5 rows)* Inherited and non-inherited updates for the pushed-down case:
postgres=# explain verbose update parent set c1 = c1 + 1;
QUERY PLAN
------------------------------------------------------------------------------
Update on public.parent (cost=0.00..376.59 rows=4819 width=10)
Update on public.parent
Update on public.ft1
Update on public.ft2
-> Seq Scan on public.parent (cost=0.00..0.00 rows=1 width=10)
Output: (parent.c1 + 1), parent.ctid
-> Foreign Update on public.ft1 (cost=100.00..188.29 rows=2409
width=10)
Remote SQL: UPDATE public.t1 SET c1 = (c1 + 1)
-> Foreign Update on public.ft2 (cost=100.00..188.29 rows=2409
width=10)
Remote SQL: UPDATE public.t2 SET c1 = (c1 + 1)
(10 rows)postgres=# explain verbose update ft1 set c1 = c1 + 1;
QUERY PLAN
------------------------------------------------------------------------------
Update on public.ft1 (cost=100.00..188.29 rows=2409 width=10)
-> Foreign Update on public.ft1 (cost=100.00..188.29 rows=2409
width=10)
Remote SQL: UPDATE public.t1 SET c1 = (c1 + 1)
(3 rows)Comments are welcome.
--
Kyotaro Horiguchi
NTT Open Source Software Center
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2015/04/21 10:07, Kyotaro HORIGUCHI wrote:
At Mon, 20 Apr 2015 16:40:52 +0900, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp> wrote in <5534AD84.3020501@lab.ntt.co.jp>
However, I'd
like to propose to rename "Foreign Update" ("Foreign Delete") of
ModifyTable simply to "Update" ("Delete") not only because (1) that
solves the duplication problem but also because (2) ISTM that is
consistent with the non-inherited updates in both of the
non-pushed-down-update case and the pushed-down-update case. Here are
examples for (2).
Update node without "Foreign" that runs "Remote SQL" looks to me
somewhat unusual..
I think that has a similarity with the existing EXPLAIN outputs for
non-inherited non-pushed-down updates, as shown in the below exaple.
postgres=# explain verbose update ft1 set c1 = trunc(random() * 9 +
1)::int;
QUERY PLAN
------------------------------------------------------------------------------------------------------
Update on public.ft1 (cost=100.00..226.03 rows=2730 width=6)
Remote SQL: UPDATE public.t1 SET c1 = $2 WHERE ctid = $1
-> Foreign Scan on public.ft1 (cost=100.00..226.03 rows=2730
width=6)
Output: (trunc(((random() * '9'::double precision) +
'1'::double precision)))::integer, ctid
Remote SQL: SELECT ctid FROM public.t1 FOR UPDATE
(5 rows)
It seems to me that the problem is "Foreign Update"s for
ModifyTable that does nothing eventually.
I think the ForeignUpdate nodes should
be removed during planning if it is really ineffective,
If removed it looks like,
| =# explain verbose update p set b = b + 1;
| QUERY PLAN
| ------------------------------------------------------------------------------
| Update on public.p (cost=0.00..360.08 rows=4311 width=14)
| Update on public.p
| -> Seq Scan on public.p (cost=0.00..0.00 rows=1 width=14)
| Output: p.a, (p.b + 1), p.ctid
| -> Foreign Update on public.ft1 (cost=100.00..180.04 rows=2155 width=14)
| Remote SQL: UPDATE public.t1 SET b = (b + 1)
| -> Foreign Update on public.ft2 (cost=100.00..180.04 rows=2155 width=14)
| Remote SQL: UPDATE public.t2 SET b = (b + 1)
On that point, I agree with Tom that that would cause the problem that
the user has to guess at which of the child plans goes with which target
relation of ModifyTable [1]/messages/by-id/22505.1426986174@sss.pgh.pa.us.
Thanks for the comments!
Best regards,
Etsuro Fujita
[1]: /messages/by-id/22505.1426986174@sss.pgh.pa.us
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hi, thank you. My understanding became a bit clearer.
At Tue, 21 Apr 2015 15:35:41 +0900, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp> wrote in <5535EFBD.8030006@lab.ntt.co.jp>
On 2015/04/21 10:07, Kyotaro HORIGUCHI wrote:
At Mon, 20 Apr 2015 16:40:52 +0900, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote in
<5534AD84.3020501@lab.ntt.co.jp>However, I'd
like to propose to rename "Foreign Update" ("Foreign Delete") of
ModifyTable simply to "Update" ("Delete") not only because (1) that
solves the duplication problem but also because (2) ISTM that is
consistent with the non-inherited updates in both of the
non-pushed-down-update case and the pushed-down-update case. Here are
examples for (2).Update node without "Foreign" that runs "Remote SQL" looks to me
somewhat unusual..I think that has a similarity with the existing EXPLAIN outputs for
non-inherited non-pushed-down updates, as shown in the below exaple.postgres=# explain verbose update ft1 set c1 = trunc(random() * 9 +
1)::int;
QUERY PLAN------------------------------------------------------------------------------------------------------
Update on public.ft1 (cost=100.00..226.03 rows=2730 width=6)
Remote SQL: UPDATE public.t1 SET c1 = $2 WHERE ctid = $1
-> Foreign Scan on public.ft1 (cost=100.00..226.03 rows=2730 width=6)
Output: (trunc(((random() * '9'::double precision) +
'1'::double precision)))::integer, ctid
Remote SQL: SELECT ctid FROM public.t1 FOR UPDATE
(5 rows)
Mmm.. It also looks confusing which needs to be fixed. Now
foreign tables are updated in two ways. One is ModifyTable on
foreign relation and the another is ForeignScan node of update
operation. Though I think that I understand the path to this
form, but I suppose they should confluent into one type of node,
perhaps ForegnScan node. Even if it is hardly archievable for
now, explain representation should be uniform.
Making ModifyTable on foreign relation have the representation
"Foreign Update", the explain results of the queries modifying
foreign tables are looks like,
Foreign Update on public.ft1 (...)
Remote SQL: UPDATE public.t1 ....
-> Foreign Scan on public.ft1...
Foreign Update on public.ft1 (...
Foreign Update on public.ft1 (...
Remote SQL: ...
If Foreign Update has only one internal representation, the two
same Foreign Updates are (ideally) easily eliminated during
planning and explain would naturally shows the following result.
Foreign Update on public.ft1 (...
Remote SQL: ...
But if not as is currently so, printing the result needs a bit
complicated calculation.
| Update on public.p (cost=0.00..360.08 rows=4311 width=14)
| Update on public.p
| -> Seq Scan on public.p (cost=0.00..0.00 rows=1 width=14)
| Output: p.a, (p.b + 1), p.ctid
| -> Foreign Update on public.ft1 (cost=100.00..180.04 rows=2155
| -> width=14)
| Remote SQL: UPDATE public.t1 SET b = (b + 1)
| -> Foreign Update on public.ft2 (cost=100.00..180.04 rows=2155
| -> width=14)
| Remote SQL: UPDATE public.t2 SET b = (b + 1)On that point, I agree with Tom that that would cause the problem that
the user has to guess at which of the child plans goes with which
target relation of ModifyTable [1].
Yeah, that seems to make the plan to be understood
clerer. Combining Tom's suggestion and my suggestion together
would result in the following output of explain.
Update on public.p (cost=0.00..360.08 rows=4311 width=14)
Update on public.p
-> Seq Scan on public.p (cost=0.00..0.00 rows=1 width=14)
Output: p.a, (p.b + 1), p.ctid
Foreign Update on public.ft1 (cost=100.00..180.04 rows=2155 width=14)
Remote SQL: UPDATE public.t1 SET b = (b + 1)
Foreign Update on public.ft2 (cost=100.00..180.04 rows=2155 width=14)
Remote SQL: UPDATE public.t2 SET b = (b + 1)
And when not pushed down it would look like,
Update on public.p (cost=0.00..360.08 rows=4311 width=14)
Update on public.p
-> Seq Scan on public.p (cost=0.00..0.00 rows=1 width=14)
Output: p.a, (p.b + 1), p.ctid
Foreign Update on public.ft1 (cost=100.00..180.04 rows=2155 width=14)
Remote SQL: UPDATE public.t1 SET b = $2 WHERE ctid = $1
-> Foreign Scan on public.ft1 (cost=....)
Output: a, b, ctid
Remote SQL: SELECT a, ctid FROM public.t1 FOR UPDATE
Foreign Update on public.ft2 (cost=100.00..180.04 rows=2155 width=14)
Remote SQL: UPDATE public.t2 SET b = (b + 1)
-> Foreign Scan on public.ft2 (cost=....)
Output: a, b, ctid
Remote SQL: SELECT a, ctid FROM public.t2 FOR UPDATE
These looks quite reasonable *for me* :)
Of course, the same discussion is applicable on Foreign Delete.
What do you think about this?
Any further thoughts?
regards,
--
Kyotaro Horiguchi
NTT Open Source Software Center
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Etsuro,
* Etsuro Fujita (fujita.etsuro@lab.ntt.co.jp) wrote:
Here is an updated version. In this version, the bug has been
fixed, but any regression tests for that hasn't been added, because
I'm not sure that the above way is a good idea and don't have any
other ideas.The EXPLAIN output has also been improved as discussed in [1].
While the EXPLAIN output changed, the structure hasn't really changed
from what was discussed previously and there's not been any real
involvment from the core code in what's happening here.
Clearly, the documentation around how to use the FDW API hasn't changed
at all and there's been no additions to it for handling bulk work.
Everything here continues to be done inside of postgres_fdw, which
essentially ignores the prescribed "Update/Delete one tuple" interface
for ExecForeignUpdate/ExecForeignDelete.
I've spent the better part of the past two days trying to reason my way
around that while reviewing this patch and I haven't come out the other
side any happier with this approach than I was back in
20140911153049.GC16422@tamriel.snowman.net.
There are other things that don't look right to me, such as what's going
on at the bottom of push_update_down(), but I don't think there's much
point going into it until we figure out what the core FDW API here
should look like. It might not be all that far from what we have now,
but I don't think we can just ignore the existing, documented, API.
Thanks!
Stephen
On 2015/05/13 0:55, Stephen Frost wrote:
Etsuro,
* Etsuro Fujita (fujita.etsuro@lab.ntt.co.jp) wrote:
Here is an updated version. In this version, the bug has been
fixed, but any regression tests for that hasn't been added, because
I'm not sure that the above way is a good idea and don't have any
other ideas.The EXPLAIN output has also been improved as discussed in [1].
While the EXPLAIN output changed, the structure hasn't really changed
from what was discussed previously and there's not been any real
involvment from the core code in what's happening here.Clearly, the documentation around how to use the FDW API hasn't changed
at all and there's been no additions to it for handling bulk work.
Everything here continues to be done inside of postgres_fdw, which
essentially ignores the prescribed "Update/Delete one tuple" interface
for ExecForeignUpdate/ExecForeignDelete.I've spent the better part of the past two days trying to reason my way
around that while reviewing this patch and I haven't come out the other
side any happier with this approach than I was back in
20140911153049.GC16422@tamriel.snowman.net.There are other things that don't look right to me, such as what's going
on at the bottom of push_update_down(), but I don't think there's much
point going into it until we figure out what the core FDW API here
should look like. It might not be all that far from what we have now,
but I don't think we can just ignore the existing, documented, API.
OK, I'll try to introduce the core FDW API for this (and make changes to
the core code) to address your previous comments.
Thanks for taking the time to review the patch!
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 13 May 2015 at 04:10, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp> wrote:
On 2015/05/13 0:55, Stephen Frost wrote:
Etsuro,
* Etsuro Fujita (fujita.etsuro@lab.ntt.co.jp) wrote:
Here is an updated version. In this version, the bug has been
fixed, but any regression tests for that hasn't been added, because
I'm not sure that the above way is a good idea and don't have any
other ideas.The EXPLAIN output has also been improved as discussed in [1].
While the EXPLAIN output changed, the structure hasn't really changed
from what was discussed previously and there's not been any real
involvment from the core code in what's happening here.Clearly, the documentation around how to use the FDW API hasn't changed
at all and there's been no additions to it for handling bulk work.
Everything here continues to be done inside of postgres_fdw, which
essentially ignores the prescribed "Update/Delete one tuple" interface
for ExecForeignUpdate/ExecForeignDelete.I've spent the better part of the past two days trying to reason my way
around that while reviewing this patch and I haven't come out the other
side any happier with this approach than I was back in
20140911153049.GC16422@tamriel.snowman.net.There are other things that don't look right to me, such as what's going
on at the bottom of push_update_down(), but I don't think there's much
point going into it until we figure out what the core FDW API here
should look like. It might not be all that far from what we have now,
but I don't think we can just ignore the existing, documented, API.OK, I'll try to introduce the core FDW API for this (and make changes to the
core code) to address your previous comments.Thanks for taking the time to review the patch!
Fujita-san,
I'm a bit behind in reading up on this, so maybe it's been covered
since, but is there a discussion of this API on another thread, or a
newer patch available?
Thanks
Thom
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hi Thom,
Thank you for paying attention to this!
On 2015/11/25 20:36, Thom Brown wrote:
On 13 May 2015 at 04:10, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp> wrote:
On 2015/05/13 0:55, Stephen Frost wrote:
While the EXPLAIN output changed, the structure hasn't really changed
from what was discussed previously and there's not been any real
involvment from the core code in what's happening here.Clearly, the documentation around how to use the FDW API hasn't changed
at all and there's been no additions to it for handling bulk work.
Everything here continues to be done inside of postgres_fdw, which
essentially ignores the prescribed "Update/Delete one tuple" interface
for ExecForeignUpdate/ExecForeignDelete.I've spent the better part of the past two days trying to reason my way
around that while reviewing this patch and I haven't come out the other
side any happier with this approach than I was back in
20140911153049.GC16422@tamriel.snowman.net.There are other things that don't look right to me, such as what's going
on at the bottom of push_update_down(), but I don't think there's much
point going into it until we figure out what the core FDW API here
should look like. It might not be all that far from what we have now,
but I don't think we can just ignore the existing, documented, API.
OK, I'll try to introduce the core FDW API for this (and make changes to the
core code) to address your previous comments.
I'm a bit behind in reading up on this, so maybe it's been covered
since, but is there a discussion of this API on another thread, or a
newer patch available?
Actually, I'm now working on this. My basic idea is to add new FDW APIs
for handling the bulk work, in order not to make messy the prescribed
"Update/Delete one tuple" interface; 1) add to nodeModifyTable.c or
nodeForeignscan.c the new FDW APIs BeginPushedDownForeignModify,
ExecPushedDownForeignModify and EndPushedDownForeignModify for that, and
2) call these FDW APIs, instead of BeginForeignModify,
ExecForeignUpdate/ExecForeignDelete and EndForeignModify, when doing
update pushdown. I'd like to propose that in more detail as soon as
possible, probably with an updated patch.
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2015/11/26 18:00, Etsuro Fujita wrote:
On 2015/11/25 20:36, Thom Brown wrote:
On 13 May 2015 at 04:10, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>
wrote:On 2015/05/13 0:55, Stephen Frost wrote:
While the EXPLAIN output changed, the structure hasn't really changed
from what was discussed previously and there's not been any real
involvment from the core code in what's happening here.Clearly, the documentation around how to use the FDW API hasn't changed
at all and there's been no additions to it for handling bulk work.
Everything here continues to be done inside of postgres_fdw, which
essentially ignores the prescribed "Update/Delete one tuple" interface
for ExecForeignUpdate/ExecForeignDelete.I've spent the better part of the past two days trying to reason my way
around that while reviewing this patch and I haven't come out the other
side any happier with this approach than I was back in
20140911153049.GC16422@tamriel.snowman.net.There are other things that don't look right to me, such as what's
going
on at the bottom of push_update_down(), but I don't think there's much
point going into it until we figure out what the core FDW API here
should look like. It might not be all that far from what we have now,
but I don't think we can just ignore the existing, documented, API.
OK, I'll try to introduce the core FDW API for this (and make changes
to the
core code) to address your previous comments.
I'm a bit behind in reading up on this, so maybe it's been covered
since, but is there a discussion of this API on another thread, or a
newer patch available?
To address Stephen's comments, I'd like to propose the following FDW APIs:
bool
PlanDMLPushdown (PlannerInfo *root,
ModifyTable *plan,
Index resultRelation,
int subplan_index);
This is called in make_modifytable, before calling PlanForeignModify.
This checks to see whether a given UPDATE/DELETE .. RETURNING .. is
pushdown-safe and if so, performs planning actions needed for the DML
pushdown. The idea is to modify a ForeignScan subplan accordingly as in
the previous patch. If the DML is pushdown-safe, this returns true, and
we don't call PlanForeignModify anymore. (Else returns false and call
PlanForeignModify as before.) When the DML is pushdown-safe, we hook
the following FDW APIs located in nodeForeignscan.c, instead of
BeginForeignModify, ExecForeignUpdate/ExecForeignDelete and
EndForeignModify:
void
BeginDMLPushdown (ForeignScanState *node,
int eflags);
This initializes the DML pushdown, like BeginForeignScan.
TupleTableSlot *
IterateDMLPushdown (ForeignScanState *node);
This fetches one returning result from the foreign server, like
IterateForeignScan, if having a RETURNING clause. If not, just return
an empty slot. (I'm thinking that it's required that the FDW replaces
the targetlist of the ForeignScan subplan to the RETURNING clause during
PlanDMLPushdown, if having the clause, so that we do nothing at
ModifyTable.)
void
EndDMLPushdown (ForeignScanState *node);
This finishes the DML pushdown, like EndForeignScan.
I'm attaching a WIP patch, which only includes changes to the core. I'm
now working on the postgres_fdw patch to demonstrate that these APIs
work well, but I'd be happy if I could get any feedback earlier.
Best regards,
Etsuro Fujita
Attachments:
fdw-dml-pushdown-v1.patchapplication/x-patch; name=fdw-dml-pushdown-v1.patchDownload
*** a/src/backend/commands/explain.c
--- b/src/backend/commands/explain.c
***************
*** 887,893 **** ExplainNode(PlanState *planstate, List *ancestors,
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
! pname = sname = "Foreign Scan";
break;
case T_CustomScan:
sname = "Custom Scan";
--- 887,911 ----
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
! sname = "Foreign Scan";
! switch (((ForeignScan *) plan)->operation)
! {
! case CMD_SELECT:
! pname = "Foreign Scan";
! operation = "Select";
! break;
! case CMD_UPDATE:
! pname = "Foreign Update";
! operation = "Update";
! break;
! case CMD_DELETE:
! pname = "Foreign Delete";
! operation = "Delete";
! break;
! default:
! pname = "???";
! break;
! }
break;
case T_CustomScan:
sname = "Custom Scan";
***************
*** 1658,1663 **** show_plan_tlist(PlanState *planstate, List *ancestors, ExplainState *es)
--- 1676,1686 ----
return;
if (IsA(plan, RecursiveUnion))
return;
+ /* Likewise for ForeignScan in case of pushed-down UPDATE/DELETE */
+ if (IsA(plan, ForeignScan) &&
+ (((ForeignScan *) plan)->operation == CMD_UPDATE ||
+ ((ForeignScan *) plan)->operation == CMD_DELETE))
+ return;
/* Set up deparsing context */
context = set_deparse_context_planstate(es->deparse_cxt,
*** a/src/backend/executor/execMain.c
--- b/src/backend/executor/execMain.c
***************
*** 1011,1020 **** InitPlan(QueryDesc *queryDesc, int eflags)
* CheckValidRowMarkRel.
*/
void
! CheckValidResultRel(Relation resultRel, CmdType operation)
{
TriggerDesc *trigDesc = resultRel->trigdesc;
FdwRoutine *fdwroutine;
switch (resultRel->rd_rel->relkind)
{
--- 1011,1022 ----
* CheckValidRowMarkRel.
*/
void
! CheckValidResultRel(ResultRelInfo *resultRelInfo, CmdType operation)
{
+ Relation resultRel = resultRelInfo->ri_RelationDesc;
TriggerDesc *trigDesc = resultRel->trigdesc;
FdwRoutine *fdwroutine;
+ bool supported;
switch (resultRel->rd_rel->relkind)
{
***************
*** 1083,1096 **** CheckValidResultRel(Relation resultRel, CmdType operation)
case RELKIND_FOREIGN_TABLE:
/* Okay only if the FDW supports it */
fdwroutine = GetFdwRoutineForRelation(resultRel, false);
switch (operation)
{
case CMD_INSERT:
! if (fdwroutine->ExecForeignInsert == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("cannot insert into foreign table \"%s\"",
! RelationGetRelationName(resultRel))));
if (fdwroutine->IsForeignRelUpdatable != NULL &&
(fdwroutine->IsForeignRelUpdatable(resultRel) & (1 << CMD_INSERT)) == 0)
ereport(ERROR,
--- 1085,1107 ----
case RELKIND_FOREIGN_TABLE:
/* Okay only if the FDW supports it */
fdwroutine = GetFdwRoutineForRelation(resultRel, false);
+ supported = ((fdwroutine->BeginDMLPushdown != NULL) &&
+ (fdwroutine->IterateDMLPushdown != NULL) &&
+ (fdwroutine->EndDMLPushdown != NULL));
switch (operation)
{
case CMD_INSERT:
! if (resultRelInfo->ri_FdwPushdown && !supported)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("cannot push down insert on foreign table \"%s\"",
! RelationGetRelationName(resultRel))));
! if (!resultRelInfo->ri_FdwPushdown &&
! fdwroutine->ExecForeignInsert == NULL)
! ereport(ERROR,
! (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("cannot insert into foreign table \"%s\"",
! RelationGetRelationName(resultRel))));
if (fdwroutine->IsForeignRelUpdatable != NULL &&
(fdwroutine->IsForeignRelUpdatable(resultRel) & (1 << CMD_INSERT)) == 0)
ereport(ERROR,
***************
*** 1099,1105 **** CheckValidResultRel(Relation resultRel, CmdType operation)
RelationGetRelationName(resultRel))));
break;
case CMD_UPDATE:
! if (fdwroutine->ExecForeignUpdate == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot update foreign table \"%s\"",
--- 1110,1122 ----
RelationGetRelationName(resultRel))));
break;
case CMD_UPDATE:
! if (resultRelInfo->ri_FdwPushdown && !supported)
! ereport(ERROR,
! (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("cannot push down update on foreign table \"%s\"",
! RelationGetRelationName(resultRel))));
! if (!resultRelInfo->ri_FdwPushdown &&
! fdwroutine->ExecForeignUpdate == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot update foreign table \"%s\"",
***************
*** 1112,1118 **** CheckValidResultRel(Relation resultRel, CmdType operation)
RelationGetRelationName(resultRel))));
break;
case CMD_DELETE:
! if (fdwroutine->ExecForeignDelete == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot delete from foreign table \"%s\"",
--- 1129,1141 ----
RelationGetRelationName(resultRel))));
break;
case CMD_DELETE:
! if (resultRelInfo->ri_FdwPushdown && !supported)
! ereport(ERROR,
! (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("cannot push down delete on foreign table \"%s\"",
! RelationGetRelationName(resultRel))));
! if (!resultRelInfo->ri_FdwPushdown &&
! fdwroutine->ExecForeignDelete == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot delete from foreign table \"%s\"",
***************
*** 1245,1250 **** InitResultRelInfo(ResultRelInfo *resultRelInfo,
--- 1268,1274 ----
else
resultRelInfo->ri_FdwRoutine = NULL;
resultRelInfo->ri_FdwState = NULL;
+ resultRelInfo->ri_FdwPushdown = false;
resultRelInfo->ri_ConstraintExprs = NULL;
resultRelInfo->ri_junkFilter = NULL;
resultRelInfo->ri_projectReturning = NULL;
*** a/src/backend/executor/nodeForeignscan.c
--- b/src/backend/executor/nodeForeignscan.c
***************
*** 48,54 **** ForeignNext(ForeignScanState *node)
/* Call the Iterate function in short-lived context */
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! slot = node->fdwroutine->IterateForeignScan(node);
MemoryContextSwitchTo(oldcontext);
/*
--- 48,57 ----
/* Call the Iterate function in short-lived context */
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! if (plan->operation != CMD_SELECT)
! slot = node->fdwroutine->IterateDMLPushdown(node);
! else
! slot = node->fdwroutine->IterateForeignScan(node);
MemoryContextSwitchTo(oldcontext);
/*
***************
*** 226,232 **** ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
/*
* Tell the FDW to initialize the scan.
*/
! fdwroutine->BeginForeignScan(scanstate, eflags);
return scanstate;
}
--- 229,238 ----
/*
* Tell the FDW to initialize the scan.
*/
! if (node->operation != CMD_SELECT)
! fdwroutine->BeginDMLPushdown(scanstate, eflags);
! else
! fdwroutine->BeginForeignScan(scanstate, eflags);
return scanstate;
}
***************
*** 240,247 **** ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
void
ExecEndForeignScan(ForeignScanState *node)
{
/* Let the FDW shut down */
! node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
--- 246,258 ----
void
ExecEndForeignScan(ForeignScanState *node)
{
+ ForeignScan *plan = (ForeignScan *) node->ss.ps.plan;
+
/* Let the FDW shut down */
! if (plan->operation != CMD_SELECT)
! node->fdwroutine->EndDMLPushdown(node);
! else
! node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
*** a/src/backend/executor/nodeModifyTable.c
--- b/src/backend/executor/nodeModifyTable.c
***************
*** 1348,1353 **** ExecModifyTable(ModifyTableState *node)
--- 1348,1359 ----
break;
}
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ estate->es_result_relation_info = saved_resultRelInfo;
+ return planSlot;
+ }
+
EvalPlanQualSetSlot(&node->mt_epqstate, planSlot);
slot = planSlot;
***************
*** 1527,1536 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
{
subplan = (Plan *) lfirst(l);
/*
* Verify result relation is a valid target for the current operation
*/
! CheckValidResultRel(resultRelInfo->ri_RelationDesc, operation);
/*
* If there are indices on the result relation, open them and save
--- 1533,1545 ----
{
subplan = (Plan *) lfirst(l);
+ /* Initialize the FdwPushdown flag */
+ resultRelInfo->ri_FdwPushdown = list_nth_int(node->fdwPushdowns, i);
+
/*
* Verify result relation is a valid target for the current operation
*/
! CheckValidResultRel(resultRelInfo, operation);
/*
* If there are indices on the result relation, open them and save
***************
*** 1551,1557 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
/* Also let FDWs init themselves for foreign-table result rels */
! if (resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, i);
--- 1560,1567 ----
mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
/* Also let FDWs init themselves for foreign-table result rels */
! if (resultRelInfo->ri_FdwPushdown == false &&
! resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, i);
***************
*** 1722,1734 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
--- 1732,1753 ----
erm = ExecFindRowMark(estate, rc->rti, false);
/* build ExecAuxRowMark for each subplan */
+ resultRelInfo = mtstate->resultRelInfo;
for (i = 0; i < nplans; i++)
{
ExecAuxRowMark *aerm;
+ /* ignore subplan if the FDW pushes the statement down */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ resultRelInfo++;
+ continue;
+ }
+
subplan = mtstate->mt_plans[i]->plan;
aerm = ExecBuildAuxRowMark(erm, subplan->targetlist);
mtstate->mt_arowmarks[i] = lappend(mtstate->mt_arowmarks[i], aerm);
+ resultRelInfo++;
}
}
***************
*** 1784,1789 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
--- 1803,1815 ----
{
JunkFilter *j;
+ /* ignore subplan if the FDW pushes the statement down */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ resultRelInfo++;
+ continue;
+ }
+
subplan = mtstate->mt_plans[i]->plan;
if (operation == CMD_INSERT || operation == CMD_UPDATE)
ExecCheckPlanOutput(resultRelInfo->ri_RelationDesc,
***************
*** 1878,1884 **** ExecEndModifyTable(ModifyTableState *node)
{
ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
! if (resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->EndForeignModify != NULL)
resultRelInfo->ri_FdwRoutine->EndForeignModify(node->ps.state,
resultRelInfo);
--- 1904,1911 ----
{
ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
! if (resultRelInfo->ri_FdwPushdown == false &&
! resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->EndForeignModify != NULL)
resultRelInfo->ri_FdwRoutine->EndForeignModify(node->ps.state,
resultRelInfo);
*** a/src/backend/nodes/copyfuncs.c
--- b/src/backend/nodes/copyfuncs.c
***************
*** 185,190 **** _copyModifyTable(const ModifyTable *from)
--- 185,191 ----
COPY_NODE_FIELD(plans);
COPY_NODE_FIELD(withCheckOptionLists);
COPY_NODE_FIELD(returningLists);
+ COPY_NODE_FIELD(fdwPushdowns);
COPY_NODE_FIELD(fdwPrivLists);
COPY_NODE_FIELD(rowMarks);
COPY_SCALAR_FIELD(epqParam);
***************
*** 645,650 **** _copyForeignScan(const ForeignScan *from)
--- 646,652 ----
/*
* copy remainder of node
*/
+ COPY_SCALAR_FIELD(operation);
COPY_SCALAR_FIELD(fs_server);
COPY_NODE_FIELD(fdw_exprs);
COPY_NODE_FIELD(fdw_private);
*** a/src/backend/nodes/outfuncs.c
--- b/src/backend/nodes/outfuncs.c
***************
*** 339,344 **** _outModifyTable(StringInfo str, const ModifyTable *node)
--- 339,345 ----
WRITE_NODE_FIELD(plans);
WRITE_NODE_FIELD(withCheckOptionLists);
WRITE_NODE_FIELD(returningLists);
+ WRITE_NODE_FIELD(fdwPushdowns);
WRITE_NODE_FIELD(fdwPrivLists);
WRITE_NODE_FIELD(rowMarks);
WRITE_INT_FIELD(epqParam);
***************
*** 591,596 **** _outForeignScan(StringInfo str, const ForeignScan *node)
--- 592,598 ----
_outScanInfo(str, (const Scan *) node);
+ WRITE_ENUM_FIELD(operation, CmdType);
WRITE_OID_FIELD(fs_server);
WRITE_NODE_FIELD(fdw_exprs);
WRITE_NODE_FIELD(fdw_private);
*** a/src/backend/nodes/readfuncs.c
--- b/src/backend/nodes/readfuncs.c
***************
*** 1470,1475 **** _readModifyTable(void)
--- 1470,1476 ----
READ_NODE_FIELD(plans);
READ_NODE_FIELD(withCheckOptionLists);
READ_NODE_FIELD(returningLists);
+ READ_NODE_FIELD(fdwPushdowns);
READ_NODE_FIELD(fdwPrivLists);
READ_NODE_FIELD(rowMarks);
READ_INT_FIELD(epqParam);
*** a/src/backend/optimizer/plan/createplan.c
--- b/src/backend/optimizer/plan/createplan.c
***************
*** 3765,3770 **** make_foreignscan(List *qptlist,
--- 3765,3771 ----
plan->lefttree = outer_plan;
plan->righttree = NULL;
node->scan.scanrelid = scanrelid;
+ node->operation = CMD_SELECT;
/* fs_server will be filled in by create_foreignscan_plan */
node->fs_server = InvalidOid;
node->fdw_exprs = fdw_exprs;
***************
*** 5039,5044 **** make_modifytable(PlannerInfo *root,
--- 5040,5046 ----
ModifyTable *node = makeNode(ModifyTable);
Plan *plan = &node->plan;
double total_size;
+ List *fdwpushdown_list;
List *fdw_private_list;
ListCell *subnode;
ListCell *lc;
***************
*** 5119,5130 **** make_modifytable(PlannerInfo *root,
--- 5121,5134 ----
* For each result relation that is a foreign table, allow the FDW to
* construct private plan data, and accumulate it all into a list.
*/
+ fdwpushdown_list = NIL;
fdw_private_list = NIL;
i = 0;
foreach(lc, resultRelations)
{
Index rti = lfirst_int(lc);
FdwRoutine *fdwroutine;
+ bool fdwpushdown;
List *fdw_private;
/*
***************
*** 5153,5158 **** make_modifytable(PlannerInfo *root,
--- 5157,5170 ----
}
if (fdwroutine != NULL &&
+ fdwroutine->PlanDMLPushdown != NULL)
+ fdwpushdown = fdwroutine->PlanDMLPushdown(root, node, rti, i);
+ else
+ fdwpushdown = false;
+ fdwpushdown_list = lappend_int(fdwpushdown_list, fdwpushdown);
+
+ if (!fdwpushdown &&
+ fdwroutine != NULL &&
fdwroutine->PlanForeignModify != NULL)
fdw_private = fdwroutine->PlanForeignModify(root, node, rti, i);
else
***************
*** 5160,5165 **** make_modifytable(PlannerInfo *root,
--- 5172,5178 ----
fdw_private_list = lappend(fdw_private_list, fdw_private);
i++;
}
+ node->fdwPushdowns = fdwpushdown_list;
node->fdwPrivLists = fdw_private_list;
return node;
*** a/src/include/executor/executor.h
--- b/src/include/executor/executor.h
***************
*** 184,190 **** extern void ExecutorEnd(QueryDesc *queryDesc);
extern void standard_ExecutorEnd(QueryDesc *queryDesc);
extern void ExecutorRewind(QueryDesc *queryDesc);
extern bool ExecCheckRTPerms(List *rangeTable, bool ereport_on_violation);
! extern void CheckValidResultRel(Relation resultRel, CmdType operation);
extern void InitResultRelInfo(ResultRelInfo *resultRelInfo,
Relation resultRelationDesc,
Index resultRelationIndex,
--- 184,190 ----
extern void standard_ExecutorEnd(QueryDesc *queryDesc);
extern void ExecutorRewind(QueryDesc *queryDesc);
extern bool ExecCheckRTPerms(List *rangeTable, bool ereport_on_violation);
! extern void CheckValidResultRel(ResultRelInfo *resultRelInfo, CmdType operation);
extern void InitResultRelInfo(ResultRelInfo *resultRelInfo,
Relation resultRelationDesc,
Index resultRelationIndex,
*** a/src/include/foreign/fdwapi.h
--- b/src/include/foreign/fdwapi.h
***************
*** 91,96 **** typedef TupleTableSlot *(*ExecForeignDelete_function) (EState *estate,
--- 91,108 ----
typedef void (*EndForeignModify_function) (EState *estate,
ResultRelInfo *rinfo);
+ typedef bool (*PlanDMLPushdown_function) (PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+
+ typedef void (*BeginDMLPushdown_function) (ForeignScanState *node,
+ int eflags);
+
+ typedef TupleTableSlot *(*IterateDMLPushdown_function) (ForeignScanState *node);
+
+ typedef void (*EndDMLPushdown_function) (ForeignScanState *node);
+
typedef int (*IsForeignRelUpdatable_function) (Relation rel);
typedef RowMarkType (*GetForeignRowMarkType_function) (RangeTblEntry *rte,
***************
*** 161,166 **** typedef struct FdwRoutine
--- 173,182 ----
ExecForeignUpdate_function ExecForeignUpdate;
ExecForeignDelete_function ExecForeignDelete;
EndForeignModify_function EndForeignModify;
+ PlanDMLPushdown_function PlanDMLPushdown;
+ BeginDMLPushdown_function BeginDMLPushdown;
+ IterateDMLPushdown_function IterateDMLPushdown;
+ EndDMLPushdown_function EndDMLPushdown;
IsForeignRelUpdatable_function IsForeignRelUpdatable;
/* Functions for SELECT FOR UPDATE/SHARE row locking */
*** a/src/include/nodes/execnodes.h
--- b/src/include/nodes/execnodes.h
***************
*** 311,316 **** typedef struct JunkFilter
--- 311,317 ----
* TrigInstrument optional runtime measurements for triggers
* FdwRoutine FDW callback functions, if foreign table
* FdwState available to save private state of FDW
+ * FdwPushdown true when the command is pushed down
* WithCheckOptions list of WithCheckOption's to be checked
* WithCheckOptionExprs list of WithCheckOption expr states
* ConstraintExprs array of constraint-checking expr states
***************
*** 334,339 **** typedef struct ResultRelInfo
--- 335,341 ----
Instrumentation *ri_TrigInstrument;
struct FdwRoutine *ri_FdwRoutine;
void *ri_FdwState;
+ bool ri_FdwPushdown;
List *ri_WithCheckOptions;
List *ri_WithCheckOptionExprs;
List **ri_ConstraintExprs;
*** a/src/include/nodes/plannodes.h
--- b/src/include/nodes/plannodes.h
***************
*** 187,192 **** typedef struct ModifyTable
--- 187,193 ----
List *plans; /* plan(s) producing source data */
List *withCheckOptionLists; /* per-target-table WCO lists */
List *returningLists; /* per-target-table RETURNING tlists */
+ List *fdwPushdowns; /* per-target-table FDW pushdown flags */
List *fdwPrivLists; /* per-target-table FDW private data lists */
List *rowMarks; /* PlanRowMarks (non-locking only) */
int epqParam; /* ID of Param for EvalPlanQual re-eval */
***************
*** 530,535 **** typedef struct WorkTableScan
--- 531,537 ----
typedef struct ForeignScan
{
Scan scan;
+ CmdType operation; /* SELECT/UPDATE/DELETE */
Oid fs_server; /* OID of foreign server */
List *fdw_exprs; /* expressions that FDW may evaluate */
List *fdw_private; /* private data for FDW */
On Mon, Dec 21, 2015 at 6:20 PM, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>
wrote:
On 2015/11/26 18:00, Etsuro Fujita wrote:
On 2015/11/25 20:36, Thom Brown wrote:
On 13 May 2015 at 04:10, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>
wrote:On 2015/05/13 0:55, Stephen Frost wrote:
While the EXPLAIN output changed, the structure hasn't really changed
from what was discussed previously and there's not been any real
involvment from the core code in what's happening here.Clearly, the documentation around how to use the FDW API hasn't changed
at all and there's been no additions to it for handling bulk work.
Everything here continues to be done inside of postgres_fdw, which
essentially ignores the prescribed "Update/Delete one tuple" interface
for ExecForeignUpdate/ExecForeignDelete.I've spent the better part of the past two days trying to reason my way
around that while reviewing this patch and I haven't come out the other
side any happier with this approach than I was back in
20140911153049.GC16422@tamriel.snowman.net.There are other things that don't look right to me, such as what's
going
on at the bottom of push_update_down(), but I don't think there's much
point going into it until we figure out what the core FDW API here
should look like. It might not be all that far from what we have now,
but I don't think we can just ignore the existing, documented, API.OK, I'll try to introduce the core FDW API for this (and make changes
to the
core code) to address your previous comments.I'm a bit behind in reading up on this, so maybe it's been covered
since, but is there a discussion of this API on another thread, or a
newer patch available?To address Stephen's comments, I'd like to propose the following FDW APIs:
bool
PlanDMLPushdown (PlannerInfo *root,
ModifyTable *plan,
Index resultRelation,
int subplan_index);This is called in make_modifytable, before calling PlanForeignModify. This
checks to see whether a given UPDATE/DELETE .. RETURNING .. is
pushdown-safe and if so, performs planning actions needed for the DML
pushdown. The idea is to modify a ForeignScan subplan accordingly as in
the previous patch. If the DML is pushdown-safe, this returns true, and we
don't call PlanForeignModify anymore. (Else returns false and call
PlanForeignModify as before.) When the DML is pushdown-safe, we hook the
following FDW APIs located in nodeForeignscan.c, instead of
BeginForeignModify, ExecForeignUpdate/ExecForeignDelete and
EndForeignModify:void
BeginDMLPushdown (ForeignScanState *node,
int eflags);This initializes the DML pushdown, like BeginForeignScan.
TupleTableSlot *
IterateDMLPushdown (ForeignScanState *node);This fetches one returning result from the foreign server, like
IterateForeignScan, if having a RETURNING clause. If not, just return an
empty slot. (I'm thinking that it's required that the FDW replaces the
targetlist of the ForeignScan subplan to the RETURNING clause during
PlanDMLPushdown, if having the clause, so that we do nothing at
ModifyTable.)void
EndDMLPushdown (ForeignScanState *node);This finishes the DML pushdown, like EndForeignScan.
+1.
I like idea of separate FDW API for the DML Pushdown. Was thinking can't we
can re-use the IterateForeignScan(ForeignScanState *node) rather then
introducing IterateDMLPushdown(ForeignScanState *node) new API ?
I'm attaching a WIP patch, which only includes changes to the core. I'm
now working on the postgres_fdw patch to demonstrate that these APIs work
well, but I'd be happy if I could get any feedback earlier.Best regards,
Etsuro Fujita--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
--
Rushabh Lathia
On Wed, Dec 23, 2015 at 5:50 AM, Rushabh Lathia
<rushabh.lathia@gmail.com> wrote:
+1.
I like idea of separate FDW API for the DML Pushdown. Was thinking can't we
can re-use the IterateForeignScan(ForeignScanState *node) rather then
introducing IterateDMLPushdown(ForeignScanState *node) new API ?
Yeah, I think we need to ask ourselves what advantage we're getting
out of adding any new core APIs. Marking the scan as a pushed-down
update or delete has some benefit in terms of making the information
visible via EXPLAIN, but even that's a pretty thin benefit. The
iterate method seems to just complicate the core code without any
benefit at all. More generally, there is very, very little code in
this patch that accomplishes anything that could not be done just as
well with the existing methods. So why are we even doing these core
changes?
Tom seemed to think that we could centralize some checks in the core
code, say, related to triggers, or to LIMIT. But there's nothing like
that in this patch, so I'm not really understanding the point.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2015/12/24 4:34, Robert Haas wrote:
On Wed, Dec 23, 2015 at 5:50 AM, Rushabh Lathia
<rushabh.lathia@gmail.com> wrote:+1.
I like idea of separate FDW API for the DML Pushdown. Was thinking can't we
can re-use the IterateForeignScan(ForeignScanState *node) rather then
introducing IterateDMLPushdown(ForeignScanState *node) new API ?
Yeah, I think we need to ask ourselves what advantage we're getting
out of adding any new core APIs. Marking the scan as a pushed-down
update or delete has some benefit in terms of making the information
visible via EXPLAIN, but even that's a pretty thin benefit. The
iterate method seems to just complicate the core code without any
benefit at all. More generally, there is very, very little code in
this patch that accomplishes anything that could not be done just as
well with the existing methods. So why are we even doing these core
changes?
From the FDWs' point of view, ISTM that what FDWs have to do for
IterateDMLPushdown is quite different from what FDWs have to do for
IterateForeignScan; eg, IterateDMLPushdown must work in accordance with
presence/absence of a RETURNING list. (In addition to that,
IterateDMLPushdown has been designed so that it must make the scan tuple
available to later RETURNING projection in nodeModifyTable.c.) So, I
think that it's better to FDWs to add separate APIs for the DML
pushdown, making the FDW code much simpler. So based on that idea, I
added the postgres_fdw changes to the patch. Attached is an updated
version of the patch, which is still WIP, but I'd be happy if I could
get any feedback.
Tom seemed to think that we could centralize some checks in the core
code, say, related to triggers, or to LIMIT. But there's nothing like
that in this patch, so I'm not really understanding the point.
For the trigger check, I added relation_has_row_level_triggers. I
placed that function in postgres_fdw.c in the updated patch, but I think
that by placing that function in the core, FDWs can share that function.
As for the LIMIT, I'm not sure we can do something about that.
I think the current design allows us to handle a pushed-down update on a
join, ie, "UPDATE foo ... FROM bar ..." where both foo and bar are
remote, which was Tom's concern, but I'll leave that for another patch.
Also, I think the current design could also extend to push down INSERT
.. RETURNING .., but I'd like to leave that for future work.
I'll add this to the next CF.
Best regards,
Etsuro Fujita
Attachments:
fdw-dml-pushdown-v2.patchapplication/x-patch; name=fdw-dml-pushdown-v2.patchDownload
*** a/contrib/postgres_fdw/deparse.c
--- b/contrib/postgres_fdw/deparse.c
***************
*** 816,822 **** deparseTargetList(StringInfo buf,
*
* If params is not NULL, it receives a list of Params and other-relation Vars
* used in the clauses; these values must be transmitted to the remote server
! * as parameter values.
*
* If params is NULL, we're generating the query for EXPLAIN purposes,
* so Params and other-relation Vars should be replaced by dummy values.
--- 816,822 ----
*
* If params is not NULL, it receives a list of Params and other-relation Vars
* used in the clauses; these values must be transmitted to the remote server
! * as parameter values. Caller is responsible for initializing it to empty.
*
* If params is NULL, we're generating the query for EXPLAIN purposes,
* so Params and other-relation Vars should be replaced by dummy values.
***************
*** 833,841 **** appendWhereClause(StringInfo buf,
int nestlevel;
ListCell *lc;
- if (params)
- *params = NIL; /* initialize result list to empty */
-
/* Set up context struct for recursion */
context.root = root;
context.foreignrel = baserel;
--- 833,838 ----
***************
*** 971,976 **** deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 968,1030 ----
}
/*
+ * deparse remote UPDATE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ deparse_expr_cxt context;
+ bool first;
+ ListCell *lc;
+
+ if (params_list)
+ *params_list = NIL; /* initialize result list to empty */
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = params_list;
+
+ appendStringInfoString(buf, "UPDATE ");
+ deparseRelation(buf, rel);
+ appendStringInfoString(buf, " SET ");
+
+ first = true;
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(targetlist, attnum);
+
+ if (!first)
+ appendStringInfoString(buf, ", ");
+ first = false;
+
+ deparseColumnRef(buf, rtindex, attnum, root);
+ appendStringInfoString(buf, " = ");
+ deparseExpr((Expr *) tle->expr, &context);
+ }
+ if (remote_conds)
+ appendWhereClause(buf, root, baserel, remote_conds,
+ true, params_list);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* deparse remote DELETE statement
*
* The statement text is appended to buf, and we also create an integer List
***************
*** 993,998 **** deparseDeleteSql(StringInfo buf, PlannerInfo *root,
--- 1047,1082 ----
}
/*
+ * deparse remote DELETE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+
+ if (params_list)
+ *params_list = NIL; /* initialize result list to empty */
+
+ appendStringInfoString(buf, "DELETE FROM ");
+ deparseRelation(buf, rel);
+ if (remote_conds)
+ appendWhereClause(buf, root, baserel, remote_conds,
+ true, params_list);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* Add a RETURNING clause, if needed, to an INSERT/UPDATE/DELETE.
*/
static void
*** a/contrib/postgres_fdw/expected/postgres_fdw.out
--- b/contrib/postgres_fdw/expected/postgres_fdw.out
***************
*** 1314,1320 **** INSERT INTO ft2 (c1,c2,c3)
--- 1314,1339 ----
(3 rows)
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 300), c3 = (c3 || '_update3'::text) WHERE ((("C 1" % 10) = 3))
+ (3 rows)
+
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Output: c1, c2, c3, c4, c5, c6, c7, c8
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 400), c3 = (c3 || '_update7'::text) WHERE ((("C 1" % 10) = 7)) RETURNING "C 1", c2, c3, c4, c5, c6, c7, c8
+ (4 rows)
+
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
c1 | c2 | c3 | c4 | c5 | c6 | c7 | c8
------+-----+--------------------+------------------------------+--------------------------+----+------------+-----
***************
*** 1424,1430 **** UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
--- 1443,1449 ----
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
***************
*** 1445,1460 **** UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
! QUERY PLAN
! ----------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1 RETURNING "C 1", c4
! -> Foreign Scan on public.ft2
! Output: ctid
! Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) FOR UPDATE
! (6 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
--- 1464,1477 ----
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
! QUERY PLAN
! --------------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
! -> Foreign Delete on public.ft2
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) RETURNING "C 1", c4
! (4 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
***************
*** 1565,1571 **** DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
--- 1582,1588 ----
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
***************
*** 2408,2413 **** SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
--- 2425,2443 ----
1104 | 204 | ddd |
(819 rows)
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c7 = DEFAULT WHERE c1 % 10 = 0 AND date(c4) = '1970-01-01'::date; -- can't be pushed down
+ QUERY PLAN
+ -----------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c7 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.ft2
+ Output: c1, c2, NULL::integer, c3, c4, c5, c6, 'ft2 '::character(10), c8, ctid
+ Filter: (date(ft2.c4) = '01-01-1970'::date)
+ Remote SQL: SELECT "C 1", c2, c3, c4, c5, c6, c8, ctid FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 0)) FOR UPDATE
+ (6 rows)
+
+ UPDATE ft2 SET c7 = DEFAULT WHERE c1 % 10 = 0 AND date(c4) = '1970-01-01'::date;
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
BEGIN
***************
*** 2554,2560 **** CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 2584,2590 ----
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 2713,2719 **** savepoint s3;
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 2743,2749 ----
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (-2) WHERE ((c2 = 42)) AND (("C 1" = 10))
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 2853,2859 **** CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2positive;
-- But inconsistent check constraints provide inconsistent results
ALTER FOREIGN TABLE ft1 ADD CONSTRAINT ft1_c2negative CHECK (c2 < 0);
--- 2883,2889 ----
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2positive;
-- But inconsistent check constraints provide inconsistent results
ALTER FOREIGN TABLE ft1 ADD CONSTRAINT ft1_c2negative CHECK (c2 < 0);
***************
*** 3246,3251 **** NOTICE: NEW: (13,"test triggered !")
--- 3276,3474 ----
(0,27)
(1 row)
+ -- cleanup
+ DROP TRIGGER trig_row_before ON rem1;
+ DROP TRIGGER trig_row_after ON rem1;
+ DROP TRIGGER trig_local_before ON loc1;
+ -- Test update-pushdown functionality
+ -- Test with statement-level triggers
+ CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_stmt_before ON rem1;
+ CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_stmt_after ON rem1;
+ -- Test with row-level ON INSERT triggers
+ CREATE TRIGGER trig_row_before_insert
+ BEFORE INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_before_insert ON rem1;
+ CREATE TRIGGER trig_row_after_insert
+ AFTER INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_after_insert ON rem1;
+ -- Test with row-level ON UPDATE triggers
+ CREATE TRIGGER trig_row_before_update
+ BEFORE UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_before_update ON rem1;
+ CREATE TRIGGER trig_row_after_update
+ AFTER UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+ -------------------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_after_update ON rem1;
+ -- Test with row-level ON DELETE triggers
+ CREATE TRIGGER trig_row_before_delete
+ BEFORE DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ DROP TRIGGER trig_row_before_delete ON rem1;
+ CREATE TRIGGER trig_row_after_delete
+ AFTER DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+ ------------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ DROP TRIGGER trig_row_after_delete ON rem1;
-- ===================================================================
-- test inheritance features
-- ===================================================================
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
***************
*** 57,63 **** PG_MODULE_MAGIC;
* planner to executor. Currently we store:
*
* 1) SELECT statement text to be sent to the remote server
! * 2) Integer list of attribute numbers retrieved by the SELECT
*
* These items are indexed with the enum FdwScanPrivateIndex, so an item
* can be fetched with list_nth(). For example, to get the SELECT statement:
--- 57,64 ----
* planner to executor. Currently we store:
*
* 1) SELECT statement text to be sent to the remote server
! * 2) List of restriction clauses that can be executed remotely
! * 3) Integer list of attribute numbers retrieved by the SELECT
*
* These items are indexed with the enum FdwScanPrivateIndex, so an item
* can be fetched with list_nth(). For example, to get the SELECT statement:
***************
*** 67,72 **** enum FdwScanPrivateIndex
--- 68,75 ----
{
/* SQL statement to execute remotely (as a String node) */
FdwScanPrivateSelectSql,
+ /* List of restriction clauses that can be executed remotely */
+ FdwScanPrivateRemoteConds,
/* Integer list of attribute numbers retrieved by the SELECT */
FdwScanPrivateRetrievedAttrs
};
***************
*** 94,99 **** enum FdwModifyPrivateIndex
--- 97,124 ----
};
/*
+ * Similarly, this enum describes what's kept in the fdw_private list for
+ * a ForeignScan node referencing a postgres_fdw foreign table when a DML
+ * query is pushed down to the remote server. We store:
+ *
+ * 1) UPDATE/DELETE statement text to be sent to the remote server
+ * 2) Boolean flag showing if the remote query has a RETURNING clause
+ * 3) Integer list of attribute numbers retrieved by RETURNING, if any
+ * 4) Boolean flag showing if we set the command es_processed
+ */
+ enum FdwDmlPushdownPrivateIndex
+ {
+ /* SQL statement to execute remotely (as a String node) */
+ FdwDmlPushdownPrivateUpdateSql,
+ /* has-returning flag (as an integer Value node) */
+ FdwDmlPushdownPrivateHasReturning,
+ /* Integer list of attribute numbers retrieved by RETURNING */
+ FdwDmlPushdownPrivateRetrievedAttrs,
+ /* set-processed flag (as an integer Value node) */
+ FdwDmlPushdownPrivateSetProcessed
+ };
+
+ /*
* Execution state of a foreign scan using postgres_fdw.
*/
typedef struct PgFdwScanState
***************
*** 156,161 **** typedef struct PgFdwModifyState
--- 181,217 ----
} PgFdwModifyState;
/*
+ * Execution state of a foreign scan using postgres_fdw.
+ */
+ typedef struct PgFdwDmlPushdownState
+ {
+ Relation rel; /* relcache entry for the foreign table */
+ AttInMetadata *attinmeta; /* attribute datatype conversion metadata */
+
+ /* extracted fdw_private data */
+ char *query; /* text of UPDATE/DELETE command */
+ List *retrieved_attrs; /* list of retrieved attribute numbers */
+ bool has_returning; /* is there a RETURNING clause? */
+ bool set_processed; /* do we set the command es_processed? */
+
+ /* for remote query execution */
+ PGconn *conn; /* connection for the scan */
+ int numParams; /* number of parameters passed to query */
+ FmgrInfo *param_flinfo; /* output conversion functions for them */
+ List *param_exprs; /* executable expressions for param values */
+ const char **param_values; /* textual values of query parameters */
+
+ /* for storing result tuples */
+ PGresult *result; /* result for query */
+ int num_tuples; /* # of tuples in array */
+ int next_tuple; /* index of next one to return */
+
+ /* working memory contexts */
+ MemoryContext batch_cxt; /* context holding current batch of tuples */
+ MemoryContext temp_cxt; /* context for per-tuple temporary data */
+ } PgFdwDmlPushdownState;
+
+ /*
* Workspace for analyzing a foreign table.
*/
typedef struct PgFdwAnalyzeState
***************
*** 246,251 **** static TupleTableSlot *postgresExecForeignDelete(EState *estate,
--- 302,314 ----
TupleTableSlot *planSlot);
static void postgresEndForeignModify(EState *estate,
ResultRelInfo *resultRelInfo);
+ static bool postgresPlanDMLPushdown(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+ static void postgresBeginDMLPushdown(ForeignScanState *node, int eflags);
+ static TupleTableSlot *postgresIterateDMLPushdown(ForeignScanState *node);
+ static void postgresEndDMLPushdown(ForeignScanState *node);
static int postgresIsForeignRelUpdatable(Relation rel);
static void postgresExplainForeignScan(ForeignScanState *node,
ExplainState *es);
***************
*** 254,259 **** static void postgresExplainForeignModify(ModifyTableState *mtstate,
--- 317,324 ----
List *fdw_private,
int subplan_index,
ExplainState *es);
+ static void postgresExplainDMLPushdown(ForeignScanState *node,
+ ExplainState *es);
static bool postgresAnalyzeForeignTable(Relation relation,
AcquireSampleRowsFunc *func,
BlockNumber *totalpages);
***************
*** 288,295 **** static void prepare_foreign_modify(PgFdwModifyState *fmstate);
static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
ItemPointer tupleid,
TupleTableSlot *slot);
! static void store_returning_result(PgFdwModifyState *fmstate,
! TupleTableSlot *slot, PGresult *res);
static int postgresAcquireSampleRowsFunc(Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
--- 353,374 ----
static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
ItemPointer tupleid,
TupleTableSlot *slot);
! static void store_returning_result(TupleTableSlot *slot,
! PGresult *res,
! int row,
! Relation rel,
! AttInMetadata *attinmeta,
! List *retrieved_attrs,
! MemoryContext temp_context);
! static bool dml_is_pushdown_safe(PlannerInfo *root,
! ModifyTable *plan,
! Index resultRelation,
! int subplan_index,
! Relation rel,
! List *targetAttrs);
! static void execute_dml_stmt(ForeignScanState *node);
! static bool relation_has_row_level_triggers(Relation rel, CmdType operation);
! static TupleTableSlot *get_returning_result(ForeignScanState *node);
static int postgresAcquireSampleRowsFunc(Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
***************
*** 331,341 **** postgres_fdw_handler(PG_FUNCTION_ARGS)
--- 410,425 ----
routine->ExecForeignUpdate = postgresExecForeignUpdate;
routine->ExecForeignDelete = postgresExecForeignDelete;
routine->EndForeignModify = postgresEndForeignModify;
+ routine->PlanDMLPushdown = postgresPlanDMLPushdown;
+ routine->BeginDMLPushdown = postgresBeginDMLPushdown;
+ routine->IterateDMLPushdown = postgresIterateDMLPushdown;
+ routine->EndDMLPushdown = postgresEndDMLPushdown;
routine->IsForeignRelUpdatable = postgresIsForeignRelUpdatable;
/* Support functions for EXPLAIN */
routine->ExplainForeignScan = postgresExplainForeignScan;
routine->ExplainForeignModify = postgresExplainForeignModify;
+ routine->ExplainDMLPushdown = postgresExplainDMLPushdown;
/* Support functions for ANALYZE */
routine->AnalyzeForeignTable = postgresAnalyzeForeignTable;
***************
*** 1067,1073 **** postgresGetForeignPlan(PlannerInfo *root,
* Build the fdw_private list that will be available to the executor.
* Items in the list must match enum FdwScanPrivateIndex, above.
*/
! fdw_private = list_make2(makeString(sql.data),
retrieved_attrs);
/*
--- 1151,1158 ----
* Build the fdw_private list that will be available to the executor.
* Items in the list must match enum FdwScanPrivateIndex, above.
*/
! fdw_private = list_make3(makeString(sql.data),
! remote_conds,
retrieved_attrs);
/*
***************
*** 1363,1375 **** postgresAddForeignUpdateTargets(Query *parsetree,
/*
* postgresPlanForeignModify
* Plan an insert/update/delete operation on a foreign table
- *
- * Note: currently, the plan tree generated for UPDATE/DELETE will always
- * include a ForeignScan that retrieves ctids (using SELECT FOR UPDATE)
- * and then the ModifyTable node will have to execute individual remote
- * UPDATE/DELETE commands. If there are no local conditions or joins
- * needed, it'd be better to let the scan node do UPDATE/DELETE RETURNING
- * and then do nothing at ModifyTable. Room for future optimization ...
*/
static List *
postgresPlanForeignModify(PlannerInfo *root,
--- 1448,1453 ----
***************
*** 1644,1650 **** postgresExecForeignInsert(EState *estate,
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(fmstate, slot, res);
}
else
n_rows = atoi(PQcmdTuples(res));
--- 1722,1732 ----
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(slot, res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
}
else
n_rows = atoi(PQcmdTuples(res));
***************
*** 1714,1720 **** postgresExecForeignUpdate(EState *estate,
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(fmstate, slot, res);
}
else
n_rows = atoi(PQcmdTuples(res));
--- 1796,1806 ----
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(slot, res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
}
else
n_rows = atoi(PQcmdTuples(res));
***************
*** 1784,1790 **** postgresExecForeignDelete(EState *estate,
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(fmstate, slot, res);
}
else
n_rows = atoi(PQcmdTuples(res));
--- 1870,1880 ----
{
n_rows = PQntuples(res);
if (n_rows > 0)
! store_returning_result(slot, res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
}
else
n_rows = atoi(PQcmdTuples(res));
***************
*** 1837,1842 **** postgresEndForeignModify(EState *estate,
--- 1927,2242 ----
}
/*
+ * postgresPlanDMLPushdown
+ * Plan to push down an update/delete operation on a foreign table
+ */
+ static bool
+ postgresPlanDMLPushdown(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index)
+ {
+ CmdType operation = plan->operation;
+ RangeTblEntry *rte = planner_rt_fetch(resultRelation, root);
+ Relation rel;
+ StringInfoData sql;
+ List *targetAttrs = NIL;
+ List *params_list = NIL;
+ List *returningList = NIL;
+ List *retrieved_attrs = NIL;
+ List *remote_conds;
+ ForeignScan *fscan;
+
+ if (operation == CMD_INSERT)
+ return false;
+
+ /*
+ * Core code already has some lock on each rel being planned, so we can
+ * use NoLock here.
+ */
+ rel = heap_open(rte->relid, NoLock);
+
+ /*
+ * In an UPDATE, we transmit only columns that were explicitly targets
+ * of the UPDATE, so as to avoid unnecessary data transmission.
+ */
+ if (operation == CMD_UPDATE)
+ {
+ int col;
+
+ col = -1;
+ while ((col = bms_next_member(rte->updatedCols, col)) >= 0)
+ {
+ /* bit numbers are offset by FirstLowInvalidHeapAttributeNumber */
+ AttrNumber attno = col + FirstLowInvalidHeapAttributeNumber;
+
+ if (attno <= InvalidAttrNumber) /* shouldn't happen */
+ elog(ERROR, "system-column update is not supported");
+ targetAttrs = lappend_int(targetAttrs, attno);
+ }
+ }
+
+ /*
+ * Ok, check to see whether it's safe to push the command down.
+ */
+ if (!dml_is_pushdown_safe(root, plan,
+ resultRelation,
+ subplan_index,
+ rel, targetAttrs))
+ {
+ heap_close(rel, NoLock);
+ return false;
+ }
+
+ /*
+ * Ok, modify subplan so as to execute the pushed-down command.
+ */
+ fscan = (ForeignScan *) list_nth(plan->plans, subplan_index);
+
+ /*
+ * Extract the baserestrictinfo clauses that can be evaluated remotely.
+ */
+ remote_conds = (List *) list_nth(fscan->fdw_private,
+ FdwScanPrivateRemoteConds);
+
+ /*
+ * Extract the relevant RETURNING list if any.
+ */
+ if (plan->returningLists)
+ returningList = (List *) list_nth(plan->returningLists, subplan_index);
+
+ /*
+ * Construct the SQL command string.
+ */
+ initStringInfo(&sql);
+ switch (operation)
+ {
+ case CMD_UPDATE:
+ deparsePushedDownUpdateSql(&sql, root, resultRelation, rel,
+ ((Plan *) fscan)->targetlist,
+ targetAttrs,
+ remote_conds, ¶ms_list,
+ returningList, &retrieved_attrs);
+ break;
+ case CMD_DELETE:
+ deparsePushedDownDeleteSql(&sql, root, resultRelation, rel,
+ remote_conds, ¶ms_list,
+ returningList, &retrieved_attrs);
+ break;
+ default:
+ elog(ERROR, "unexpected operation: %d", (int) operation);
+ break;
+ }
+
+ /*
+ * Update the operation info.
+ */
+ fscan->operation = operation;
+
+ /*
+ * Update the fdw_exprs list that will be available to the executor.
+ */
+ fscan->fdw_exprs = params_list;
+
+ /*
+ * Update the fdw_private list that will be available to the executor.
+ * Items in the list must match enum FdwDmlPushdownPrivateIndex, above.
+ */
+ fscan->fdw_private = list_make4(makeString(sql.data),
+ makeInteger((retrieved_attrs != NIL)),
+ retrieved_attrs,
+ makeInteger(plan->canSetTag));
+
+ heap_close(rel, NoLock);
+ return true;
+ }
+
+ /*
+ * postgresBeginDMLPushdown
+ * Initiate a pushed-down update/delete operation on a foreign table
+ */
+ static void
+ postgresBeginDMLPushdown(ForeignScanState *node, int eflags)
+ {
+ ForeignScan *fsplan = (ForeignScan *) node->ss.ps.plan;
+ EState *estate = node->ss.ps.state;
+ PgFdwDmlPushdownState *dpstate;
+ RangeTblEntry *rte;
+ Oid userid;
+ ForeignTable *table;
+ ForeignServer *server;
+ UserMapping *user;
+ int numParams;
+ int i;
+ ListCell *lc;
+
+ /*
+ * Do nothing in EXPLAIN (no ANALYZE) case. node->fdw_state stays NULL.
+ */
+ if (eflags & EXEC_FLAG_EXPLAIN_ONLY)
+ return;
+
+ /*
+ * We'll save private state in node->fdw_state.
+ */
+ dpstate = (PgFdwDmlPushdownState *) palloc0(sizeof(PgFdwDmlPushdownState));
+ node->fdw_state = (void *) dpstate;
+
+ /*
+ * Identify which user to do the remote access as. This should match what
+ * ExecCheckRTEPerms() does.
+ */
+ rte = rt_fetch(fsplan->scan.scanrelid, estate->es_range_table);
+ userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
+
+ /* Get info about foreign table. */
+ dpstate->rel = node->ss.ss_currentRelation;
+ table = GetForeignTable(RelationGetRelid(dpstate->rel));
+ server = GetForeignServer(table->serverid);
+ user = GetUserMapping(userid, server->serverid);
+
+ /*
+ * Get connection to the foreign server. Connection manager will
+ * establish new connection if necessary.
+ */
+ dpstate->conn = GetConnection(server, user, false);
+
+ /* Initialize state variable */
+ dpstate->num_tuples = -1; /* -1 means not set yet */
+
+ /* Get private info created by planner functions. */
+ dpstate->query = strVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateUpdateSql));
+ dpstate->has_returning = intVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateHasReturning));
+ dpstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateRetrievedAttrs);
+ dpstate->set_processed = intVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateSetProcessed));
+
+ /* Create contexts for batches of tuples and per-tuple temp workspace. */
+ dpstate->batch_cxt = AllocSetContextCreate(estate->es_query_cxt,
+ "postgres_fdw tuple data",
+ ALLOCSET_DEFAULT_MINSIZE,
+ ALLOCSET_DEFAULT_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+ dpstate->temp_cxt = AllocSetContextCreate(estate->es_query_cxt,
+ "postgres_fdw temporary data",
+ ALLOCSET_SMALL_MINSIZE,
+ ALLOCSET_SMALL_INITSIZE,
+ ALLOCSET_SMALL_MAXSIZE);
+
+ /* Prepare for input conversion of RETURNING results. */
+ if (dpstate->has_returning)
+ dpstate->attinmeta = TupleDescGetAttInMetadata(RelationGetDescr(dpstate->rel));
+
+ /* Prepare for output conversion of parameters used in remote query. */
+ numParams = list_length(fsplan->fdw_exprs);
+ dpstate->numParams = numParams;
+ dpstate->param_flinfo = (FmgrInfo *) palloc0(sizeof(FmgrInfo) * numParams);
+
+ i = 0;
+ foreach(lc, fsplan->fdw_exprs)
+ {
+ Node *param_expr = (Node *) lfirst(lc);
+ Oid typefnoid;
+ bool isvarlena;
+
+ getTypeOutputInfo(exprType(param_expr), &typefnoid, &isvarlena);
+ fmgr_info(typefnoid, &dpstate->param_flinfo[i]);
+ i++;
+ }
+
+ /*
+ * Prepare remote-parameter expressions for evaluation. (Note: in
+ * practice, we expect that all these expressions will be just Params, so
+ * we could possibly do something more efficient than using the full
+ * expression-eval machinery for this. But probably there would be little
+ * benefit, and it'd require postgres_fdw to know more than is desirable
+ * about Param evaluation.)
+ */
+ dpstate->param_exprs = (List *)
+ ExecInitExpr((Expr *) fsplan->fdw_exprs,
+ (PlanState *) node);
+
+ /*
+ * Allocate buffer for text form of query parameters, if any.
+ */
+ if (numParams > 0)
+ dpstate->param_values = (const char **) palloc0(numParams * sizeof(char *));
+ else
+ dpstate->param_values = NULL;
+ }
+
+ /*
+ * postgresIterateDMLPushdown
+ * Get the result of a pushed-down update/delete operation on a foreign table
+ */
+ static TupleTableSlot *
+ postgresIterateDMLPushdown(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ EState *estate = node->ss.ps.state;
+ ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
+
+ /*
+ * If this is the first call after Begin, execute the statement.
+ */
+ if (dpstate->num_tuples == -1)
+ execute_dml_stmt(node);
+
+ /*
+ * If the given query doesn't specify RETURNING, just return an empty slot.
+ */
+ if (!resultRelInfo->ri_projectReturning)
+ {
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+ Instrumentation *instr = node->ss.ps.instrument;
+
+ Assert(!dpstate->has_returning);
+
+ /* Increment the command es_processed count if necessary. */
+ if (dpstate->set_processed)
+ estate->es_processed += dpstate->num_tuples;
+
+ /* Increment the tuple count for EXPLAIN ANALYZE if necessary. */
+ if (node->ss.ps.instrument)
+ instr->ntuples += dpstate->num_tuples;
+
+ return ExecClearTuple(slot);
+ }
+
+ /*
+ * Get the next returning result.
+ */
+ return get_returning_result(node);
+ }
+
+ /*
+ * postgresEndDMLPushdown
+ * Finish a pushed-down update/delete operation on a foreign table
+ */
+ static void
+ postgresEndDMLPushdown(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+
+ /* if dpstate is NULL, we are in EXPLAIN; nothing to do */
+ if (dpstate == NULL)
+ return;
+
+ /* Clean up */
+ if (dpstate->result)
+ PQclear(dpstate->result);
+
+ /* Release remote connection */
+ ReleaseConnection(dpstate->conn);
+ dpstate->conn = NULL;
+
+ /* MemoryContexts will be deleted automatically. */
+ }
+
+ /*
* postgresIsForeignRelUpdatable
* Determine whether a foreign table supports INSERT, UPDATE and/or
* DELETE.
***************
*** 1919,1924 **** postgresExplainForeignModify(ModifyTableState *mtstate,
--- 2319,2343 ----
}
}
+ /*
+ * postgresExplainDMLPushdown
+ * Produce extra output for EXPLAIN of a ForeignScan for a pushed-down
+ * update/delete operation on a foreign table
+ */
+ static void
+ postgresExplainDMLPushdown(ForeignScanState *node, ExplainState *es)
+ {
+ List *fdw_private;
+ char *sql;
+
+ if (es->verbose)
+ {
+ fdw_private = ((ForeignScan *) node->ss.ps.plan)->fdw_private;
+ sql = strVal(list_nth(fdw_private, FdwDmlPushdownPrivateUpdateSql));
+ ExplainPropertyText("Remote SQL", sql, es);
+ }
+ }
+
/*
* estimate_path_cost_size
***************
*** 2510,2527 **** convert_prep_stmt_params(PgFdwModifyState *fmstate,
* have PG_TRY blocks to ensure this happens.
*/
static void
! store_returning_result(PgFdwModifyState *fmstate,
! TupleTableSlot *slot, PGresult *res)
{
PG_TRY();
{
HeapTuple newtup;
! newtup = make_tuple_from_result_row(res, 0,
! fmstate->rel,
! fmstate->attinmeta,
! fmstate->retrieved_attrs,
! fmstate->temp_cxt);
/* tuple will be deleted when it is cleared from the slot */
ExecStoreTuple(newtup, slot, InvalidBuffer, true);
}
--- 2929,2951 ----
* have PG_TRY blocks to ensure this happens.
*/
static void
! store_returning_result(TupleTableSlot *slot,
! PGresult *res,
! int row,
! Relation rel,
! AttInMetadata *attinmeta,
! List *retrieved_attrs,
! MemoryContext temp_context)
{
PG_TRY();
{
HeapTuple newtup;
! newtup = make_tuple_from_result_row(res, row,
! rel,
! attinmeta,
! retrieved_attrs,
! temp_context);
/* tuple will be deleted when it is cleared from the slot */
ExecStoreTuple(newtup, slot, InvalidBuffer, true);
}
***************
*** 2535,2540 **** store_returning_result(PgFdwModifyState *fmstate,
--- 2959,3183 ----
}
/*
+ * Check to see whether it's safe to push an UPDATE/DELETE command down.
+ *
+ * Conditions checked here:
+ *
+ * 1. If there are any local joins needed, we mustn't push the command down,
+ * because that breaks execution of the joins.
+ *
+ * 2. If there are any quals that can't be evaluated remotely, we mustn't push
+ * the command down, because that breaks evaluation of the quals.
+ *
+ * 3. If the target relation has any row-level local triggers, we mustn't push
+ * the command down, because that breaks execution of the triggers.
+ *
+ * 4. For UPDATE, if it is unsafe to evaluate on the remote server any
+ * expressions to assign to the target columns, we can't push the command down.
+ */
+ static bool
+ dml_is_pushdown_safe(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs)
+ {
+ CmdType operation = plan->operation;
+ RelOptInfo *baserel = root->simple_rel_array[resultRelation];
+ Plan *subplan = (Plan *) list_nth(plan->plans, subplan_index);
+ ListCell *lc;
+
+ Assert(operation == CMD_UPDATE || operation == CMD_DELETE);
+
+ /* Check point 1 */
+ if (nodeTag(subplan) != T_ForeignScan)
+ return false;
+
+ /* Check point 2 */
+ if (subplan->qual != NIL)
+ return false;
+
+ /* Check point 3 */
+ if (relation_has_row_level_triggers(rel, operation))
+ return false;
+
+ /* Check point 4 */
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(subplan->targetlist,
+ attnum);
+
+ if (!is_foreign_expr(root, baserel, (Expr *) tle->expr))
+ return false;
+ }
+
+ return true;
+ }
+
+ /*
+ * relation_has_row_level_triggers
+ * Determine if relation has any row-level triggers for a given operation.
+ */
+ static bool
+ relation_has_row_level_triggers(Relation rel, CmdType operation)
+ {
+ bool result = false;
+
+ Assert(rel != NULL);
+
+ if (rel->trigdesc == NULL)
+ return result;
+
+ switch (operation)
+ {
+ case CMD_INSERT:
+ result = (rel->trigdesc->trig_insert_after_row ||
+ rel->trigdesc->trig_insert_before_row);
+ break;
+ case CMD_UPDATE:
+ result = (rel->trigdesc->trig_update_after_row ||
+ rel->trigdesc->trig_update_before_row);
+ break;
+ case CMD_DELETE:
+ result = (rel->trigdesc->trig_delete_after_row ||
+ rel->trigdesc->trig_delete_before_row);
+ break;
+ default:
+ elog(ERROR, "unexpected operation: %d", (int) operation);
+ break;
+ }
+ return result;
+ }
+
+ /*
+ * Execute a pushed-down UPDATE/DELETE statement.
+ */
+ static void
+ execute_dml_stmt(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ ExprContext *econtext = node->ss.ps.ps_ExprContext;
+ int numParams = dpstate->numParams;
+ const char **values = dpstate->param_values;
+
+ /*
+ * Construct array of query parameter values in text format. We do the
+ * conversions in the short-lived per-tuple context, so as not to cause a
+ * memory leak over repeated scans.
+ */
+ if (numParams > 0)
+ {
+ int nestlevel;
+ MemoryContext oldcontext;
+ int i;
+ ListCell *lc;
+
+ oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
+
+ nestlevel = set_transmission_modes();
+
+ i = 0;
+ foreach(lc, dpstate->param_exprs)
+ {
+ ExprState *expr_state = (ExprState *) lfirst(lc);
+ Datum expr_value;
+ bool isNull;
+
+ /* Evaluate the parameter expression */
+ expr_value = ExecEvalExpr(expr_state, econtext, &isNull, NULL);
+
+ /*
+ * Get string representation of each parameter value by invoking
+ * type-specific output function, unless the value is null.
+ */
+ if (isNull)
+ values[i] = NULL;
+ else
+ values[i] = OutputFunctionCall(&dpstate->param_flinfo[i],
+ expr_value);
+ i++;
+ }
+
+ reset_transmission_modes(nestlevel);
+
+ MemoryContextSwitchTo(oldcontext);
+ }
+
+ /*
+ * Notice that we pass NULL for paramTypes, thus forcing the remote server
+ * to infer types for all parameters. Since we explicitly cast every
+ * parameter (see deparse.c), the "inference" is trivial and will produce
+ * the desired result. This allows us to avoid assuming that the remote
+ * server has the same OIDs we do for the parameters' types.
+ *
+ * We don't use a PG_TRY block here, so be careful not to throw error
+ * without releasing the PGresult.
+ */
+ dpstate->result = PQexecParams(dpstate->conn, dpstate->query,
+ numParams, NULL, values, NULL, NULL, 0);
+ if (PQresultStatus(dpstate->result) !=
+ (dpstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
+ pgfdw_report_error(ERROR, dpstate->result, dpstate->conn, true,
+ dpstate->query);
+
+ /* Get the number of rows affected. */
+ if (dpstate->has_returning)
+ dpstate->num_tuples = PQntuples(dpstate->result);
+ else
+ dpstate->num_tuples = atoi(PQcmdTuples(dpstate->result));
+ }
+
+ /*
+ * Get the result of an UPDATE/DELETE RETURNING.
+ */
+ static TupleTableSlot *
+ get_returning_result(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+ EState *estate = node->ss.ps.state;
+ ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
+ MemoryContext oldcontext;
+
+ Assert(resultRelInfo->ri_projectReturning);
+
+ /* If we didn't get any tuples, must be end of data. */
+ if (dpstate->next_tuple >= dpstate->num_tuples)
+ return ExecClearTuple(slot);
+
+ /* Increment the command es_processed count if necessary. */
+ if (dpstate->set_processed)
+ estate->es_processed += 1;
+
+ /* OK, we'll store RETURNING tuples. */
+ if (!dpstate->has_returning)
+ ExecStoreAllNullTuple(slot);
+ else
+ {
+ oldcontext = MemoryContextSwitchTo(dpstate->batch_cxt);
+
+ /* Fetch the next tuple. */
+ store_returning_result(slot,
+ dpstate->result,
+ dpstate->next_tuple,
+ dpstate->rel,
+ dpstate->attinmeta,
+ dpstate->retrieved_attrs,
+ dpstate->temp_cxt);
+
+ MemoryContextSwitchTo(oldcontext);
+ }
+ dpstate->next_tuple++;
+
+ /* Set tuple for later evaluation of the RETURNING list. */
+ resultRelInfo->ri_projectReturning->pi_exprContext->ecxt_scantuple = slot;
+
+ return slot;
+ }
+
+ /*
* postgresAnalyzeForeignTable
* Test whether analyzing this foreign table is supported
*/
*** a/contrib/postgres_fdw/postgres_fdw.h
--- b/contrib/postgres_fdw/postgres_fdw.h
***************
*** 103,112 **** extern void deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 103,126 ----
Index rtindex, Relation rel,
List *targetAttrs, List *returningList,
List **retrieved_attrs);
+ extern void deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseDeleteSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *returningList,
List **retrieved_attrs);
+ extern void deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseAnalyzeSizeSql(StringInfo buf, Relation rel);
extern void deparseAnalyzeSql(StringInfo buf, Relation rel,
List **retrieved_attrs);
*** a/contrib/postgres_fdw/sql/postgres_fdw.sql
--- b/contrib/postgres_fdw/sql/postgres_fdw.sql
***************
*** 399,418 **** INSERT INTO ft2 (c1,c2,c3) SELECT c1+1000,c2+100, c3 || c3 FROM ft2 LIMIT 20;
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
--- 399,425 ----
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c7 = DEFAULT WHERE c1 % 10 = 0 AND date(c4) = '1970-01-01'::date; -- can't be pushed down
+ UPDATE ft2 SET c7 = DEFAULT WHERE c1 % 10 = 0 AND date(c4) = '1970-01-01'::date;
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
***************
*** 728,733 **** UPDATE rem1 SET f2 = 'testo';
--- 735,824 ----
-- Test returning a system attribute
INSERT INTO rem1(f2) VALUES ('test') RETURNING ctid;
+ -- cleanup
+ DROP TRIGGER trig_row_before ON rem1;
+ DROP TRIGGER trig_row_after ON rem1;
+ DROP TRIGGER trig_local_before ON loc1;
+
+
+ -- Test update-pushdown functionality
+
+ -- Test with statement-level triggers
+ CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_stmt_before ON rem1;
+
+ CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_stmt_after ON rem1;
+
+ -- Test with row-level ON INSERT triggers
+ CREATE TRIGGER trig_row_before_insert
+ BEFORE INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_before_insert ON rem1;
+
+ CREATE TRIGGER trig_row_after_insert
+ AFTER INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_after_insert ON rem1;
+
+ -- Test with row-level ON UPDATE triggers
+ CREATE TRIGGER trig_row_before_update
+ BEFORE UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_before_update ON rem1;
+
+ CREATE TRIGGER trig_row_after_update
+ AFTER UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_after_update ON rem1;
+
+ -- Test with row-level ON DELETE triggers
+ CREATE TRIGGER trig_row_before_delete
+ BEFORE DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ DROP TRIGGER trig_row_before_delete ON rem1;
+
+ CREATE TRIGGER trig_row_after_delete
+ AFTER DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ DROP TRIGGER trig_row_after_delete ON rem1;
+
-- ===================================================================
-- test inheritance features
-- ===================================================================
*** a/doc/src/sgml/postgres-fdw.sgml
--- b/doc/src/sgml/postgres-fdw.sgml
***************
*** 471,476 ****
--- 471,485 ----
extension that's listed in the foreign server's <literal>extensions</>
option. Operators and functions in such clauses must
be <literal>IMMUTABLE</> as well.
+ For an <command>UPDATE</> or <command>DELETE</> query,
+ <filename>postgres_fdw</> attempts to optimize the query execution by
+ sending the whole query to the remote server if there are no query
+ <literal>WHERE</> clauses that cannot be sent to the remote server,
+ no local joins for the query, or no row-level local <literal>BEFORE</> or
+ <literal>AFTER</> triggers on the target table. In <command>UPDATE</>,
+ expressions to assign to target columns must use only built-in data types,
+ <literal>IMMUTABLE</> operators, and <literal>IMMUTABLE</> functions,
+ to reduce the risk of misexecution of the query.
</para>
<para>
*** a/src/backend/commands/explain.c
--- b/src/backend/commands/explain.c
***************
*** 888,894 **** ExplainNode(PlanState *planstate, List *ancestors,
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
! pname = sname = "Foreign Scan";
break;
case T_CustomScan:
sname = "Custom Scan";
--- 888,912 ----
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
! sname = "Foreign Scan";
! switch (((ForeignScan *) plan)->operation)
! {
! case CMD_SELECT:
! pname = "Foreign Scan";
! operation = "Select";
! break;
! case CMD_UPDATE:
! pname = "Foreign Update";
! operation = "Update";
! break;
! case CMD_DELETE:
! pname = "Foreign Delete";
! operation = "Delete";
! break;
! default:
! pname = "???";
! break;
! }
break;
case T_CustomScan:
sname = "Custom Scan";
***************
*** 1624,1629 **** show_plan_tlist(PlanState *planstate, List *ancestors, ExplainState *es)
--- 1642,1652 ----
return;
if (IsA(plan, RecursiveUnion))
return;
+ /* Likewise for ForeignScan in case of pushed-down UPDATE/DELETE */
+ if (IsA(plan, ForeignScan) &&
+ (((ForeignScan *) plan)->operation == CMD_UPDATE ||
+ ((ForeignScan *) plan)->operation == CMD_DELETE))
+ return;
/* Set up deparsing context */
context = set_deparse_context_planstate(es->deparse_cxt,
***************
*** 2212,2219 **** show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es)
FdwRoutine *fdwroutine = fsstate->fdwroutine;
/* Let the FDW emit whatever fields it wants */
! if (fdwroutine->ExplainForeignScan != NULL)
! fdwroutine->ExplainForeignScan(fsstate, es);
}
/*
--- 2235,2250 ----
FdwRoutine *fdwroutine = fsstate->fdwroutine;
/* Let the FDW emit whatever fields it wants */
! if (((ForeignScan *) fsstate->ss.ps.plan)->operation != CMD_SELECT)
! {
! if (fdwroutine->ExplainDMLPushdown != NULL)
! fdwroutine->ExplainDMLPushdown(fsstate, es);
! }
! else
! {
! if (fdwroutine->ExplainForeignScan != NULL)
! fdwroutine->ExplainForeignScan(fsstate, es);
! }
}
/*
***************
*** 2599,2606 **** show_modifytable_info(ModifyTableState *mtstate, List *ancestors,
}
}
! /* Give FDW a chance */
! if (fdwroutine && fdwroutine->ExplainForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, j);
--- 2630,2639 ----
}
}
! /* Give FDW a chance if needed */
! if (!resultRelInfo->ri_FdwPushdown &&
! fdwroutine &&
! fdwroutine->ExplainForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, j);
*** a/src/backend/executor/execMain.c
--- b/src/backend/executor/execMain.c
***************
*** 1011,1020 **** InitPlan(QueryDesc *queryDesc, int eflags)
* CheckValidRowMarkRel.
*/
void
! CheckValidResultRel(Relation resultRel, CmdType operation)
{
TriggerDesc *trigDesc = resultRel->trigdesc;
FdwRoutine *fdwroutine;
switch (resultRel->rd_rel->relkind)
{
--- 1011,1022 ----
* CheckValidRowMarkRel.
*/
void
! CheckValidResultRel(ResultRelInfo *resultRelInfo, CmdType operation)
{
+ Relation resultRel = resultRelInfo->ri_RelationDesc;
TriggerDesc *trigDesc = resultRel->trigdesc;
FdwRoutine *fdwroutine;
+ bool allow_pushdown;
switch (resultRel->rd_rel->relkind)
{
***************
*** 1083,1096 **** CheckValidResultRel(Relation resultRel, CmdType operation)
case RELKIND_FOREIGN_TABLE:
/* Okay only if the FDW supports it */
fdwroutine = GetFdwRoutineForRelation(resultRel, false);
switch (operation)
{
case CMD_INSERT:
! if (fdwroutine->ExecForeignInsert == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("cannot insert into foreign table \"%s\"",
! RelationGetRelationName(resultRel))));
if (fdwroutine->IsForeignRelUpdatable != NULL &&
(fdwroutine->IsForeignRelUpdatable(resultRel) & (1 << CMD_INSERT)) == 0)
ereport(ERROR,
--- 1085,1107 ----
case RELKIND_FOREIGN_TABLE:
/* Okay only if the FDW supports it */
fdwroutine = GetFdwRoutineForRelation(resultRel, false);
+ allow_pushdown = ((fdwroutine->BeginDMLPushdown != NULL) &&
+ (fdwroutine->IterateDMLPushdown != NULL) &&
+ (fdwroutine->EndDMLPushdown != NULL));
switch (operation)
{
case CMD_INSERT:
! if (resultRelInfo->ri_FdwPushdown && !allow_pushdown)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("cannot push down insert on foreign table \"%s\"",
! RelationGetRelationName(resultRel))));
! if (!resultRelInfo->ri_FdwPushdown &&
! fdwroutine->ExecForeignInsert == NULL)
! ereport(ERROR,
! (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("cannot insert into foreign table \"%s\"",
! RelationGetRelationName(resultRel))));
if (fdwroutine->IsForeignRelUpdatable != NULL &&
(fdwroutine->IsForeignRelUpdatable(resultRel) & (1 << CMD_INSERT)) == 0)
ereport(ERROR,
***************
*** 1099,1105 **** CheckValidResultRel(Relation resultRel, CmdType operation)
RelationGetRelationName(resultRel))));
break;
case CMD_UPDATE:
! if (fdwroutine->ExecForeignUpdate == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot update foreign table \"%s\"",
--- 1110,1122 ----
RelationGetRelationName(resultRel))));
break;
case CMD_UPDATE:
! if (resultRelInfo->ri_FdwPushdown && !allow_pushdown)
! ereport(ERROR,
! (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("cannot push down update on foreign table \"%s\"",
! RelationGetRelationName(resultRel))));
! if (!resultRelInfo->ri_FdwPushdown &&
! fdwroutine->ExecForeignUpdate == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot update foreign table \"%s\"",
***************
*** 1112,1118 **** CheckValidResultRel(Relation resultRel, CmdType operation)
RelationGetRelationName(resultRel))));
break;
case CMD_DELETE:
! if (fdwroutine->ExecForeignDelete == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot delete from foreign table \"%s\"",
--- 1129,1141 ----
RelationGetRelationName(resultRel))));
break;
case CMD_DELETE:
! if (resultRelInfo->ri_FdwPushdown && !allow_pushdown)
! ereport(ERROR,
! (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("cannot push down delete on foreign table \"%s\"",
! RelationGetRelationName(resultRel))));
! if (!resultRelInfo->ri_FdwPushdown &&
! fdwroutine->ExecForeignDelete == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot delete from foreign table \"%s\"",
***************
*** 1245,1250 **** InitResultRelInfo(ResultRelInfo *resultRelInfo,
--- 1268,1274 ----
else
resultRelInfo->ri_FdwRoutine = NULL;
resultRelInfo->ri_FdwState = NULL;
+ resultRelInfo->ri_FdwPushdown = false;
resultRelInfo->ri_ConstraintExprs = NULL;
resultRelInfo->ri_junkFilter = NULL;
resultRelInfo->ri_projectReturning = NULL;
*** a/src/backend/executor/nodeForeignscan.c
--- b/src/backend/executor/nodeForeignscan.c
***************
*** 48,54 **** ForeignNext(ForeignScanState *node)
/* Call the Iterate function in short-lived context */
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! slot = node->fdwroutine->IterateForeignScan(node);
MemoryContextSwitchTo(oldcontext);
/*
--- 48,57 ----
/* Call the Iterate function in short-lived context */
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! if (plan->operation != CMD_SELECT)
! slot = node->fdwroutine->IterateDMLPushdown(node);
! else
! slot = node->fdwroutine->IterateForeignScan(node);
MemoryContextSwitchTo(oldcontext);
/*
***************
*** 226,232 **** ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
/*
* Tell the FDW to initialize the scan.
*/
! fdwroutine->BeginForeignScan(scanstate, eflags);
return scanstate;
}
--- 229,238 ----
/*
* Tell the FDW to initialize the scan.
*/
! if (node->operation != CMD_SELECT)
! fdwroutine->BeginDMLPushdown(scanstate, eflags);
! else
! fdwroutine->BeginForeignScan(scanstate, eflags);
return scanstate;
}
***************
*** 240,247 **** ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
void
ExecEndForeignScan(ForeignScanState *node)
{
/* Let the FDW shut down */
! node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
--- 246,258 ----
void
ExecEndForeignScan(ForeignScanState *node)
{
+ ForeignScan *plan = (ForeignScan *) node->ss.ps.plan;
+
/* Let the FDW shut down */
! if (plan->operation != CMD_SELECT)
! node->fdwroutine->EndDMLPushdown(node);
! else
! node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
*** a/src/backend/executor/nodeModifyTable.c
--- b/src/backend/executor/nodeModifyTable.c
***************
*** 1357,1362 **** ExecModifyTable(ModifyTableState *node)
--- 1357,1383 ----
break;
}
+ if (resultRelInfo->ri_FdwPushdown && resultRelInfo->ri_projectReturning)
+ {
+ ProjectionInfo *projectReturning = resultRelInfo->ri_projectReturning;
+ ExprContext *econtext = projectReturning->pi_exprContext;
+
+ /*
+ * Reset per-tuple memory context to free any expression evaluation
+ * storage allocated in the previous cycle.
+ */
+ ResetExprContext(econtext);
+
+ /* Make any needed join variables available to ExecProject */
+ econtext->ecxt_outertuple = planSlot;
+
+ /* Compute the RETURNING expressions */
+ slot = ExecProject(resultRelInfo->ri_projectReturning, NULL);
+
+ estate->es_result_relation_info = saved_resultRelInfo;
+ return slot;
+ }
+
EvalPlanQualSetSlot(&node->mt_epqstate, planSlot);
slot = planSlot;
***************
*** 1536,1545 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
{
subplan = (Plan *) lfirst(l);
/*
* Verify result relation is a valid target for the current operation
*/
! CheckValidResultRel(resultRelInfo->ri_RelationDesc, operation);
/*
* If there are indices on the result relation, open them and save
--- 1557,1569 ----
{
subplan = (Plan *) lfirst(l);
+ /* Initialize the FdwPushdown flag */
+ resultRelInfo->ri_FdwPushdown = list_nth_int(node->fdwPushdowns, i);
+
/*
* Verify result relation is a valid target for the current operation
*/
! CheckValidResultRel(resultRelInfo, operation);
/*
* If there are indices on the result relation, open them and save
***************
*** 1560,1566 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
/* Also let FDWs init themselves for foreign-table result rels */
! if (resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, i);
--- 1584,1591 ----
mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
/* Also let FDWs init themselves for foreign-table result rels */
! if (!resultRelInfo->ri_FdwPushdown &&
! resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, i);
***************
*** 1731,1743 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
--- 1756,1777 ----
erm = ExecFindRowMark(estate, rc->rti, false);
/* build ExecAuxRowMark for each subplan */
+ resultRelInfo = mtstate->resultRelInfo;
for (i = 0; i < nplans; i++)
{
ExecAuxRowMark *aerm;
+ /* ignore subplan if the FDW pushes the command down */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ resultRelInfo++;
+ continue;
+ }
+
subplan = mtstate->mt_plans[i]->plan;
aerm = ExecBuildAuxRowMark(erm, subplan->targetlist);
mtstate->mt_arowmarks[i] = lappend(mtstate->mt_arowmarks[i], aerm);
+ resultRelInfo++;
}
}
***************
*** 1793,1798 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
--- 1827,1839 ----
{
JunkFilter *j;
+ /* ignore subplan if the FDW pushes the command down */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ resultRelInfo++;
+ continue;
+ }
+
subplan = mtstate->mt_plans[i]->plan;
if (operation == CMD_INSERT || operation == CMD_UPDATE)
ExecCheckPlanOutput(resultRelInfo->ri_RelationDesc,
***************
*** 1887,1893 **** ExecEndModifyTable(ModifyTableState *node)
{
ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
! if (resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->EndForeignModify != NULL)
resultRelInfo->ri_FdwRoutine->EndForeignModify(node->ps.state,
resultRelInfo);
--- 1928,1935 ----
{
ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
! if (!resultRelInfo->ri_FdwPushdown &&
! resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->EndForeignModify != NULL)
resultRelInfo->ri_FdwRoutine->EndForeignModify(node->ps.state,
resultRelInfo);
*** a/src/backend/nodes/copyfuncs.c
--- b/src/backend/nodes/copyfuncs.c
***************
*** 186,191 **** _copyModifyTable(const ModifyTable *from)
--- 186,192 ----
COPY_NODE_FIELD(withCheckOptionLists);
COPY_NODE_FIELD(returningLists);
COPY_NODE_FIELD(fdwPrivLists);
+ COPY_NODE_FIELD(fdwPushdowns);
COPY_NODE_FIELD(rowMarks);
COPY_SCALAR_FIELD(epqParam);
COPY_SCALAR_FIELD(onConflictAction);
***************
*** 645,650 **** _copyForeignScan(const ForeignScan *from)
--- 646,652 ----
/*
* copy remainder of node
*/
+ COPY_SCALAR_FIELD(operation);
COPY_SCALAR_FIELD(fs_server);
COPY_NODE_FIELD(fdw_exprs);
COPY_NODE_FIELD(fdw_private);
*** a/src/backend/nodes/outfuncs.c
--- b/src/backend/nodes/outfuncs.c
***************
*** 340,345 **** _outModifyTable(StringInfo str, const ModifyTable *node)
--- 340,346 ----
WRITE_NODE_FIELD(withCheckOptionLists);
WRITE_NODE_FIELD(returningLists);
WRITE_NODE_FIELD(fdwPrivLists);
+ WRITE_NODE_FIELD(fdwPushdowns);
WRITE_NODE_FIELD(rowMarks);
WRITE_INT_FIELD(epqParam);
WRITE_ENUM_FIELD(onConflictAction, OnConflictAction);
***************
*** 591,596 **** _outForeignScan(StringInfo str, const ForeignScan *node)
--- 592,598 ----
_outScanInfo(str, (const Scan *) node);
+ WRITE_ENUM_FIELD(operation, CmdType);
WRITE_OID_FIELD(fs_server);
WRITE_NODE_FIELD(fdw_exprs);
WRITE_NODE_FIELD(fdw_private);
*** a/src/backend/nodes/readfuncs.c
--- b/src/backend/nodes/readfuncs.c
***************
*** 1471,1476 **** _readModifyTable(void)
--- 1471,1477 ----
READ_NODE_FIELD(withCheckOptionLists);
READ_NODE_FIELD(returningLists);
READ_NODE_FIELD(fdwPrivLists);
+ READ_NODE_FIELD(fdwPushdowns);
READ_NODE_FIELD(rowMarks);
READ_INT_FIELD(epqParam);
READ_ENUM_FIELD(onConflictAction, OnConflictAction);
*** a/src/backend/optimizer/plan/createplan.c
--- b/src/backend/optimizer/plan/createplan.c
***************
*** 3768,3773 **** make_foreignscan(List *qptlist,
--- 3768,3774 ----
plan->lefttree = outer_plan;
plan->righttree = NULL;
node->scan.scanrelid = scanrelid;
+ node->operation = CMD_SELECT;
/* fs_server will be filled in by create_foreignscan_plan */
node->fs_server = InvalidOid;
node->fdw_exprs = fdw_exprs;
***************
*** 5043,5048 **** make_modifytable(PlannerInfo *root,
--- 5044,5050 ----
Plan *plan = &node->plan;
double total_size;
List *fdw_private_list;
+ List *fdwpushdown_list;
ListCell *subnode;
ListCell *lc;
int i;
***************
*** 5123,5134 **** make_modifytable(PlannerInfo *root,
--- 5125,5138 ----
* construct private plan data, and accumulate it all into a list.
*/
fdw_private_list = NIL;
+ fdwpushdown_list = NIL;
i = 0;
foreach(lc, resultRelations)
{
Index rti = lfirst_int(lc);
FdwRoutine *fdwroutine;
List *fdw_private;
+ bool fdwpushdown;
/*
* If possible, we want to get the FdwRoutine from our RelOptInfo for
***************
*** 5156,5161 **** make_modifytable(PlannerInfo *root,
--- 5160,5173 ----
}
if (fdwroutine != NULL &&
+ fdwroutine->PlanDMLPushdown != NULL)
+ fdwpushdown = fdwroutine->PlanDMLPushdown(root, node, rti, i);
+ else
+ fdwpushdown = false;
+ fdwpushdown_list = lappend_int(fdwpushdown_list, fdwpushdown);
+
+ if (!fdwpushdown &&
+ fdwroutine != NULL &&
fdwroutine->PlanForeignModify != NULL)
fdw_private = fdwroutine->PlanForeignModify(root, node, rti, i);
else
***************
*** 5164,5169 **** make_modifytable(PlannerInfo *root,
--- 5176,5182 ----
i++;
}
node->fdwPrivLists = fdw_private_list;
+ node->fdwPushdowns = fdwpushdown_list;
return node;
}
*** a/src/include/executor/executor.h
--- b/src/include/executor/executor.h
***************
*** 184,190 **** extern void ExecutorEnd(QueryDesc *queryDesc);
extern void standard_ExecutorEnd(QueryDesc *queryDesc);
extern void ExecutorRewind(QueryDesc *queryDesc);
extern bool ExecCheckRTPerms(List *rangeTable, bool ereport_on_violation);
! extern void CheckValidResultRel(Relation resultRel, CmdType operation);
extern void InitResultRelInfo(ResultRelInfo *resultRelInfo,
Relation resultRelationDesc,
Index resultRelationIndex,
--- 184,190 ----
extern void standard_ExecutorEnd(QueryDesc *queryDesc);
extern void ExecutorRewind(QueryDesc *queryDesc);
extern bool ExecCheckRTPerms(List *rangeTable, bool ereport_on_violation);
! extern void CheckValidResultRel(ResultRelInfo *resultRelInfo, CmdType operation);
extern void InitResultRelInfo(ResultRelInfo *resultRelInfo,
Relation resultRelationDesc,
Index resultRelationIndex,
*** a/src/include/foreign/fdwapi.h
--- b/src/include/foreign/fdwapi.h
***************
*** 91,96 **** typedef TupleTableSlot *(*ExecForeignDelete_function) (EState *estate,
--- 91,108 ----
typedef void (*EndForeignModify_function) (EState *estate,
ResultRelInfo *rinfo);
+ typedef bool (*PlanDMLPushdown_function) (PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+
+ typedef void (*BeginDMLPushdown_function) (ForeignScanState *node,
+ int eflags);
+
+ typedef TupleTableSlot *(*IterateDMLPushdown_function) (ForeignScanState *node);
+
+ typedef void (*EndDMLPushdown_function) (ForeignScanState *node);
+
typedef int (*IsForeignRelUpdatable_function) (Relation rel);
typedef RowMarkType (*GetForeignRowMarkType_function) (RangeTblEntry *rte,
***************
*** 110,115 **** typedef void (*ExplainForeignModify_function) (ModifyTableState *mtstate,
--- 122,130 ----
int subplan_index,
struct ExplainState *es);
+ typedef void (*ExplainDMLPushdown_function) (ForeignScanState *node,
+ struct ExplainState *es);
+
typedef int (*AcquireSampleRowsFunc) (Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
***************
*** 161,166 **** typedef struct FdwRoutine
--- 176,185 ----
ExecForeignUpdate_function ExecForeignUpdate;
ExecForeignDelete_function ExecForeignDelete;
EndForeignModify_function EndForeignModify;
+ PlanDMLPushdown_function PlanDMLPushdown;
+ BeginDMLPushdown_function BeginDMLPushdown;
+ IterateDMLPushdown_function IterateDMLPushdown;
+ EndDMLPushdown_function EndDMLPushdown;
IsForeignRelUpdatable_function IsForeignRelUpdatable;
/* Functions for SELECT FOR UPDATE/SHARE row locking */
***************
*** 171,176 **** typedef struct FdwRoutine
--- 190,196 ----
/* Support functions for EXPLAIN */
ExplainForeignScan_function ExplainForeignScan;
ExplainForeignModify_function ExplainForeignModify;
+ ExplainDMLPushdown_function ExplainDMLPushdown;
/* Support functions for ANALYZE */
AnalyzeForeignTable_function AnalyzeForeignTable;
*** a/src/include/nodes/execnodes.h
--- b/src/include/nodes/execnodes.h
***************
*** 311,316 **** typedef struct JunkFilter
--- 311,317 ----
* TrigInstrument optional runtime measurements for triggers
* FdwRoutine FDW callback functions, if foreign table
* FdwState available to save private state of FDW
+ * FdwPushdown true when the command is pushed down
* WithCheckOptions list of WithCheckOption's to be checked
* WithCheckOptionExprs list of WithCheckOption expr states
* ConstraintExprs array of constraint-checking expr states
***************
*** 334,339 **** typedef struct ResultRelInfo
--- 335,341 ----
Instrumentation *ri_TrigInstrument;
struct FdwRoutine *ri_FdwRoutine;
void *ri_FdwState;
+ bool ri_FdwPushdown;
List *ri_WithCheckOptions;
List *ri_WithCheckOptionExprs;
List **ri_ConstraintExprs;
*** a/src/include/nodes/plannodes.h
--- b/src/include/nodes/plannodes.h
***************
*** 188,193 **** typedef struct ModifyTable
--- 188,194 ----
List *withCheckOptionLists; /* per-target-table WCO lists */
List *returningLists; /* per-target-table RETURNING tlists */
List *fdwPrivLists; /* per-target-table FDW private data lists */
+ List *fdwPushdowns; /* per-target-table FDW pushdown flags */
List *rowMarks; /* PlanRowMarks (non-locking only) */
int epqParam; /* ID of Param for EvalPlanQual re-eval */
OnConflictAction onConflictAction; /* ON CONFLICT action */
***************
*** 530,535 **** typedef struct WorkTableScan
--- 531,537 ----
typedef struct ForeignScan
{
Scan scan;
+ CmdType operation; /* SELECT/UPDATE/DELETE */
Oid fs_server; /* OID of foreign server */
List *fdw_exprs; /* expressions that FDW may evaluate */
List *fdw_private; /* private data for FDW */
I started looking at updated patch and its definitely iked the new
approach.
Patch passing the mandatory checks:
1) Patch applied cleanly using patch -p1 command
2) Source got compiled cleanly
3) make check, running cleanly
Explain output for the remote table and inheritance relations looks better
then earlier version of patch.
-- Inheritance foreign relation
postgres=# explain (ANALYZE, VERBOSE) update fp set a = 20;
QUERY
PLAN
-----------------------------------------------------------------------------------------------------------------------
Update on public.fp (cost=100.00..767.60 rows=10920 width=6) (actual
time=1.000..1.000 rows=0 loops=1)
Foreign Update on public.fp
Foreign Update on public.fc1
Foreign Update on public.fc2
Foreign Update on public.fc3
-> Foreign Update on public.fp (cost=100.00..191.90 rows=2730 width=6)
(actual time=0.493..0.493 rows=0 loops=1)
Remote SQL: UPDATE public.p SET a = 20
-> Foreign Update on public.fc1 (cost=100.00..191.90 rows=2730
width=6) (actual time=0.177..0.177 rows=0 loops=1)
Remote SQL: UPDATE public.c1 SET a = 20
-> Foreign Update on public.fc2 (cost=100.00..191.90 rows=2730
width=6) (actual time=0.163..0.163 rows=0 loops=1)
Remote SQL: UPDATE public.c2 SET a = 20
-> Foreign Update on public.fc3 (cost=100.00..191.90 rows=2730
width=6) (actual time=0.158..0.158 rows=0 loops=1)
Remote SQL: UPDATE public.c3 SET a = 20
Planning time: 0.228 ms
Execution time: 1.359 ms
(15 rows)
-- Foreign table update
postgres=# explain (ANALYZE, VERBOSE) update ft1 set c8 = 'updated' where
c1 = '200';
QUERY
PLAN
----------------------------------------------------------------------------------------------------------------------
Update on public.ft1 (cost=100.00..116.13 rows=2 width=144) (actual
time=0.485..0.485 rows=0 loops=1)
-> Foreign Update on public.ft1 (cost=100.00..116.13 rows=2 width=144)
(actual time=0.483..0.483 rows=0 loops=1)
Remote SQL: UPDATE public.t1 SET c8 = 'updated '::character(10)
WHERE ((c1 = 200))
Planning time: 0.158 ms
Execution time: 0.786 ms
(5 rows)
-- Explain output for the returning clause:
postgres=# explain (ANALYZE, VERBOSE) update ft1 set c8 = 'updated' where
c1 = '200' returning c2;
QUERY
PLAN
----------------------------------------------------------------------------------------------------------------------
Update on public.ft1 (cost=100.00..116.13 rows=2 width=144) (actual
time=0.516..0.516 rows=0 loops=1)
Output: c2
-> Foreign Update on public.ft1 (cost=100.00..116.13 rows=2 width=144)
(actual time=0.514..0.514 rows=0 loops=1)
Remote SQL: UPDATE public.t1 SET c8 = 'updated '::character(10)
WHERE ((c1 = 200)) RETURNING c2
Planning time: 0.172 ms
Execution time: 0.938 ms
(6 rows)
-- Explain output when returning clause is not pushdown safe:
postgres=# explain (ANALYZE, VERBOSE) update ft1 set c8 = 'updated' where
c1 = '200' returning local_func(20);
QUERY
PLAN
----------------------------------------------------------------------------------------------------------------------
Update on public.ft1 (cost=100.00..116.13 rows=2 width=144) (actual
time=0.364..0.364 rows=0 loops=1)
Output: local_func(20)
-> Foreign Update on public.ft1 (cost=100.00..116.13 rows=2 width=144)
(actual time=0.363..0.363 rows=0 loops=1)
Remote SQL: UPDATE public.t1 SET c8 = 'updated '::character(10)
WHERE ((c1 = 200))
Planning time: 0.142 ms
Execution time: 0.623 ms
(6 rows)
-- Explain output with PREPARE:
postgres=# explain (ANALYZE, VERBOSE) execute ftupdate(20);
QUERY
PLAN
----------------------------------------------------------------------------------------------------------------------
Update on public.ft1 (cost=100.00..116.13 rows=2 width=144) (actual
time=0.712..0.712 rows=0 loops=1)
-> Foreign Update on public.ft1 (cost=100.00..116.13 rows=2 width=144)
(actual time=0.711..0.711 rows=1 loops=1)
Remote SQL: UPDATE public.t1 SET c8 = 'updated '::character(10)
WHERE ((c1 = 20))
Execution time: 1.001 ms
(4 rows)
With the initial look and test overall things looking great, I am still
reviewing the code changes but here are few early doubts/questions:
.) What the need of following change ?
@@ -833,9 +833,6 @@ appendWhereClause(StringInfo buf,
int nestlevel;
ListCell *lc;
- if (params)
- *params = NIL; /* initialize result list to empty */
-
/* Set up context struct for recursion */
context.root = root;
context.foreignrel = baserel;
@@ -971,6 +968,63 @@ deparseUpdateSql(StringInfo buf, PlannerInfo *root,
}
.) When Tom Lane and Stephen Frost suggested getting the core code involved,
I thought that we can do the mandatory checks into core it self and making
completely out of dml_is_pushdown_safe(). Please correct me
.) Documentation for the new API is missing (fdw-callbacks).
Regards,
On Fri, Dec 25, 2015 at 3:30 PM, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>
wrote:
On 2015/12/24 4:34, Robert Haas wrote:
On Wed, Dec 23, 2015 at 5:50 AM, Rushabh Lathia
<rushabh.lathia@gmail.com> wrote:+1.
I like idea of separate FDW API for the DML Pushdown. Was thinking can't
we
can re-use the IterateForeignScan(ForeignScanState *node) rather then
introducing IterateDMLPushdown(ForeignScanState *node) new API ?Yeah, I think we need to ask ourselves what advantage we're getting
out of adding any new core APIs. Marking the scan as a pushed-down
update or delete has some benefit in terms of making the information
visible via EXPLAIN, but even that's a pretty thin benefit. The
iterate method seems to just complicate the core code without any
benefit at all. More generally, there is very, very little code in
this patch that accomplishes anything that could not be done just as
well with the existing methods. So why are we even doing these core
changes?From the FDWs' point of view, ISTM that what FDWs have to do for
IterateDMLPushdown is quite different from what FDWs have to do for
IterateForeignScan; eg, IterateDMLPushdown must work in accordance with
presence/absence of a RETURNING list. (In addition to that,
IterateDMLPushdown has been designed so that it must make the scan tuple
available to later RETURNING projection in nodeModifyTable.c.) So, I think
that it's better to FDWs to add separate APIs for the DML pushdown, making
the FDW code much simpler. So based on that idea, I added the postgres_fdw
changes to the patch. Attached is an updated version of the patch, which
is still WIP, but I'd be happy if I could get any feedback.Tom seemed to think that we could centralize some checks in the core
code, say, related to triggers, or to LIMIT. But there's nothing like
that in this patch, so I'm not really understanding the point.For the trigger check, I added relation_has_row_level_triggers. I placed
that function in postgres_fdw.c in the updated patch, but I think that by
placing that function in the core, FDWs can share that function. As for
the LIMIT, I'm not sure we can do something about that.I think the current design allows us to handle a pushed-down update on a
join, ie, "UPDATE foo ... FROM bar ..." where both foo and bar are remote,
which was Tom's concern, but I'll leave that for another patch. Also, I
think the current design could also extend to push down INSERT .. RETURNING
.., but I'd like to leave that for future work.I'll add this to the next CF.
Best regards,
Etsuro Fujita
Rushabh Lathia
On 25 December 2015 at 10:00, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp> wrote:
On 2015/12/24 4:34, Robert Haas wrote:
On Wed, Dec 23, 2015 at 5:50 AM, Rushabh Lathia
<rushabh.lathia@gmail.com> wrote:+1.
I like idea of separate FDW API for the DML Pushdown. Was thinking can't
we
can re-use the IterateForeignScan(ForeignScanState *node) rather then
introducing IterateDMLPushdown(ForeignScanState *node) new API ?Yeah, I think we need to ask ourselves what advantage we're getting
out of adding any new core APIs. Marking the scan as a pushed-down
update or delete has some benefit in terms of making the information
visible via EXPLAIN, but even that's a pretty thin benefit. The
iterate method seems to just complicate the core code without any
benefit at all. More generally, there is very, very little code in
this patch that accomplishes anything that could not be done just as
well with the existing methods. So why are we even doing these core
changes?From the FDWs' point of view, ISTM that what FDWs have to do for
IterateDMLPushdown is quite different from what FDWs have to do for
IterateForeignScan; eg, IterateDMLPushdown must work in accordance with
presence/absence of a RETURNING list. (In addition to that,
IterateDMLPushdown has been designed so that it must make the scan tuple
available to later RETURNING projection in nodeModifyTable.c.) So, I think
that it's better to FDWs to add separate APIs for the DML pushdown, making
the FDW code much simpler. So based on that idea, I added the postgres_fdw
changes to the patch. Attached is an updated version of the patch, which is
still WIP, but I'd be happy if I could get any feedback.Tom seemed to think that we could centralize some checks in the core
code, say, related to triggers, or to LIMIT. But there's nothing like
that in this patch, so I'm not really understanding the point.For the trigger check, I added relation_has_row_level_triggers. I placed
that function in postgres_fdw.c in the updated patch, but I think that by
placing that function in the core, FDWs can share that function. As for the
LIMIT, I'm not sure we can do something about that.I think the current design allows us to handle a pushed-down update on a
join, ie, "UPDATE foo ... FROM bar ..." where both foo and bar are remote,
which was Tom's concern, but I'll leave that for another patch. Also, I
think the current design could also extend to push down INSERT .. RETURNING
.., but I'd like to leave that for future work.I'll add this to the next CF.
I've run into an issue:
*# UPDATE master_customers SET id = 22 WHERE id = 16 RETURNING
tableoid::regclass;
ERROR:
CONTEXT: Remote SQL command: UPDATE public.customers SET id = 22
WHERE ((id = 16)) RETURNING NULL
However, this works:
*# UPDATE master_customers SET id = 22 WHERE id = 16 RETURNING
tableoid::regclass, *;
tableoid | id | name | company | registered_date |
expiry_date | active | status | account_level
-----------------+----+-------+---------------+-----------------+-------------+--------+---------+---------------
local_customers | 22 | Bruce | Jo's Cupcakes | 2015-01-15 |
2017-01-14 | t | running | basic
(1 row)
In this example, "local_customers" inherits from the remote table
"public"."customers", which inherits again from the local table
"master_customers"
Same issue with DELETE of course, and the ::regclass isn't important here.
Thom
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/01/06 18:58, Rushabh Lathia wrote:
I started looking at updated patch and its definitely iked the new
approach.
Thanks for the review!
With the initial look and test overall things looking great, I am still
reviewing the code changes but here are few early doubts/questions:
.) What the need of following change ?
@@ -833,9 +833,6 @@ appendWhereClause(StringInfo buf,
int nestlevel;
ListCell *lc;- if (params) - *params = NIL; /* initialize result list to empty */ - /* Set up context struct for recursion */ context.root = root; context.foreignrel = baserel; @@ -971,6 +968,63 @@ deparseUpdateSql(StringInfo buf, PlannerInfo *root, }
It is needed for deparsePushedDownUpdateSql to store params in both
WHERE clauses and expressions to assign to the target columns
into one params_list list.
.) When Tom Lane and Stephen Frost suggested getting the core code involved,
I thought that we can do the mandatory checks into core it self and making
completely out of dml_is_pushdown_safe(). Please correct me
The reason why I put that function in postgres_fdw.c is Check point 4:
+ * 4. We can't push an UPDATE down, if any expressions to assign to
the target
+ * columns are unsafe to evaluate on the remote server.
I think this depends on the capabilities of the FDW.
.) Documentation for the new API is missing (fdw-callbacks).
Will add the docs.
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/01/06 20:37, Thom Brown wrote:
On 25 December 2015 at 10:00, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp> wrote:
Attached is an updated version of the patch, which is
still WIP, but I'd be happy if I could get any feedback.
I've run into an issue:
*# UPDATE master_customers SET id = 22 WHERE id = 16 RETURNING
tableoid::regclass;
ERROR:
CONTEXT: Remote SQL command: UPDATE public.customers SET id = 22
WHERE ((id = 16)) RETURNING NULLHowever, this works:
*# UPDATE master_customers SET id = 22 WHERE id = 16 RETURNING
tableoid::regclass, *;
tableoid | id | name | company | registered_date |
expiry_date | active | status | account_level
-----------------+----+-------+---------------+-----------------+-------------+--------+---------+---------------
local_customers | 22 | Bruce | Jo's Cupcakes | 2015-01-15 |
2017-01-14 | t | running | basic
(1 row)In this example, "local_customers" inherits from the remote table
"public"."customers", which inherits again from the local table
"master_customers"Same issue with DELETE of course, and the ::regclass isn't important here.
Will fix.
Thanks for the testing!
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/01/07 21:50, Etsuro Fujita wrote:
On 2016/01/06 20:37, Thom Brown wrote:
On 25 December 2015 at 10:00, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:Attached is an updated version of the patch, which is
still WIP, but I'd be happy if I could get any feedback.
I've run into an issue:
*# UPDATE master_customers SET id = 22 WHERE id = 16 RETURNING
tableoid::regclass;
ERROR:
CONTEXT: Remote SQL command: UPDATE public.customers SET id = 22
WHERE ((id = 16)) RETURNING NULL
Will fix.
While working on this, I noticed that the existing postgres_fdw system
shows similar behavior, so I changed the subject.
IIUC, the reason for that is when the local query specifies "RETURNING
tableoid::regclass", the FDW has fmstate->has_returning=false while the
remote query executed at ModifyTable has "RETURNING NULL", as shown in
the above example; that would cause an abnormal exit in executing the
remote query in postgresExecForeignUpdate, since that the FDW would get
PGRES_TUPLES_OK as a result of the query while the FDW would think that
the right result to get should be PGRES_COMMAND_OK, from the flag
fmstate->has_returning=false.
Attached is a patch to fix that.
Best regards,
Etsuro Fujita
Attachments:
fdw-foreign-modify.patchapplication/x-patch; name=fdw-foreign-modify.patchDownload
*** a/contrib/postgres_fdw/deparse.c
--- b/contrib/postgres_fdw/deparse.c
***************
*** 1003,1008 **** deparseReturningList(StringInfo buf, PlannerInfo *root,
--- 1003,1009 ----
List **retrieved_attrs)
{
Bitmapset *attrs_used = NULL;
+ bool has_returning = false;
if (trig_after_row)
{
***************
*** 1021,1031 **** deparseReturningList(StringInfo buf, PlannerInfo *root,
--- 1022,1072 ----
&attrs_used);
}
+ /*
+ * Check to see whether the remote query has a RETURNING clause.
+ *
+ * XXX be careful to keep this in sync with deparseTargetList.
+ */
if (attrs_used != NULL)
{
+ if (bms_is_member(SelfItemPointerAttributeNumber - FirstLowInvalidHeapAttributeNumber,
+ attrs_used))
+ has_returning = true;
+ else
+ {
+ TupleDesc tupdesc = RelationGetDescr(rel);
+ bool have_wholerow;
+ int i;
+
+ /* If there's a whole-row reference, we'll need all the columns. */
+ have_wholerow = bms_is_member(0 - FirstLowInvalidHeapAttributeNumber,
+ attrs_used);
+
+ for (i = 1; i <= tupdesc->natts; i++)
+ {
+ Form_pg_attribute attr = tupdesc->attrs[i - 1];
+
+ /* Ignore dropped attributes. */
+ if (attr->attisdropped)
+ continue;
+
+ if (have_wholerow ||
+ bms_is_member(i - FirstLowInvalidHeapAttributeNumber,
+ attrs_used))
+ {
+ has_returning = true;
+ break;
+ }
+ }
+ }
+ }
+
+ if (has_returning != false)
+ {
appendStringInfoString(buf, " RETURNING ");
deparseTargetList(buf, root, rtindex, rel, attrs_used,
retrieved_attrs);
+ Assert(*retrieved_attrs != NIL);
}
else
*retrieved_attrs = NIL;
*** a/contrib/postgres_fdw/expected/postgres_fdw.out
--- b/contrib/postgres_fdw/expected/postgres_fdw.out
***************
*** 2408,2413 **** SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
--- 2408,2466 ----
1104 | 204 | ddd |
(819 rows)
+ EXPLAIN (verbose, costs off)
+ INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
+ QUERY PLAN
+ -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Insert on public.ft2
+ Output: (tableoid)::regclass
+ Remote SQL: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6, c7, c8) VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
+ -> Result
+ Output: 9999, 999, NULL::integer, 'foo'::text, NULL::timestamp with time zone, NULL::timestamp without time zone, NULL::character varying, 'ft2 '::character(10), NULL::user_enum
+ (5 rows)
+
+ INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
+ tableoid
+ ----------
+ ft2
+ (1 row)
+
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
+ QUERY PLAN
+ -------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Output: (tableoid)::regclass
+ Remote SQL: UPDATE "S 1"."T 1" SET c3 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.ft2
+ Output: c1, c2, NULL::integer, 'bar'::text, c4, c5, c6, c7, c8, ctid
+ Remote SQL: SELECT "C 1", c2, c4, c5, c6, c7, c8, ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
+ (6 rows)
+
+ UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
+ tableoid
+ ----------
+ ft2
+ (1 row)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
+ QUERY PLAN
+ ------------------------------------------------------------------------------------
+ Delete on public.ft2
+ Output: (tableoid)::regclass
+ Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1
+ -> Foreign Scan on public.ft2
+ Output: ctid
+ Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
+ (6 rows)
+
+ DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
+ tableoid
+ ----------
+ ft2
+ (1 row)
+
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
BEGIN
*** a/contrib/postgres_fdw/sql/postgres_fdw.sql
--- b/contrib/postgres_fdw/sql/postgres_fdw.sql
***************
*** 413,418 **** EXPLAIN (verbose, costs off)
--- 413,427 ----
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
+ EXPLAIN (verbose, costs off)
+ INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
+ INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
+ UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
+ EXPLAIN (verbose, costs off)
+ DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
+ DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
*** a/src/backend/executor/nodeModifyTable.c
--- b/src/backend/executor/nodeModifyTable.c
***************
*** 308,313 **** ExecInsert(ModifyTableState *mtstate,
--- 308,319 ----
/* FDW might have changed tuple */
tuple = ExecMaterializeSlot(slot);
+ /*
+ * AFTER ROW Triggers or RETURNING expressions might reference the
+ * tableoid column, so initialize t_tableOid before evaluating them.
+ */
+ tuple->t_tableOid = RelationGetRelid(resultRelationDesc);
+
newId = InvalidOid;
}
else
***************
*** 561,566 **** ExecDelete(ItemPointer tupleid,
--- 567,574 ----
}
else if (resultRelInfo->ri_FdwRoutine)
{
+ HeapTuple tuple;
+
/*
* delete from foreign table: let the FDW do it
*
***************
*** 579,584 **** ExecDelete(ItemPointer tupleid,
--- 587,601 ----
if (slot == NULL) /* "do nothing" */
return NULL;
+
+ /*
+ * RETURNING expressions might reference the tableoid column, so
+ * initialize t_tableOid before evaluating them.
+ */
+ if (slot->tts_isempty)
+ ExecStoreAllNullTuple(slot);
+ tuple = ExecMaterializeSlot(slot);
+ tuple->t_tableOid = RelationGetRelid(resultRelationDesc);
}
else
{
***************
*** 838,843 **** ExecUpdate(ItemPointer tupleid,
--- 855,866 ----
/* FDW might have changed tuple */
tuple = ExecMaterializeSlot(slot);
+
+ /*
+ * AFTER ROW Triggers or RETURNING expressions might reference the
+ * tableoid column, so initialize t_tableOid before evaluating them.
+ */
+ tuple->t_tableOid = RelationGetRelid(resultRelationDesc);
}
else
{
On 2016/01/07 21:45, Etsuro Fujita wrote:
On 2016/01/06 18:58, Rushabh Lathia wrote:
.) Documentation for the new API is missing (fdw-callbacks).
Will add the docs.
I added docs for new FDW APIs.
Other changes:
* Rename relation_has_row_level_triggers to relation_has_row_triggers
shortly, and move it to rewriteHandler.c. I'm not sure rewriteHandler.c
is a good place for that, though.
* Revise code, including a helper function get_result_result, whcih I
implemented using a modified version of store_returning_result in the
previous patch, but on second thought, I think that that is a bit too
invasive. So, I re-implemented that function directly using
make_tuple_from_result_row.
* Add more comments.
* Add more regression tests.
Attached is an updated version of the patch. Comments are wellcome!
(If the fix [1]/messages/by-id/568F4430.6060805@lab.ntt.co.jp is okay, I'd like to update this patch on top of the
patch in [1]/messages/by-id/568F4430.6060805@lab.ntt.co.jp.)
Best regards,
Etsuro Fujita
Attachments:
fdw-dml-pushdown-v3.patchapplication/x-patch; name=fdw-dml-pushdown-v3.patchDownload
*** a/contrib/postgres_fdw/deparse.c
--- b/contrib/postgres_fdw/deparse.c
***************
*** 816,822 **** deparseTargetList(StringInfo buf,
*
* If params is not NULL, it receives a list of Params and other-relation Vars
* used in the clauses; these values must be transmitted to the remote server
! * as parameter values.
*
* If params is NULL, we're generating the query for EXPLAIN purposes,
* so Params and other-relation Vars should be replaced by dummy values.
--- 816,822 ----
*
* If params is not NULL, it receives a list of Params and other-relation Vars
* used in the clauses; these values must be transmitted to the remote server
! * as parameter values. Caller is responsible for initializing it to empty.
*
* If params is NULL, we're generating the query for EXPLAIN purposes,
* so Params and other-relation Vars should be replaced by dummy values.
***************
*** 833,841 **** appendWhereClause(StringInfo buf,
int nestlevel;
ListCell *lc;
- if (params)
- *params = NIL; /* initialize result list to empty */
-
/* Set up context struct for recursion */
context.root = root;
context.foreignrel = baserel;
--- 833,838 ----
***************
*** 971,976 **** deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 968,1030 ----
}
/*
+ * deparse remote UPDATE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ deparse_expr_cxt context;
+ bool first;
+ ListCell *lc;
+
+ if (params_list)
+ *params_list = NIL; /* initialize result list to empty */
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = params_list;
+
+ appendStringInfoString(buf, "UPDATE ");
+ deparseRelation(buf, rel);
+ appendStringInfoString(buf, " SET ");
+
+ first = true;
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(targetlist, attnum);
+
+ if (!first)
+ appendStringInfoString(buf, ", ");
+ first = false;
+
+ deparseColumnRef(buf, rtindex, attnum, root);
+ appendStringInfoString(buf, " = ");
+ deparseExpr((Expr *) tle->expr, &context);
+ }
+ if (remote_conds)
+ appendWhereClause(buf, root, baserel, remote_conds,
+ true, params_list);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* deparse remote DELETE statement
*
* The statement text is appended to buf, and we also create an integer List
***************
*** 993,998 **** deparseDeleteSql(StringInfo buf, PlannerInfo *root,
--- 1047,1082 ----
}
/*
+ * deparse remote DELETE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+
+ if (params_list)
+ *params_list = NIL; /* initialize result list to empty */
+
+ appendStringInfoString(buf, "DELETE FROM ");
+ deparseRelation(buf, rel);
+ if (remote_conds)
+ appendWhereClause(buf, root, baserel, remote_conds,
+ true, params_list);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* Add a RETURNING clause, if needed, to an INSERT/UPDATE/DELETE.
*/
static void
*** a/contrib/postgres_fdw/expected/postgres_fdw.out
--- b/contrib/postgres_fdw/expected/postgres_fdw.out
***************
*** 1314,1320 **** INSERT INTO ft2 (c1,c2,c3)
--- 1314,1339 ----
(3 rows)
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 300), c3 = (c3 || '_update3'::text) WHERE ((("C 1" % 10) = 3))
+ (3 rows)
+
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Output: c1, c2, c3, c4, c5, c6, c7, c8
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 400), c3 = (c3 || '_update7'::text) WHERE ((("C 1" % 10) = 7)) RETURNING "C 1", c2, c3, c4, c5, c6, c7, c8
+ (4 rows)
+
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
c1 | c2 | c3 | c4 | c5 | c6 | c7 | c8
------+-----+--------------------+------------------------------+--------------------------+----+------------+-----
***************
*** 1424,1430 **** UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
--- 1443,1449 ----
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
***************
*** 1445,1460 **** UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
! QUERY PLAN
! ----------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1 RETURNING "C 1", c4
! -> Foreign Scan on public.ft2
! Output: ctid
! Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) FOR UPDATE
! (6 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
--- 1464,1477 ----
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
! QUERY PLAN
! --------------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
! -> Foreign Delete on public.ft2
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) RETURNING "C 1", c4
! (4 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
***************
*** 1565,1571 **** DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
--- 1582,1588 ----
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
***************
*** 2408,2413 **** SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
--- 2425,2498 ----
1104 | 204 | ddd |
(819 rows)
+ INSERT INTO ft2 (c1,c2,c3)
+ VALUES (1201,1201,'aaa'), (1202,1202,'bbb'), (1203,1203,'ccc') RETURNING *;
+ c1 | c2 | c3 | c4 | c5 | c6 | c7 | c8
+ ------+------+-----+----+----+----+------------+----
+ 1201 | 1201 | aaa | | | | ft2 |
+ 1202 | 1202 | bbb | | | | ft2 |
+ 1203 | 1203 | ccc | | | | ft2 |
+ (3 rows)
+
+ INSERT INTO ft2 (c1,c2,c3) VALUES (1204,1204,'ddd'), (1205,1205,'eee');
+ PREPARE mt1(int, int, text) AS UPDATE ft2 SET c2 = c2 + $2, c3 = c3 || $3 WHERE c1 % 10 = $1; -- can be pushed down
+ EXPLAIN (verbose, costs off) EXECUTE mt1(1, 0, '_update1');
+ QUERY PLAN
+ --------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 0), c3 = (c3 || '_update1'::text) WHERE ((("C 1" % 10) = 1))
+ (3 rows)
+
+ EXECUTE mt1(1, 0, '_update1');
+ PREPARE mt2(int) AS DELETE FROM ft2 WHERE c1 = $1 RETURNING c1, c2, c3; -- can be pushed down
+ EXPLAIN (verbose, costs off) EXECUTE mt2(1201);
+ QUERY PLAN
+ --------------------------------------------------------------------------------------------
+ Delete on public.ft2
+ Output: c1, c2, c3
+ -> Foreign Delete on public.ft2
+ Remote SQL: DELETE FROM "S 1"."T 1" WHERE (("C 1" = 1201)) RETURNING "C 1", c2, c3
+ (4 rows)
+
+ EXECUTE mt2(1201);
+ c1 | c2 | c3
+ ------+------+-------------
+ 1201 | 1201 | aaa_update1
+ (1 row)
+
+ PREPARE mt3(int) AS DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft2.c2 > $1 RETURNING ft2.*; -- can't be pushed down
+ EXPLAIN (verbose, costs off) EXECUTE mt3(1201);
+ QUERY PLAN
+ ---------------------------------------------------------------------------------------------------
+ Delete on public.ft2
+ Output: ft2.c1, ft2.c2, ft2.c3, ft2.c4, ft2.c5, ft2.c6, ft2.c7, ft2.c8
+ Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1 RETURNING "C 1", c2, c3, c4, c5, c6, c7, c8
+ -> Hash Join
+ Output: ft2.ctid, ft1.*
+ Hash Cond: (ft1.c1 = ft2.c2)
+ -> Foreign Scan on public.ft1
+ Output: ft1.*, ft1.c1
+ Remote SQL: SELECT "C 1", c2, c3, c4, c5, c6, c7, c8 FROM "S 1"."T 1"
+ -> Hash
+ Output: ft2.ctid, ft2.c2
+ -> Foreign Scan on public.ft2
+ Output: ft2.ctid, ft2.c2
+ Remote SQL: SELECT c2, ctid FROM "S 1"."T 1" WHERE ((c2 > 1201)) FOR UPDATE
+ (14 rows)
+
+ EXECUTE mt3(1201);
+ c1 | c2 | c3 | c4 | c5 | c6 | c7 | c8
+ ------+------+-----+----+----+----+------------+----
+ 1202 | 1202 | bbb | | | | ft2 |
+ 1203 | 1203 | ccc | | | | ft2 |
+ 1204 | 1204 | ddd | | | | ft2 |
+ 1205 | 1205 | eee | | | | ft2 |
+ (4 rows)
+
+ DEALLOCATE mt1;
+ DEALLOCATE mt2;
+ DEALLOCATE mt3;
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
BEGIN
***************
*** 2553,2560 **** DETAIL: Failing row contains (1111, -2, null, null, null, null, ft1 , nul
CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6, c7, c8) VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
! DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 2638,2645 ----
CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6, c7, c8) VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
! DETAIL: Failing row contains (1, -1, 00001_update1_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 2713,2719 **** savepoint s3;
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 2798,2804 ----
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (-2) WHERE ((c2 = 42)) AND (("C 1" = 10))
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 2852,2859 **** DETAIL: Failing row contains (1111, -2, null, null, null, null, ft1 , nul
CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6, c7, c8) VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
! DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2positive;
-- But inconsistent check constraints provide inconsistent results
ALTER FOREIGN TABLE ft1 ADD CONSTRAINT ft1_c2negative CHECK (c2 < 0);
--- 2937,2944 ----
CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6, c7, c8) VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
! DETAIL: Failing row contains (1, -1, 00001_update1_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2positive;
-- But inconsistent check constraints provide inconsistent results
ALTER FOREIGN TABLE ft1 ADD CONSTRAINT ft1_c2negative CHECK (c2 < 0);
***************
*** 3246,3251 **** NOTICE: NEW: (13,"test triggered !")
--- 3331,3529 ----
(0,27)
(1 row)
+ -- cleanup
+ DROP TRIGGER trig_row_before ON rem1;
+ DROP TRIGGER trig_row_after ON rem1;
+ DROP TRIGGER trig_local_before ON loc1;
+ -- Test DML pushdown functionality
+ -- Test with statement-level triggers
+ CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_stmt_before ON rem1;
+ CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_stmt_after ON rem1;
+ -- Test with row-level ON INSERT triggers
+ CREATE TRIGGER trig_row_before_insert
+ BEFORE INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_before_insert ON rem1;
+ CREATE TRIGGER trig_row_after_insert
+ AFTER INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_after_insert ON rem1;
+ -- Test with row-level ON UPDATE triggers
+ CREATE TRIGGER trig_row_before_update
+ BEFORE UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_before_update ON rem1;
+ CREATE TRIGGER trig_row_after_update
+ AFTER UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+ -------------------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_after_update ON rem1;
+ -- Test with row-level ON DELETE triggers
+ CREATE TRIGGER trig_row_before_delete
+ BEFORE DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ DROP TRIGGER trig_row_before_delete ON rem1;
+ CREATE TRIGGER trig_row_after_delete
+ AFTER DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+ ------------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ DROP TRIGGER trig_row_after_delete ON rem1;
-- ===================================================================
-- test inheritance features
-- ===================================================================
***************
*** 3715,3720 **** fetch from c;
--- 3993,4048 ----
update bar set f2 = null where current of c;
ERROR: WHERE CURRENT OF is not supported for this table type
rollback;
+ explain (verbose, costs off)
+ delete from foo where f1 < 5 returning *;
+ QUERY PLAN
+ --------------------------------------------------------------------------------
+ Delete on public.foo
+ Output: foo.f1, foo.f2
+ Delete on public.foo
+ Foreign Delete on public.foo2
+ -> Index Scan using i_foo_f1 on public.foo
+ Output: foo.ctid
+ Index Cond: (foo.f1 < 5)
+ -> Foreign Delete on public.foo2
+ Remote SQL: DELETE FROM public.loct1 WHERE ((f1 < 5)) RETURNING f1, f2
+ (9 rows)
+
+ delete from foo where f1 < 5 returning *;
+ f1 | f2
+ ----+----
+ 1 | 1
+ 3 | 3
+ 0 | 0
+ 2 | 2
+ 4 | 4
+ (5 rows)
+
+ explain (verbose, costs off)
+ update bar set f2 = f2 + 100 returning *;
+ QUERY PLAN
+ ------------------------------------------------------------------------------
+ Update on public.bar
+ Output: bar.f1, bar.f2
+ Update on public.bar
+ Foreign Update on public.bar2
+ -> Seq Scan on public.bar
+ Output: bar.f1, (bar.f2 + 100), bar.ctid
+ -> Foreign Update on public.bar2
+ Remote SQL: UPDATE public.loct2 SET f2 = (f2 + 100) RETURNING f1, f2
+ (8 rows)
+
+ update bar set f2 = f2 + 100 returning *;
+ f1 | f2
+ ----+-----
+ 1 | 311
+ 2 | 322
+ 6 | 266
+ 3 | 333
+ 4 | 344
+ 7 | 277
+ (6 rows)
+
drop table foo cascade;
NOTICE: drop cascades to foreign table foo2
drop table bar cascade;
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
***************
*** 32,37 ****
--- 32,38 ----
#include "optimizer/restrictinfo.h"
#include "optimizer/var.h"
#include "parser/parsetree.h"
+ #include "rewrite/rewriteHandler.h"
#include "utils/builtins.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
***************
*** 57,63 **** PG_MODULE_MAGIC;
* planner to executor. Currently we store:
*
* 1) SELECT statement text to be sent to the remote server
! * 2) Integer list of attribute numbers retrieved by the SELECT
*
* These items are indexed with the enum FdwScanPrivateIndex, so an item
* can be fetched with list_nth(). For example, to get the SELECT statement:
--- 58,65 ----
* planner to executor. Currently we store:
*
* 1) SELECT statement text to be sent to the remote server
! * 2) List of restriction clauses that can be executed remotely
! * 3) Integer list of attribute numbers retrieved by the SELECT
*
* These items are indexed with the enum FdwScanPrivateIndex, so an item
* can be fetched with list_nth(). For example, to get the SELECT statement:
***************
*** 67,72 **** enum FdwScanPrivateIndex
--- 69,76 ----
{
/* SQL statement to execute remotely (as a String node) */
FdwScanPrivateSelectSql,
+ /* List of restriction clauses that can be executed remotely */
+ FdwScanPrivateRemoteConds,
/* Integer list of attribute numbers retrieved by the SELECT */
FdwScanPrivateRetrievedAttrs
};
***************
*** 94,99 **** enum FdwModifyPrivateIndex
--- 98,125 ----
};
/*
+ * Similarly, this enum describes what's kept in the fdw_private list for
+ * a ForeignScan node that has pushed down an UPDATE/DELETE to the remote
+ * server. We store:
+ *
+ * 1) UPDATE/DELETE statement text to be sent to the remote server
+ * 2) Boolean flag showing if the remote query has a RETURNING clause
+ * 3) Integer list of attribute numbers retrieved by RETURNING, if any
+ * 4) Boolean flag showing if we set the command es_processed
+ */
+ enum FdwDmlPushdownPrivateIndex
+ {
+ /* SQL statement to execute remotely (as a String node) */
+ FdwDmlPushdownPrivateUpdateSql,
+ /* has-returning flag (as an integer Value node) */
+ FdwDmlPushdownPrivateHasReturning,
+ /* Integer list of attribute numbers retrieved by RETURNING */
+ FdwDmlPushdownPrivateRetrievedAttrs,
+ /* set-processed flag (as an integer Value node) */
+ FdwDmlPushdownPrivateSetProcessed
+ };
+
+ /*
* Execution state of a foreign scan using postgres_fdw.
*/
typedef struct PgFdwScanState
***************
*** 156,161 **** typedef struct PgFdwModifyState
--- 182,217 ----
} PgFdwModifyState;
/*
+ * Execution state of a pushed-down update/delete operation.
+ */
+ typedef struct PgFdwDmlPushdownState
+ {
+ Relation rel; /* relcache entry for the foreign table */
+ AttInMetadata *attinmeta; /* attribute datatype conversion metadata */
+
+ /* extracted fdw_private data */
+ char *query; /* text of UPDATE/DELETE command */
+ bool has_returning; /* is there a RETURNING clause? */
+ List *retrieved_attrs; /* attr numbers retrieved by RETURNING */
+ bool set_processed; /* do we set the command es_processed? */
+
+ /* for remote query execution */
+ PGconn *conn; /* connection for the scan */
+ int numParams; /* number of parameters passed to query */
+ FmgrInfo *param_flinfo; /* output conversion functions for them */
+ List *param_exprs; /* executable expressions for param values */
+ const char **param_values; /* textual values of query parameters */
+
+ /* for storing result tuples */
+ PGresult *result; /* result for query */
+ int num_tuples; /* # of result tuples */
+ int next_tuple; /* index of next one to return */
+
+ /* working memory contexts */
+ MemoryContext temp_cxt; /* context for per-tuple temporary data */
+ } PgFdwDmlPushdownState;
+
+ /*
* Workspace for analyzing a foreign table.
*/
typedef struct PgFdwAnalyzeState
***************
*** 247,252 **** static TupleTableSlot *postgresExecForeignDelete(EState *estate,
--- 303,315 ----
static void postgresEndForeignModify(EState *estate,
ResultRelInfo *resultRelInfo);
static int postgresIsForeignRelUpdatable(Relation rel);
+ static bool postgresPlanDMLPushdown(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+ static void postgresBeginDMLPushdown(ForeignScanState *node, int eflags);
+ static TupleTableSlot *postgresIterateDMLPushdown(ForeignScanState *node);
+ static void postgresEndDMLPushdown(ForeignScanState *node);
static void postgresExplainForeignScan(ForeignScanState *node,
ExplainState *es);
static void postgresExplainForeignModify(ModifyTableState *mtstate,
***************
*** 254,259 **** static void postgresExplainForeignModify(ModifyTableState *mtstate,
--- 317,324 ----
List *fdw_private,
int subplan_index,
ExplainState *es);
+ static void postgresExplainDMLPushdown(ForeignScanState *node,
+ ExplainState *es);
static bool postgresAnalyzeForeignTable(Relation relation,
AcquireSampleRowsFunc *func,
BlockNumber *totalpages);
***************
*** 290,295 **** static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
--- 355,368 ----
TupleTableSlot *slot);
static void store_returning_result(PgFdwModifyState *fmstate,
TupleTableSlot *slot, PGresult *res);
+ static bool dml_is_pushdown_safe(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs);
+ static void execute_dml_stmt(ForeignScanState *node);
+ static TupleTableSlot *get_returning_data(ForeignScanState *node);
static int postgresAcquireSampleRowsFunc(Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
***************
*** 332,341 **** postgres_fdw_handler(PG_FUNCTION_ARGS)
--- 405,419 ----
routine->ExecForeignDelete = postgresExecForeignDelete;
routine->EndForeignModify = postgresEndForeignModify;
routine->IsForeignRelUpdatable = postgresIsForeignRelUpdatable;
+ routine->PlanDMLPushdown = postgresPlanDMLPushdown;
+ routine->BeginDMLPushdown = postgresBeginDMLPushdown;
+ routine->IterateDMLPushdown = postgresIterateDMLPushdown;
+ routine->EndDMLPushdown = postgresEndDMLPushdown;
/* Support functions for EXPLAIN */
routine->ExplainForeignScan = postgresExplainForeignScan;
routine->ExplainForeignModify = postgresExplainForeignModify;
+ routine->ExplainDMLPushdown = postgresExplainDMLPushdown;
/* Support functions for ANALYZE */
routine->AnalyzeForeignTable = postgresAnalyzeForeignTable;
***************
*** 1067,1073 **** postgresGetForeignPlan(PlannerInfo *root,
* Build the fdw_private list that will be available to the executor.
* Items in the list must match enum FdwScanPrivateIndex, above.
*/
! fdw_private = list_make2(makeString(sql.data),
retrieved_attrs);
/*
--- 1145,1152 ----
* Build the fdw_private list that will be available to the executor.
* Items in the list must match enum FdwScanPrivateIndex, above.
*/
! fdw_private = list_make3(makeString(sql.data),
! remote_conds,
retrieved_attrs);
/*
***************
*** 1363,1375 **** postgresAddForeignUpdateTargets(Query *parsetree,
/*
* postgresPlanForeignModify
* Plan an insert/update/delete operation on a foreign table
- *
- * Note: currently, the plan tree generated for UPDATE/DELETE will always
- * include a ForeignScan that retrieves ctids (using SELECT FOR UPDATE)
- * and then the ModifyTable node will have to execute individual remote
- * UPDATE/DELETE commands. If there are no local conditions or joins
- * needed, it'd be better to let the scan node do UPDATE/DELETE RETURNING
- * and then do nothing at ModifyTable. Room for future optimization ...
*/
static List *
postgresPlanForeignModify(PlannerInfo *root,
--- 1442,1447 ----
***************
*** 1882,1887 **** postgresIsForeignRelUpdatable(Relation rel)
--- 1954,2274 ----
}
/*
+ * postgresPlanDMLPushdown
+ * Consider pushing down a foreign table modification to the remote server
+ */
+ static bool
+ postgresPlanDMLPushdown(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index)
+ {
+ CmdType operation = plan->operation;
+ RangeTblEntry *rte = planner_rt_fetch(resultRelation, root);
+ Relation rel;
+ StringInfoData sql;
+ List *targetAttrs = NIL;
+ List *returningList = NIL;
+ List *retrieved_attrs = NIL;
+ ForeignScan *fscan;
+ List *remote_conds;
+ List *params_list = NIL;
+
+ /*
+ * We don't currently support pushing down an insert to the remote server
+ */
+ if (operation == CMD_INSERT)
+ return false;
+
+ initStringInfo(&sql);
+
+ /*
+ * Core code already has some lock on each rel being planned, so we can
+ * use NoLock here.
+ */
+ rel = heap_open(rte->relid, NoLock);
+
+ /*
+ * In an UPDATE, we transmit only columns that were explicitly targets
+ * of the UPDATE, so as to avoid unnecessary data transmission.
+ */
+ if (operation == CMD_UPDATE)
+ {
+ int col;
+
+ col = -1;
+ while ((col = bms_next_member(rte->updatedCols, col)) >= 0)
+ {
+ /* bit numbers are offset by FirstLowInvalidHeapAttributeNumber */
+ AttrNumber attno = col + FirstLowInvalidHeapAttributeNumber;
+
+ if (attno <= InvalidAttrNumber) /* shouldn't happen */
+ elog(ERROR, "system-column update is not supported");
+ targetAttrs = lappend_int(targetAttrs, attno);
+ }
+ }
+
+ /*
+ * Ok, check to see whether the query is safe to push down.
+ */
+ if (!dml_is_pushdown_safe(root, plan,
+ resultRelation,
+ subplan_index,
+ rel, targetAttrs))
+ {
+ heap_close(rel, NoLock);
+ return false;
+ }
+
+ /*
+ * Ok, modify subplan to push down the query.
+ */
+ fscan = (ForeignScan *) list_nth(plan->plans, subplan_index);
+
+ /*
+ * Extract the baserestrictinfo clauses that can be evaluated remotely.
+ */
+ remote_conds = (List *) list_nth(fscan->fdw_private,
+ FdwScanPrivateRemoteConds);
+
+ /*
+ * Extract the relevant RETURNING list if any.
+ */
+ if (plan->returningLists)
+ returningList = (List *) list_nth(plan->returningLists, subplan_index);
+
+ /*
+ * Construct the SQL command string.
+ */
+ switch (operation)
+ {
+ case CMD_UPDATE:
+ deparsePushedDownUpdateSql(&sql, root, resultRelation, rel,
+ ((Plan *) fscan)->targetlist,
+ targetAttrs,
+ remote_conds, ¶ms_list,
+ returningList, &retrieved_attrs);
+ break;
+ case CMD_DELETE:
+ deparsePushedDownDeleteSql(&sql, root, resultRelation, rel,
+ remote_conds, ¶ms_list,
+ returningList, &retrieved_attrs);
+ break;
+ default:
+ elog(ERROR, "unexpected operation: %d", (int) operation);
+ break;
+ }
+
+ /*
+ * Update the operation info.
+ */
+ fscan->operation = operation;
+
+ /*
+ * Update the fdw_exprs list that will be available to the executor.
+ */
+ fscan->fdw_exprs = params_list;
+
+ /*
+ * Update the fdw_private list that will be available to the executor.
+ * Items in the list must match enum FdwDmlPushdownPrivateIndex, above.
+ */
+ fscan->fdw_private = list_make4(makeString(sql.data),
+ makeInteger((retrieved_attrs != NIL)),
+ retrieved_attrs,
+ makeInteger(plan->canSetTag));
+
+ /*
+ * RETURNING expressions might reference the tableoid column, so initialize
+ * t_tableOid before evaluating them (see ForeignNext).
+ */
+ Assert(fscan->fsSystemCol);
+
+ heap_close(rel, NoLock);
+ return true;
+ }
+
+ /*
+ * postgresBeginDMLPushdown
+ * Initiate pushing down a foreign table modification to the remote server
+ */
+ static void
+ postgresBeginDMLPushdown(ForeignScanState *node, int eflags)
+ {
+ ForeignScan *fsplan = (ForeignScan *) node->ss.ps.plan;
+ EState *estate = node->ss.ps.state;
+ PgFdwDmlPushdownState *dpstate;
+ RangeTblEntry *rte;
+ Oid userid;
+ ForeignTable *table;
+ ForeignServer *server;
+ UserMapping *user;
+ int numParams;
+ int i;
+ ListCell *lc;
+
+ /*
+ * Do nothing in EXPLAIN (no ANALYZE) case. node->fdw_state stays NULL.
+ */
+ if (eflags & EXEC_FLAG_EXPLAIN_ONLY)
+ return;
+
+ /*
+ * We'll save private state in node->fdw_state.
+ */
+ dpstate = (PgFdwDmlPushdownState *) palloc0(sizeof(PgFdwDmlPushdownState));
+ node->fdw_state = (void *) dpstate;
+
+ /*
+ * Identify which user to do the remote access as. This should match what
+ * ExecCheckRTEPerms() does.
+ */
+ rte = rt_fetch(fsplan->scan.scanrelid, estate->es_range_table);
+ userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
+
+ /* Get info about foreign table. */
+ dpstate->rel = node->ss.ss_currentRelation;
+ table = GetForeignTable(RelationGetRelid(dpstate->rel));
+ server = GetForeignServer(table->serverid);
+ user = GetUserMapping(userid, server->serverid);
+
+ /*
+ * Get connection to the foreign server. Connection manager will
+ * establish new connection if necessary.
+ */
+ dpstate->conn = GetConnection(server, user, false);
+
+ /* Initialize state variable */
+ dpstate->num_tuples = -1; /* -1 means not set yet */
+
+ /* Get private info created by planner functions. */
+ dpstate->query = strVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateUpdateSql));
+ dpstate->has_returning = intVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateHasReturning));
+ dpstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateRetrievedAttrs);
+ dpstate->set_processed = intVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateSetProcessed));
+
+ /* Create context for per-tuple temp workspace. */
+ dpstate->temp_cxt = AllocSetContextCreate(estate->es_query_cxt,
+ "postgres_fdw temporary data",
+ ALLOCSET_SMALL_MINSIZE,
+ ALLOCSET_SMALL_INITSIZE,
+ ALLOCSET_SMALL_MAXSIZE);
+
+ /* Prepare for input conversion of RETURNING results. */
+ if (dpstate->has_returning)
+ dpstate->attinmeta = TupleDescGetAttInMetadata(RelationGetDescr(dpstate->rel));
+
+ /* Prepare for output conversion of parameters used in remote query. */
+ numParams = list_length(fsplan->fdw_exprs);
+ dpstate->numParams = numParams;
+ dpstate->param_flinfo = (FmgrInfo *) palloc0(sizeof(FmgrInfo) * numParams);
+
+ i = 0;
+ foreach(lc, fsplan->fdw_exprs)
+ {
+ Node *param_expr = (Node *) lfirst(lc);
+ Oid typefnoid;
+ bool isvarlena;
+
+ getTypeOutputInfo(exprType(param_expr), &typefnoid, &isvarlena);
+ fmgr_info(typefnoid, &dpstate->param_flinfo[i]);
+ i++;
+ }
+
+ /*
+ * Prepare remote-parameter expressions for evaluation. (Note: in
+ * practice, we expect that all these expressions will be just Params, so
+ * we could possibly do something more efficient than using the full
+ * expression-eval machinery for this. But probably there would be little
+ * benefit, and it'd require postgres_fdw to know more than is desirable
+ * about Param evaluation.)
+ */
+ dpstate->param_exprs = (List *)
+ ExecInitExpr((Expr *) fsplan->fdw_exprs,
+ (PlanState *) node);
+
+ /*
+ * Allocate buffer for text form of query parameters, if any.
+ */
+ if (numParams > 0)
+ dpstate->param_values = (const char **) palloc0(numParams * sizeof(char *));
+ else
+ dpstate->param_values = NULL;
+ }
+
+ /*
+ * postgresIterateDMLPushdown
+ * Execute pushing down a foreign table modification to the remote server
+ */
+ static TupleTableSlot *
+ postgresIterateDMLPushdown(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ EState *estate = node->ss.ps.state;
+ ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
+
+ /*
+ * If this is the first call after Begin, execute the statement.
+ */
+ if (dpstate->num_tuples == -1)
+ execute_dml_stmt(node);
+
+ /*
+ * If the local query doesn't specify RETURNING, just clear tuple slot.
+ */
+ if (!resultRelInfo->ri_projectReturning)
+ {
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+ Instrumentation *instr = node->ss.ps.instrument;
+
+ Assert(!dpstate->has_returning);
+
+ /* Increment the command es_processed count if necessary. */
+ if (dpstate->set_processed)
+ estate->es_processed += dpstate->num_tuples;
+
+ /* Increment the tuple count for EXPLAIN ANALYZE if necessary. */
+ if (instr)
+ instr->tuplecount += dpstate->num_tuples;
+
+ return ExecClearTuple(slot);
+ }
+
+ /*
+ * Get the next RETURNING tuple.
+ */
+ return get_returning_data(node);
+ }
+
+ /*
+ * postgresEndDMLPushdown
+ * Finish pushing down a foreign table modification to the remote server
+ */
+ static void
+ postgresEndDMLPushdown(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+
+ /* if dpstate is NULL, we are in EXPLAIN; nothing to do */
+ if (dpstate == NULL)
+ return;
+
+ /* Release PGresult */
+ if (dpstate->result)
+ PQclear(dpstate->result);
+
+ /* Release remote connection */
+ ReleaseConnection(dpstate->conn);
+ dpstate->conn = NULL;
+
+ /* MemoryContexts will be deleted automatically. */
+ }
+
+ /*
* postgresExplainForeignScan
* Produce extra output for EXPLAIN of a ForeignScan on a foreign table
*/
***************
*** 1919,1924 **** postgresExplainForeignModify(ModifyTableState *mtstate,
--- 2306,2330 ----
}
}
+ /*
+ * postgresExplainDMLPushdown
+ * Produce extra output for EXPLAIN of a ForeignScan on a foreign table
+ * that has pushed down an UPDATE/DELETE to the remote server
+ */
+ static void
+ postgresExplainDMLPushdown(ForeignScanState *node, ExplainState *es)
+ {
+ List *fdw_private;
+ char *sql;
+
+ if (es->verbose)
+ {
+ fdw_private = ((ForeignScan *) node->ss.ps.plan)->fdw_private;
+ sql = strVal(list_nth(fdw_private, FdwDmlPushdownPrivateUpdateSql));
+ ExplainPropertyText("Remote SQL", sql, es);
+ }
+ }
+
/*
* estimate_path_cost_size
***************
*** 2535,2540 **** store_returning_result(PgFdwModifyState *fmstate,
--- 2941,3134 ----
}
/*
+ * Check to see whether it's safe to push down an UPDATE/DELETE to the remote
+ * server
+ *
+ * Conditions checked here:
+ *
+ * 1. If there are any local joins needed, we mustn't push the command down,
+ * because that breaks execution of the joins.
+ *
+ * 2. If there are any quals that can't be evaluated remotely, we mustn't push
+ * the command down, because that breaks evaluation of the quals.
+ *
+ * 3. If the target relation has any row-level local triggers, we mustn't push
+ * the command down, because that breaks execution of the triggers.
+ *
+ * 4. We can't push an UPDATE down, if any expressions to assign to the target
+ * columns are unsafe to evaluate on the remote server.
+ */
+ static bool
+ dml_is_pushdown_safe(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index,
+ Relation rel,
+ List *targetAttrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[resultRelation];
+ Plan *subplan = (Plan *) list_nth(plan->plans, subplan_index);
+ CmdType operation = plan->operation;
+ ListCell *lc;
+
+ Assert(operation == CMD_UPDATE || operation == CMD_DELETE);
+
+ /* Check point 1 */
+ if (nodeTag(subplan) != T_ForeignScan)
+ return false;
+
+ /* Check point 2 */
+ if (subplan->qual != NIL)
+ return false;
+
+ /* Check point 3 */
+ if (relation_has_row_triggers(rel, operation))
+ return false;
+
+ /* Check point 4 */
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(subplan->targetlist,
+ attnum);
+
+ if (!is_foreign_expr(root, baserel, (Expr *) tle->expr))
+ return false;
+ }
+
+ return true;
+ }
+
+ /*
+ * Execute the pushed-down UPDATE/DELETE statement.
+ */
+ static void
+ execute_dml_stmt(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ ExprContext *econtext = node->ss.ps.ps_ExprContext;
+ int numParams = dpstate->numParams;
+ const char **values = dpstate->param_values;
+
+ /*
+ * Construct array of query parameter values in text format.
+ */
+ if (numParams > 0)
+ {
+ int nestlevel;
+ int i;
+ ListCell *lc;
+
+ nestlevel = set_transmission_modes();
+
+ i = 0;
+ foreach(lc, dpstate->param_exprs)
+ {
+ ExprState *expr_state = (ExprState *) lfirst(lc);
+ Datum expr_value;
+ bool isNull;
+
+ /* Evaluate the parameter expression */
+ expr_value = ExecEvalExpr(expr_state, econtext, &isNull, NULL);
+
+ /*
+ * Get string representation of each parameter value by invoking
+ * type-specific output function, unless the value is null.
+ */
+ if (isNull)
+ values[i] = NULL;
+ else
+ values[i] = OutputFunctionCall(&dpstate->param_flinfo[i],
+ expr_value);
+ i++;
+ }
+
+ reset_transmission_modes(nestlevel);
+ }
+
+ /*
+ * Notice that we pass NULL for paramTypes, thus forcing the remote server
+ * to infer types for all parameters. Since we explicitly cast every
+ * parameter (see deparse.c), the "inference" is trivial and will produce
+ * the desired result. This allows us to avoid assuming that the remote
+ * server has the same OIDs we do for the parameters' types.
+ *
+ * We don't use a PG_TRY block here, so be careful not to throw error
+ * without releasing the PGresult.
+ */
+ dpstate->result = PQexecParams(dpstate->conn, dpstate->query,
+ numParams, NULL, values, NULL, NULL, 0);
+ if (PQresultStatus(dpstate->result) !=
+ (dpstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
+ pgfdw_report_error(ERROR, dpstate->result, dpstate->conn, true,
+ dpstate->query);
+
+ /* Get the number of rows affected. */
+ if (dpstate->has_returning)
+ dpstate->num_tuples = PQntuples(dpstate->result);
+ else
+ dpstate->num_tuples = atoi(PQcmdTuples(dpstate->result));
+ }
+
+ /*
+ * Get the result of a RETURNING clause.
+ */
+ static TupleTableSlot *
+ get_returning_data(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ EState *estate = node->ss.ps.state;
+ ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+
+ Assert(resultRelInfo->ri_projectReturning);
+
+ /* If we didn't get any tuples, must be end of data. */
+ if (dpstate->next_tuple >= dpstate->num_tuples)
+ return ExecClearTuple(slot);
+
+ /* Increment the command es_processed count if necessary. */
+ if (dpstate->set_processed)
+ estate->es_processed += 1;
+
+ /*
+ * Store a RETURNING tuple. Note that if the local query is of the form
+ * e.g., UPDATE/DELETE .. RETURNING 1, we have has_returning=false, so
+ * just emit a dummy tuple in that case.
+ */
+ if (!dpstate->has_returning)
+ ExecStoreAllNullTuple(slot);
+ else
+ {
+ PG_TRY();
+ {
+ HeapTuple newtup;
+
+ newtup = make_tuple_from_result_row(dpstate->result,
+ dpstate->next_tuple,
+ dpstate->rel,
+ dpstate->attinmeta,
+ dpstate->retrieved_attrs,
+ dpstate->temp_cxt);
+ ExecStoreTuple(newtup, slot, InvalidBuffer, false);
+ }
+ PG_CATCH();
+ {
+ if (dpstate->result)
+ PQclear(dpstate->result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ }
+ dpstate->next_tuple++;
+
+ /* Make slot available for evaluation of the local query RETURNING list. */
+ resultRelInfo->ri_projectReturning->pi_exprContext->ecxt_scantuple = slot;
+
+ return slot;
+ }
+
+ /*
* postgresAnalyzeForeignTable
* Test whether analyzing this foreign table is supported
*/
*** a/contrib/postgres_fdw/postgres_fdw.h
--- b/contrib/postgres_fdw/postgres_fdw.h
***************
*** 103,112 **** extern void deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 103,126 ----
Index rtindex, Relation rel,
List *targetAttrs, List *returningList,
List **retrieved_attrs);
+ extern void deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseDeleteSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *returningList,
List **retrieved_attrs);
+ extern void deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseAnalyzeSizeSql(StringInfo buf, Relation rel);
extern void deparseAnalyzeSql(StringInfo buf, Relation rel,
List **retrieved_attrs);
*** a/contrib/postgres_fdw/sql/postgres_fdw.sql
--- b/contrib/postgres_fdw/sql/postgres_fdw.sql
***************
*** 399,419 **** INSERT INTO ft2 (c1,c2,c3) SELECT c1+1000,c2+100, c3 || c3 FROM ft2 LIMIT 20;
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
BEGIN
--- 399,439 ----
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
+ INSERT INTO ft2 (c1,c2,c3)
+ VALUES (1201,1201,'aaa'), (1202,1202,'bbb'), (1203,1203,'ccc') RETURNING *;
+ INSERT INTO ft2 (c1,c2,c3) VALUES (1204,1204,'ddd'), (1205,1205,'eee');
+ PREPARE mt1(int, int, text) AS UPDATE ft2 SET c2 = c2 + $2, c3 = c3 || $3 WHERE c1 % 10 = $1; -- can be pushed down
+ EXPLAIN (verbose, costs off) EXECUTE mt1(1, 0, '_update1');
+ EXECUTE mt1(1, 0, '_update1');
+ PREPARE mt2(int) AS DELETE FROM ft2 WHERE c1 = $1 RETURNING c1, c2, c3; -- can be pushed down
+ EXPLAIN (verbose, costs off) EXECUTE mt2(1201);
+ EXECUTE mt2(1201);
+ PREPARE mt3(int) AS DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft2.c2 > $1 RETURNING ft2.*; -- can't be pushed down
+ EXPLAIN (verbose, costs off) EXECUTE mt3(1201);
+ EXECUTE mt3(1201);
+ DEALLOCATE mt1;
+ DEALLOCATE mt2;
+ DEALLOCATE mt3;
+
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
BEGIN
***************
*** 728,733 **** UPDATE rem1 SET f2 = 'testo';
--- 748,837 ----
-- Test returning a system attribute
INSERT INTO rem1(f2) VALUES ('test') RETURNING ctid;
+ -- cleanup
+ DROP TRIGGER trig_row_before ON rem1;
+ DROP TRIGGER trig_row_after ON rem1;
+ DROP TRIGGER trig_local_before ON loc1;
+
+
+ -- Test DML pushdown functionality
+
+ -- Test with statement-level triggers
+ CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_stmt_before ON rem1;
+
+ CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_stmt_after ON rem1;
+
+ -- Test with row-level ON INSERT triggers
+ CREATE TRIGGER trig_row_before_insert
+ BEFORE INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_before_insert ON rem1;
+
+ CREATE TRIGGER trig_row_after_insert
+ AFTER INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_after_insert ON rem1;
+
+ -- Test with row-level ON UPDATE triggers
+ CREATE TRIGGER trig_row_before_update
+ BEFORE UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_before_update ON rem1;
+
+ CREATE TRIGGER trig_row_after_update
+ AFTER UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_after_update ON rem1;
+
+ -- Test with row-level ON DELETE triggers
+ CREATE TRIGGER trig_row_before_delete
+ BEFORE DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ DROP TRIGGER trig_row_before_delete ON rem1;
+
+ CREATE TRIGGER trig_row_after_delete
+ AFTER DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ DROP TRIGGER trig_row_after_delete ON rem1;
+
-- ===================================================================
-- test inheritance features
-- ===================================================================
***************
*** 859,864 **** fetch from c;
--- 963,975 ----
update bar set f2 = null where current of c;
rollback;
+ explain (verbose, costs off)
+ delete from foo where f1 < 5 returning *;
+ delete from foo where f1 < 5 returning *;
+ explain (verbose, costs off)
+ update bar set f2 = f2 + 100 returning *;
+ update bar set f2 = f2 + 100 returning *;
+
drop table foo cascade;
drop table bar cascade;
drop table loct1;
*** a/doc/src/sgml/fdwhandler.sgml
--- b/doc/src/sgml/fdwhandler.sgml
***************
*** 664,669 **** IsForeignRelUpdatable (Relation rel);
--- 664,793 ----
updatability for display in the <literal>information_schema</> views.)
</para>
+ <para>
+ <programlisting>
+ bool
+ PlanDMLPushdown (PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+ </programlisting>
+
+ Check to see whether a foreign table modification is safe to push down
+ to the remote server. If so, return <literal>true</> after performing
+ planning actions needed for that. Otherwise, return <literal>false</>.
+ This optional function is called during query planning.
+ The parameters are the same as for <function>PlanForeignModify</>.
+ If this function succeeds in pushing down the table modification
+ to the remote server, <function>BeginDMLPushdown</>,
+ <function>IterateDMLPushdown</>, and <function>EndDMLPushdown</> will
+ be called at the execution stage, instead. Otherwise, the table
+ modification will be performed using the above table-updating functions.
+ </para>
+
+ <para>
+ To push down the table modification to the remote server, this function
+ must rewrite the target subplan with a <structname>ForeignScan</> plan
+ node that performs the table modification. The <structfield>operation</>
+ field of the <structname>ForeignScan</> must be set to the
+ <literal>CmdType</> enumeration appropriately; that is,
+ <literal>CMD_UPDATE</> for <command>UPDATE</>,
+ <literal>CMD_INSERT</> for <command>INSERT</>, and
+ <literal>CMD_DELETE</> for <command>DELETE</>.
+ </para>
+
+ <para>
+ See <xref linkend="fdw-planning"> for additional information.
+ </para>
+
+ <para>
+ If the <function>PlanDMLPushdown</> pointer is set to
+ <literal>NULL</>, no attempts to push down the foreign table modification
+ to the remote server are taken.
+ </para>
+
+ <para>
+ <programlisting>
+ void
+ BeginDMLPushdown (ForeignScanState *node,
+ int eflags);
+ </programlisting>
+
+ Begin pushing down a foreign table modification to the remote server.
+ This routine is called during executor startup. It should perform any
+ initialization needed prior to the actual table modification (that
+ should be done upon the first call to <function>IterateDMLPushdown</>).
+ The <structname>ForeignScanState</> node has already been created, but
+ its <structfield>fdw_state</> field is still NULL. Information about
+ the table to update is accessible through the
+ <structname>ForeignScanState</> node (in particular, from the underlying
+ <structname>ForeignScan</> plan node, which contains any FDW-private
+ information provided by <function>PlanDMLPushdown</>).
+ <literal>eflags</> contains flag bits describing the executor's
+ operating mode for this plan node.
+ </para>
+
+ <para>
+ Note that when <literal>(eflags & EXEC_FLAG_EXPLAIN_ONLY)</> is
+ true, this function should not perform any externally-visible actions;
+ it should only do the minimum required to make the node state valid
+ for <function>ExplainDMLPushdown</> and <function>EndDMLPushdown</>.
+ </para>
+
+ <para>
+ If the <function>BeginDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to push down the foreign table modification
+ to the remote server will fail with an error message.
+ </para>
+
+ <para>
+ <programlisting>
+ TupleTableSlot *
+ IterateForeignScan (ForeignScanState *node);
+ </programlisting>
+
+ When the local query has a <literal>RETURNING</> clause, return one row
+ containing the data that was actually inserted, updated, or deleted, in
+ a tuple table slot (the node's <structfield>ScanTupleSlot</> should be
+ used for this purpose). Return NULL if no more rows are available.
+ When the query doesn't have the clause, return NULL. Note that this is
+ called in a short-lived memory context that will be reset between
+ invocations. Create a memory context in <function>BeginDMLPushdown</>
+ if you need longer-lived storage, or use the <structfield>es_query_cxt</>
+ of the node's <structname>EState</>.
+ </para>
+
+ <para>
+ The data in the returned slot is used only if the local query has
+ a <literal>RETURNING</> clause. The FDW could choose to optimize away
+ returning some or all columns depending on the contents of the
+ <literal>RETURNING</> clause. Regardless, some slot must be returned to
+ indicate success, or the query's reported row count will be wrong.
+ </para>
+
+ <para>
+ If the <function>IterateDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to push down the foreign table modification
+ to the remote server will fail with an error message.
+ </para>
+
+ <para>
+ <programlisting>
+ void
+ EndDMLPushdown (ForeignScanState *node);
+ </programlisting>
+
+ End the table update and release resources. It is normally not important
+ to release palloc'd memory, but for example open files and connections
+ to remote servers should be cleaned up.
+ </para>
+
+ <para>
+ If the <function>EndDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to push down the foreign table modification
+ to the remote server will fail with an error message.
+ </para>
+
</sect2>
<sect2 id="fdw-callbacks-row-locking">
***************
*** 848,853 **** ExplainForeignModify (ModifyTableState *mtstate,
--- 972,1000 ----
<command>EXPLAIN</>.
</para>
+ <para>
+ <programlisting>
+ void
+ ExplainDMLPushdown (ForeignScanState *node,
+ ExplainState *es);
+ </programlisting>
+
+ Print additional <command>EXPLAIN</> output for a foreign table update
+ that has been pushed down to the remote server.
+ This function can call <function>ExplainPropertyText</> and
+ related functions to add fields to the <command>EXPLAIN</> output.
+ The flag fields in <literal>es</> can be used to determine what to
+ print, and the state of the <structname>ForeignScanState</> node
+ can be inspected to provide run-time statistics in the <command>EXPLAIN
+ ANALYZE</> case.
+ </para>
+
+ <para>
+ If the <function>ExplainDMLPushdown</> pointer is set to
+ <literal>NULL</>, no additional information is printed during
+ <command>EXPLAIN</>.
+ </para>
+
</sect2>
<sect2 id="fdw-callbacks-analyze">
***************
*** 1068,1074 **** GetForeignServerByName(const char *name, bool missing_ok);
<para>
The FDW callback functions <function>GetForeignRelSize</>,
<function>GetForeignPaths</>, <function>GetForeignPlan</>,
! <function>PlanForeignModify</>, and <function>GetForeignJoinPaths</>
must fit into the workings of the <productname>PostgreSQL</> planner.
Here are some notes about what they must do.
</para>
--- 1215,1222 ----
<para>
The FDW callback functions <function>GetForeignRelSize</>,
<function>GetForeignPaths</>, <function>GetForeignPlan</>,
! <function>PlanForeignModify</>, <function>PlanDMLPushdown</>, and
! <function>GetForeignJoinPaths</>
must fit into the workings of the <productname>PostgreSQL</> planner.
Here are some notes about what they must do.
</para>
***************
*** 1228,1237 **** GetForeignServerByName(const char *name, bool missing_ok);
<para>
When planning an <command>UPDATE</> or <command>DELETE</>,
! <function>PlanForeignModify</> can look up the <structname>RelOptInfo</>
! struct for the foreign table and make use of the
! <literal>baserel->fdw_private</> data previously created by the
! scan-planning functions. However, in <command>INSERT</> the target
table is not scanned so there is no <structname>RelOptInfo</> for it.
The <structname>List</> returned by <function>PlanForeignModify</> has
the same restrictions as the <structfield>fdw_private</> list of a
--- 1376,1386 ----
<para>
When planning an <command>UPDATE</> or <command>DELETE</>,
! <function>PlanForeignModify</> and <function>PlanDMLPushdown</> can
! look up the <structname>RelOptInfo</> struct for the foreign table
! and make use of the <literal>baserel->fdw_private</> data previously
! created by the scan-planning functions.
! However, in <command>INSERT</> the target
table is not scanned so there is no <structname>RelOptInfo</> for it.
The <structname>List</> returned by <function>PlanForeignModify</> has
the same restrictions as the <structfield>fdw_private</> list of a
*** a/doc/src/sgml/postgres-fdw.sgml
--- b/doc/src/sgml/postgres-fdw.sgml
***************
*** 471,476 ****
--- 471,485 ----
extension that's listed in the foreign server's <literal>extensions</>
option. Operators and functions in such clauses must
be <literal>IMMUTABLE</> as well.
+ For an <command>UPDATE</> or <command>DELETE</> query,
+ <filename>postgres_fdw</> attempts to optimize the query execution by
+ sending the whole query to the remote server if there are no query
+ <literal>WHERE</> clauses that cannot be sent to the remote server,
+ no local joins for the query, or no row-level local <literal>BEFORE</> or
+ <literal>AFTER</> triggers on the target table. In <command>UPDATE</>,
+ expressions to assign to target columns must use only built-in data types,
+ <literal>IMMUTABLE</> operators, or <literal>IMMUTABLE</> functions,
+ to reduce the risk of misexecution of the query.
</para>
<para>
*** a/src/backend/commands/explain.c
--- b/src/backend/commands/explain.c
***************
*** 888,894 **** ExplainNode(PlanState *planstate, List *ancestors,
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
! pname = sname = "Foreign Scan";
break;
case T_CustomScan:
sname = "Custom Scan";
--- 888,916 ----
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
! sname = "Foreign Scan";
! switch (((ForeignScan *) plan)->operation)
! {
! case CMD_SELECT:
! pname = "Foreign Scan";
! operation = "Select";
! break;
! case CMD_INSERT:
! pname = "Foreign Insert";
! operation = "Insert";
! break;
! case CMD_UPDATE:
! pname = "Foreign Update";
! operation = "Update";
! break;
! case CMD_DELETE:
! pname = "Foreign Delete";
! operation = "Delete";
! break;
! default:
! pname = "???";
! break;
! }
break;
case T_CustomScan:
sname = "Custom Scan";
***************
*** 1624,1629 **** show_plan_tlist(PlanState *planstate, List *ancestors, ExplainState *es)
--- 1646,1657 ----
return;
if (IsA(plan, RecursiveUnion))
return;
+ /* Likewise for ForeignScan that has pushed down INSERT/UPDATE/DELETE */
+ if (IsA(plan, ForeignScan) &&
+ (((ForeignScan *) plan)->operation == CMD_INSERT ||
+ ((ForeignScan *) plan)->operation == CMD_UPDATE ||
+ ((ForeignScan *) plan)->operation == CMD_DELETE))
+ return;
/* Set up deparsing context */
context = set_deparse_context_planstate(es->deparse_cxt,
***************
*** 2212,2219 **** show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es)
FdwRoutine *fdwroutine = fsstate->fdwroutine;
/* Let the FDW emit whatever fields it wants */
! if (fdwroutine->ExplainForeignScan != NULL)
! fdwroutine->ExplainForeignScan(fsstate, es);
}
/*
--- 2240,2255 ----
FdwRoutine *fdwroutine = fsstate->fdwroutine;
/* Let the FDW emit whatever fields it wants */
! if (((ForeignScan *) fsstate->ss.ps.plan)->operation != CMD_SELECT)
! {
! if (fdwroutine->ExplainDMLPushdown != NULL)
! fdwroutine->ExplainDMLPushdown(fsstate, es);
! }
! else
! {
! if (fdwroutine->ExplainForeignScan != NULL)
! fdwroutine->ExplainForeignScan(fsstate, es);
! }
}
/*
***************
*** 2599,2606 **** show_modifytable_info(ModifyTableState *mtstate, List *ancestors,
}
}
! /* Give FDW a chance */
! if (fdwroutine && fdwroutine->ExplainForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, j);
--- 2635,2644 ----
}
}
! /* Give FDW a chance if needed */
! if (!resultRelInfo->ri_FdwPushdown &&
! fdwroutine != NULL &&
! fdwroutine->ExplainForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, j);
*** a/src/backend/executor/execMain.c
--- b/src/backend/executor/execMain.c
***************
*** 1011,1020 **** InitPlan(QueryDesc *queryDesc, int eflags)
* CheckValidRowMarkRel.
*/
void
! CheckValidResultRel(Relation resultRel, CmdType operation)
{
TriggerDesc *trigDesc = resultRel->trigdesc;
FdwRoutine *fdwroutine;
switch (resultRel->rd_rel->relkind)
{
--- 1011,1022 ----
* CheckValidRowMarkRel.
*/
void
! CheckValidResultRel(ResultRelInfo *resultRelInfo, CmdType operation)
{
+ Relation resultRel = resultRelInfo->ri_RelationDesc;
TriggerDesc *trigDesc = resultRel->trigdesc;
FdwRoutine *fdwroutine;
+ bool allow_pushdown;
switch (resultRel->rd_rel->relkind)
{
***************
*** 1083,1096 **** CheckValidResultRel(Relation resultRel, CmdType operation)
case RELKIND_FOREIGN_TABLE:
/* Okay only if the FDW supports it */
fdwroutine = GetFdwRoutineForRelation(resultRel, false);
switch (operation)
{
case CMD_INSERT:
! if (fdwroutine->ExecForeignInsert == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("cannot insert into foreign table \"%s\"",
! RelationGetRelationName(resultRel))));
if (fdwroutine->IsForeignRelUpdatable != NULL &&
(fdwroutine->IsForeignRelUpdatable(resultRel) & (1 << CMD_INSERT)) == 0)
ereport(ERROR,
--- 1085,1107 ----
case RELKIND_FOREIGN_TABLE:
/* Okay only if the FDW supports it */
fdwroutine = GetFdwRoutineForRelation(resultRel, false);
+ allow_pushdown = ((fdwroutine->BeginDMLPushdown != NULL) &&
+ (fdwroutine->IterateDMLPushdown != NULL) &&
+ (fdwroutine->EndDMLPushdown != NULL));
switch (operation)
{
case CMD_INSERT:
! if (resultRelInfo->ri_FdwPushdown && !allow_pushdown)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("cannot push down insert on foreign table \"%s\"",
! RelationGetRelationName(resultRel))));
! if (!resultRelInfo->ri_FdwPushdown &&
! fdwroutine->ExecForeignInsert == NULL)
! ereport(ERROR,
! (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("cannot insert into foreign table \"%s\"",
! RelationGetRelationName(resultRel))));
if (fdwroutine->IsForeignRelUpdatable != NULL &&
(fdwroutine->IsForeignRelUpdatable(resultRel) & (1 << CMD_INSERT)) == 0)
ereport(ERROR,
***************
*** 1099,1105 **** CheckValidResultRel(Relation resultRel, CmdType operation)
RelationGetRelationName(resultRel))));
break;
case CMD_UPDATE:
! if (fdwroutine->ExecForeignUpdate == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot update foreign table \"%s\"",
--- 1110,1122 ----
RelationGetRelationName(resultRel))));
break;
case CMD_UPDATE:
! if (resultRelInfo->ri_FdwPushdown && !allow_pushdown)
! ereport(ERROR,
! (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("cannot push down update on foreign table \"%s\"",
! RelationGetRelationName(resultRel))));
! if (!resultRelInfo->ri_FdwPushdown &&
! fdwroutine->ExecForeignUpdate == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot update foreign table \"%s\"",
***************
*** 1112,1118 **** CheckValidResultRel(Relation resultRel, CmdType operation)
RelationGetRelationName(resultRel))));
break;
case CMD_DELETE:
! if (fdwroutine->ExecForeignDelete == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot delete from foreign table \"%s\"",
--- 1129,1141 ----
RelationGetRelationName(resultRel))));
break;
case CMD_DELETE:
! if (resultRelInfo->ri_FdwPushdown && !allow_pushdown)
! ereport(ERROR,
! (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("cannot push down delete on foreign table \"%s\"",
! RelationGetRelationName(resultRel))));
! if (!resultRelInfo->ri_FdwPushdown &&
! fdwroutine->ExecForeignDelete == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot delete from foreign table \"%s\"",
***************
*** 1245,1250 **** InitResultRelInfo(ResultRelInfo *resultRelInfo,
--- 1268,1274 ----
else
resultRelInfo->ri_FdwRoutine = NULL;
resultRelInfo->ri_FdwState = NULL;
+ resultRelInfo->ri_FdwPushdown = false;
resultRelInfo->ri_ConstraintExprs = NULL;
resultRelInfo->ri_junkFilter = NULL;
resultRelInfo->ri_projectReturning = NULL;
*** a/src/backend/executor/nodeForeignscan.c
--- b/src/backend/executor/nodeForeignscan.c
***************
*** 48,54 **** ForeignNext(ForeignScanState *node)
/* Call the Iterate function in short-lived context */
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! slot = node->fdwroutine->IterateForeignScan(node);
MemoryContextSwitchTo(oldcontext);
/*
--- 48,57 ----
/* Call the Iterate function in short-lived context */
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! if (plan->operation != CMD_SELECT)
! slot = node->fdwroutine->IterateDMLPushdown(node);
! else
! slot = node->fdwroutine->IterateForeignScan(node);
MemoryContextSwitchTo(oldcontext);
/*
***************
*** 226,232 **** ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
/*
* Tell the FDW to initialize the scan.
*/
! fdwroutine->BeginForeignScan(scanstate, eflags);
return scanstate;
}
--- 229,238 ----
/*
* Tell the FDW to initialize the scan.
*/
! if (node->operation != CMD_SELECT)
! fdwroutine->BeginDMLPushdown(scanstate, eflags);
! else
! fdwroutine->BeginForeignScan(scanstate, eflags);
return scanstate;
}
***************
*** 240,247 **** ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
void
ExecEndForeignScan(ForeignScanState *node)
{
/* Let the FDW shut down */
! node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
--- 246,258 ----
void
ExecEndForeignScan(ForeignScanState *node)
{
+ ForeignScan *plan = (ForeignScan *) node->ss.ps.plan;
+
/* Let the FDW shut down */
! if (plan->operation != CMD_SELECT)
! node->fdwroutine->EndDMLPushdown(node);
! else
! node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
*** a/src/backend/executor/nodeModifyTable.c
--- b/src/backend/executor/nodeModifyTable.c
***************
*** 138,143 **** ExecCheckPlanOutput(Relation resultRel, List *targetList)
--- 138,146 ----
* tupleSlot: slot holding tuple actually inserted/updated/deleted
* planSlot: slot holding tuple returned by top subplan node
*
+ * Note: If tupleSlot is NULL, the FDW should have already provided econtext's
+ * scan tuple.
+ *
* Returns a slot holding the result tuple
*/
static TupleTableSlot *
***************
*** 154,160 **** ExecProcessReturning(ProjectionInfo *projectReturning,
ResetExprContext(econtext);
/* Make tuple and any needed join variables available to ExecProject */
! econtext->ecxt_scantuple = tupleSlot;
econtext->ecxt_outertuple = planSlot;
/* Compute the RETURNING expressions */
--- 157,166 ----
ResetExprContext(econtext);
/* Make tuple and any needed join variables available to ExecProject */
! if (tupleSlot)
! econtext->ecxt_scantuple = tupleSlot;
! else
! Assert(econtext->ecxt_scantuple);
econtext->ecxt_outertuple = planSlot;
/* Compute the RETURNING expressions */
***************
*** 1357,1362 **** ExecModifyTable(ModifyTableState *node)
--- 1363,1384 ----
break;
}
+ /*
+ * If ri_FdwPushdown is true, all we need to do here is compute the
+ * RETURNING expressions.
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ Assert(resultRelInfo->ri_projectReturning);
+
+ /* No need to provide scan tuple (see ExecProcessReturning) */
+ slot = ExecProcessReturning(resultRelInfo->ri_projectReturning,
+ NULL, planSlot);
+
+ estate->es_result_relation_info = saved_resultRelInfo;
+ return slot;
+ }
+
EvalPlanQualSetSlot(&node->mt_epqstate, planSlot);
slot = planSlot;
***************
*** 1536,1545 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
{
subplan = (Plan *) lfirst(l);
/*
* Verify result relation is a valid target for the current operation
*/
! CheckValidResultRel(resultRelInfo->ri_RelationDesc, operation);
/*
* If there are indices on the result relation, open them and save
--- 1558,1570 ----
{
subplan = (Plan *) lfirst(l);
+ /* Initialize the FdwPushdown flag */
+ resultRelInfo->ri_FdwPushdown = list_nth_int(node->fdwPushdowns, i);
+
/*
* Verify result relation is a valid target for the current operation
*/
! CheckValidResultRel(resultRelInfo, operation);
/*
* If there are indices on the result relation, open them and save
***************
*** 1560,1566 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
/* Also let FDWs init themselves for foreign-table result rels */
! if (resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, i);
--- 1585,1592 ----
mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
/* Also let FDWs init themselves for foreign-table result rels */
! if (!resultRelInfo->ri_FdwPushdown &&
! resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, i);
***************
*** 1731,1743 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
--- 1757,1781 ----
erm = ExecFindRowMark(estate, rc->rti, false);
/* build ExecAuxRowMark for each subplan */
+ resultRelInfo = mtstate->resultRelInfo;
for (i = 0; i < nplans; i++)
{
ExecAuxRowMark *aerm;
+ /*
+ * ignore subplan if the FDW pushes the command down to the remote
+ * server
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ resultRelInfo++;
+ continue;
+ }
+
subplan = mtstate->mt_plans[i]->plan;
aerm = ExecBuildAuxRowMark(erm, subplan->targetlist);
mtstate->mt_arowmarks[i] = lappend(mtstate->mt_arowmarks[i], aerm);
+ resultRelInfo++;
}
}
***************
*** 1798,1803 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
--- 1836,1851 ----
ExecCheckPlanOutput(resultRelInfo->ri_RelationDesc,
subplan->targetlist);
+ /*
+ * ignore subplan if the FDW pushes the command down to the
+ * remote server
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ resultRelInfo++;
+ continue;
+ }
+
j = ExecInitJunkFilter(subplan->targetlist,
resultRelInfo->ri_RelationDesc->rd_att->tdhasoid,
ExecInitExtraTupleSlot(estate));
***************
*** 1887,1893 **** ExecEndModifyTable(ModifyTableState *node)
{
ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
! if (resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->EndForeignModify != NULL)
resultRelInfo->ri_FdwRoutine->EndForeignModify(node->ps.state,
resultRelInfo);
--- 1935,1942 ----
{
ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
! if (!resultRelInfo->ri_FdwPushdown &&
! resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->EndForeignModify != NULL)
resultRelInfo->ri_FdwRoutine->EndForeignModify(node->ps.state,
resultRelInfo);
*** a/src/backend/nodes/copyfuncs.c
--- b/src/backend/nodes/copyfuncs.c
***************
*** 186,191 **** _copyModifyTable(const ModifyTable *from)
--- 186,192 ----
COPY_NODE_FIELD(withCheckOptionLists);
COPY_NODE_FIELD(returningLists);
COPY_NODE_FIELD(fdwPrivLists);
+ COPY_NODE_FIELD(fdwPushdowns);
COPY_NODE_FIELD(rowMarks);
COPY_SCALAR_FIELD(epqParam);
COPY_SCALAR_FIELD(onConflictAction);
***************
*** 645,650 **** _copyForeignScan(const ForeignScan *from)
--- 646,652 ----
/*
* copy remainder of node
*/
+ COPY_SCALAR_FIELD(operation);
COPY_SCALAR_FIELD(fs_server);
COPY_NODE_FIELD(fdw_exprs);
COPY_NODE_FIELD(fdw_private);
*** a/src/backend/nodes/outfuncs.c
--- b/src/backend/nodes/outfuncs.c
***************
*** 340,345 **** _outModifyTable(StringInfo str, const ModifyTable *node)
--- 340,346 ----
WRITE_NODE_FIELD(withCheckOptionLists);
WRITE_NODE_FIELD(returningLists);
WRITE_NODE_FIELD(fdwPrivLists);
+ WRITE_NODE_FIELD(fdwPushdowns);
WRITE_NODE_FIELD(rowMarks);
WRITE_INT_FIELD(epqParam);
WRITE_ENUM_FIELD(onConflictAction, OnConflictAction);
***************
*** 591,596 **** _outForeignScan(StringInfo str, const ForeignScan *node)
--- 592,598 ----
_outScanInfo(str, (const Scan *) node);
+ WRITE_ENUM_FIELD(operation, CmdType);
WRITE_OID_FIELD(fs_server);
WRITE_NODE_FIELD(fdw_exprs);
WRITE_NODE_FIELD(fdw_private);
*** a/src/backend/nodes/readfuncs.c
--- b/src/backend/nodes/readfuncs.c
***************
*** 1471,1476 **** _readModifyTable(void)
--- 1471,1477 ----
READ_NODE_FIELD(withCheckOptionLists);
READ_NODE_FIELD(returningLists);
READ_NODE_FIELD(fdwPrivLists);
+ READ_NODE_FIELD(fdwPushdowns);
READ_NODE_FIELD(rowMarks);
READ_INT_FIELD(epqParam);
READ_ENUM_FIELD(onConflictAction, OnConflictAction);
*** a/src/backend/optimizer/plan/createplan.c
--- b/src/backend/optimizer/plan/createplan.c
***************
*** 3768,3773 **** make_foreignscan(List *qptlist,
--- 3768,3774 ----
plan->lefttree = outer_plan;
plan->righttree = NULL;
node->scan.scanrelid = scanrelid;
+ node->operation = CMD_SELECT;
/* fs_server will be filled in by create_foreignscan_plan */
node->fs_server = InvalidOid;
node->fdw_exprs = fdw_exprs;
***************
*** 5043,5048 **** make_modifytable(PlannerInfo *root,
--- 5044,5050 ----
Plan *plan = &node->plan;
double total_size;
List *fdw_private_list;
+ List *fdwpushdown_list;
ListCell *subnode;
ListCell *lc;
int i;
***************
*** 5123,5134 **** make_modifytable(PlannerInfo *root,
--- 5125,5138 ----
* construct private plan data, and accumulate it all into a list.
*/
fdw_private_list = NIL;
+ fdwpushdown_list = NIL;
i = 0;
foreach(lc, resultRelations)
{
Index rti = lfirst_int(lc);
FdwRoutine *fdwroutine;
List *fdw_private;
+ bool fdwpushdown;
/*
* If possible, we want to get the FdwRoutine from our RelOptInfo for
***************
*** 5156,5161 **** make_modifytable(PlannerInfo *root,
--- 5160,5173 ----
}
if (fdwroutine != NULL &&
+ fdwroutine->PlanDMLPushdown != NULL)
+ fdwpushdown = fdwroutine->PlanDMLPushdown(root, node, rti, i);
+ else
+ fdwpushdown = false;
+ fdwpushdown_list = lappend_int(fdwpushdown_list, fdwpushdown);
+
+ if (!fdwpushdown &&
+ fdwroutine != NULL &&
fdwroutine->PlanForeignModify != NULL)
fdw_private = fdwroutine->PlanForeignModify(root, node, rti, i);
else
***************
*** 5164,5169 **** make_modifytable(PlannerInfo *root,
--- 5176,5182 ----
i++;
}
node->fdwPrivLists = fdw_private_list;
+ node->fdwPushdowns = fdwpushdown_list;
return node;
}
*** a/src/backend/rewrite/rewriteHandler.c
--- b/src/backend/rewrite/rewriteHandler.c
***************
*** 2530,2535 **** relation_is_updatable(Oid reloid,
--- 2530,2571 ----
/*
+ * relation_has_row_triggers - does relation have row level triggers for event?
+ */
+ bool
+ relation_has_row_triggers(Relation rel, CmdType event)
+ {
+ TriggerDesc *trigDesc = rel->trigdesc;
+
+ switch (event)
+ {
+ case CMD_INSERT:
+ if (trigDesc &&
+ (trigDesc->trig_insert_after_row ||
+ trigDesc->trig_insert_before_row))
+ return true;
+ break;
+ case CMD_UPDATE:
+ if (trigDesc &&
+ (trigDesc->trig_update_after_row ||
+ trigDesc->trig_update_before_row))
+ return true;
+ break;
+ case CMD_DELETE:
+ if (trigDesc &&
+ (trigDesc->trig_delete_after_row ||
+ trigDesc->trig_delete_before_row))
+ return true;
+ break;
+ default:
+ elog(ERROR, "unrecognized CmdType: %d", (int) event);
+ break;
+ }
+ return false;
+ }
+
+
+ /*
* adjust_view_column_set - map a set of column numbers according to targetlist
*
* This is used with simply-updatable views to map column-permissions sets for
*** a/src/include/executor/executor.h
--- b/src/include/executor/executor.h
***************
*** 184,190 **** extern void ExecutorEnd(QueryDesc *queryDesc);
extern void standard_ExecutorEnd(QueryDesc *queryDesc);
extern void ExecutorRewind(QueryDesc *queryDesc);
extern bool ExecCheckRTPerms(List *rangeTable, bool ereport_on_violation);
! extern void CheckValidResultRel(Relation resultRel, CmdType operation);
extern void InitResultRelInfo(ResultRelInfo *resultRelInfo,
Relation resultRelationDesc,
Index resultRelationIndex,
--- 184,190 ----
extern void standard_ExecutorEnd(QueryDesc *queryDesc);
extern void ExecutorRewind(QueryDesc *queryDesc);
extern bool ExecCheckRTPerms(List *rangeTable, bool ereport_on_violation);
! extern void CheckValidResultRel(ResultRelInfo *resultRelInfo, CmdType operation);
extern void InitResultRelInfo(ResultRelInfo *resultRelInfo,
Relation resultRelationDesc,
Index resultRelationIndex,
*** a/src/include/foreign/fdwapi.h
--- b/src/include/foreign/fdwapi.h
***************
*** 93,98 **** typedef void (*EndForeignModify_function) (EState *estate,
--- 93,110 ----
typedef int (*IsForeignRelUpdatable_function) (Relation rel);
+ typedef bool (*PlanDMLPushdown_function) (PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+
+ typedef void (*BeginDMLPushdown_function) (ForeignScanState *node,
+ int eflags);
+
+ typedef TupleTableSlot *(*IterateDMLPushdown_function) (ForeignScanState *node);
+
+ typedef void (*EndDMLPushdown_function) (ForeignScanState *node);
+
typedef RowMarkType (*GetForeignRowMarkType_function) (RangeTblEntry *rte,
LockClauseStrength strength);
***************
*** 110,115 **** typedef void (*ExplainForeignModify_function) (ModifyTableState *mtstate,
--- 122,130 ----
int subplan_index,
struct ExplainState *es);
+ typedef void (*ExplainDMLPushdown_function) (ForeignScanState *node,
+ struct ExplainState *es);
+
typedef int (*AcquireSampleRowsFunc) (Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
***************
*** 162,167 **** typedef struct FdwRoutine
--- 177,186 ----
ExecForeignDelete_function ExecForeignDelete;
EndForeignModify_function EndForeignModify;
IsForeignRelUpdatable_function IsForeignRelUpdatable;
+ PlanDMLPushdown_function PlanDMLPushdown;
+ BeginDMLPushdown_function BeginDMLPushdown;
+ IterateDMLPushdown_function IterateDMLPushdown;
+ EndDMLPushdown_function EndDMLPushdown;
/* Functions for SELECT FOR UPDATE/SHARE row locking */
GetForeignRowMarkType_function GetForeignRowMarkType;
***************
*** 171,176 **** typedef struct FdwRoutine
--- 190,196 ----
/* Support functions for EXPLAIN */
ExplainForeignScan_function ExplainForeignScan;
ExplainForeignModify_function ExplainForeignModify;
+ ExplainDMLPushdown_function ExplainDMLPushdown;
/* Support functions for ANALYZE */
AnalyzeForeignTable_function AnalyzeForeignTable;
*** a/src/include/nodes/execnodes.h
--- b/src/include/nodes/execnodes.h
***************
*** 311,316 **** typedef struct JunkFilter
--- 311,317 ----
* TrigInstrument optional runtime measurements for triggers
* FdwRoutine FDW callback functions, if foreign table
* FdwState available to save private state of FDW
+ * FdwPushdown true when the command is pushed down
* WithCheckOptions list of WithCheckOption's to be checked
* WithCheckOptionExprs list of WithCheckOption expr states
* ConstraintExprs array of constraint-checking expr states
***************
*** 334,339 **** typedef struct ResultRelInfo
--- 335,341 ----
Instrumentation *ri_TrigInstrument;
struct FdwRoutine *ri_FdwRoutine;
void *ri_FdwState;
+ bool ri_FdwPushdown;
List *ri_WithCheckOptions;
List *ri_WithCheckOptionExprs;
List **ri_ConstraintExprs;
*** a/src/include/nodes/plannodes.h
--- b/src/include/nodes/plannodes.h
***************
*** 188,193 **** typedef struct ModifyTable
--- 188,194 ----
List *withCheckOptionLists; /* per-target-table WCO lists */
List *returningLists; /* per-target-table RETURNING tlists */
List *fdwPrivLists; /* per-target-table FDW private data lists */
+ List *fdwPushdowns; /* per-target-table FDW pushdown flags */
List *rowMarks; /* PlanRowMarks (non-locking only) */
int epqParam; /* ID of Param for EvalPlanQual re-eval */
OnConflictAction onConflictAction; /* ON CONFLICT action */
***************
*** 530,535 **** typedef struct WorkTableScan
--- 531,537 ----
typedef struct ForeignScan
{
Scan scan;
+ CmdType operation; /* SELECT/INSERT/UPDATE/DELETE */
Oid fs_server; /* OID of foreign server */
List *fdw_exprs; /* expressions that FDW may evaluate */
List *fdw_private; /* private data for FDW */
*** a/src/include/rewrite/rewriteHandler.h
--- b/src/include/rewrite/rewriteHandler.h
***************
*** 29,33 **** extern const char *view_query_is_auto_updatable(Query *viewquery,
--- 29,34 ----
extern int relation_is_updatable(Oid reloid,
bool include_triggers,
Bitmapset *include_cols);
+ extern bool relation_has_row_triggers(Relation rel, CmdType event);
#endif /* REWRITEHANDLER_H */
On Thu, Jan 7, 2016 at 6:15 PM, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>
wrote:
On 2016/01/06 18:58, Rushabh Lathia wrote:
I started looking at updated patch and its definitely iked the new
approach.Thanks for the review!
With the initial look and test overall things looking great, I am still
reviewing the code changes but here are few early doubts/questions:
.) What the need of following change ?
@@ -833,9 +833,6 @@ appendWhereClause(StringInfo buf,
int nestlevel;
ListCell *lc;- if (params) - *params = NIL; /* initialize result list to empty */ - /* Set up context struct for recursion */ context.root = root; context.foreignrel = baserel; @@ -971,6 +968,63 @@ deparseUpdateSql(StringInfo buf, PlannerInfo *root, }It is needed for deparsePushedDownUpdateSql to store params in both WHERE
clauses and expressions to assign to the target columns
into one params_list list.
Hmm sorry but I am still not getting the point, can you provide some
example to explain this ?
.) When Tom Lane and Stephen Frost suggested getting the core code involved,
I thought that we can do the mandatory checks into core it self and making
completely out of dml_is_pushdown_safe(). Please correct meThe reason why I put that function in postgres_fdw.c is Check point 4:
+ * 4. We can't push an UPDATE down, if any expressions to assign to the target + * columns are unsafe to evaluate on the remote server.
Here I was talking about checks related to triggers, or to LIMIT. I think
earlier thread talked about those mandatory check to the core. So may
be we can move those checks into make_modifytable() before calling
the PlanDMLPushdown.
I think this depends on the capabilities of the FDW.
.) Documentation for the new API is missing (fdw-callbacks).
Will add the docs.
--
Rushabh Lathia
On 8 January 2016 at 05:08, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp> wrote:
On 2016/01/07 21:50, Etsuro Fujita wrote:
On 2016/01/06 20:37, Thom Brown wrote:
On 25 December 2015 at 10:00, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:Attached is an updated version of the patch, which is
still WIP, but I'd be happy if I could get any feedback.I've run into an issue:
*# UPDATE master_customers SET id = 22 WHERE id = 16 RETURNING
tableoid::regclass;
ERROR:
CONTEXT: Remote SQL command: UPDATE public.customers SET id = 22
WHERE ((id = 16)) RETURNING NULLWill fix.
While working on this, I noticed that the existing postgres_fdw system shows
similar behavior, so I changed the subject.IIUC, the reason for that is when the local query specifies "RETURNING
tableoid::regclass", the FDW has fmstate->has_returning=false while the
remote query executed at ModifyTable has "RETURNING NULL", as shown in the
above example; that would cause an abnormal exit in executing the remote
query in postgresExecForeignUpdate, since that the FDW would get
PGRES_TUPLES_OK as a result of the query while the FDW would think that the
right result to get should be PGRES_COMMAND_OK, from the flag
fmstate->has_returning=false.Attached is a patch to fix that.
I can't apply this patch in tandem with FDW DML pushdown patch (either
v2 or v3).
Thom
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/01/12 20:36, Thom Brown wrote:
On 8 January 2016 at 05:08, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp> wrote:
On 2016/01/06 20:37, Thom Brown wrote:
I've run into an issue:
*# UPDATE master_customers SET id = 22 WHERE id = 16 RETURNING
tableoid::regclass;
ERROR:
CONTEXT: Remote SQL command: UPDATE public.customers SET id = 22
WHERE ((id = 16)) RETURNING NULL
While working on this, I noticed that the existing postgres_fdw system shows
similar behavior, so I changed the subject.IIUC, the reason for that is when the local query specifies "RETURNING
tableoid::regclass", the FDW has fmstate->has_returning=false while the
remote query executed at ModifyTable has "RETURNING NULL", as shown in the
above example; that would cause an abnormal exit in executing the remote
query in postgresExecForeignUpdate, since that the FDW would get
PGRES_TUPLES_OK as a result of the query while the FDW would think that the
right result to get should be PGRES_COMMAND_OK, from the flag
fmstate->has_returning=false.
Attached is a patch to fix that.
I can't apply this patch in tandem with FDW DML pushdown patch (either
v2 or v3).
That patch is for fixing the similar issue in the existing postgres_fdw
system. So, please apply that patch without the DML pushdown patch. If
that patch is reasonable as a fix for the issue, I'll update the DML
pushdown patch (v3) on top of that patch.
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/01/12 20:31, Rushabh Lathia wrote:
On Thu, Jan 7, 2016 at 6:15 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp <mailto:fujita.etsuro@lab.ntt.co.jp>> wrote:
On 2016/01/06 18:58, Rushabh Lathia wrote:
.) What the need of following change ?@@ -833,9 +833,6 @@ appendWhereClause(StringInfo buf,
int nestlevel;
ListCell *lc;- if (params) - *params = NIL; /* initialize result list to empty */ - /* Set up context struct for recursion */ context.root = root; context.foreignrel = baserel; @@ -971,6 +968,63 @@ deparseUpdateSql(StringInfo buf, PlannerInfo *root, }
It is needed for deparsePushedDownUpdateSql to store params in both
WHERE clauses and expressions to assign to the target columns
into one params_list list.
Hmm sorry but I am still not getting the point, can you provide some
example to explain this ?
Sorry, maybe my explanation was not enough. Consider:
postgres=# create foreign table ft1 (a int, b int) server myserver
options (table_name 't1');
postgres=# insert into ft1 values (0, 0);
postgres=# prepare mt(int, int) as update ft1 set a = $1 where b = $2;
postgres=# explain verbose execute mt(1, 0);
postgres=# explain verbose execute mt(1, 0);
postgres=# explain verbose execute mt(1, 0);
postgres=# explain verbose execute mt(1, 0);
postgres=# explain verbose execute mt(1, 0);
After the 5 executions of mt we have
postgres=# explain verbose execute mt(1, 0);
QUERY PLAN
------------------------------------------------------------------------------------
Update on public.ft1 (cost=100.00..140.35 rows=12 width=10)
-> Foreign Update on public.ft1 (cost=100.00..140.35 rows=12 width=10)
Remote SQL: UPDATE public.t1 SET a = $1::integer WHERE ((b =
$2::integer))
(3 rows)
If we do that initialization in appendWhereClause, we would get a wrong
params_list list and a wrong remote pushed-down query for the last mt()
in deparsePushedDownUpdateSql.
.) When Tom Lane and Stephen Frost suggested getting the core
code involved,
I thought that we can do the mandatory checks into core it self
and making
completely out of dml_is_pushdown_safe(). Please correct me
The reason why I put that function in postgres_fdw.c is Check point 4:
+ * 4. We can't push an UPDATE down, if any expressions to assign to the target + * columns are unsafe to evaluate on the remote server.
Here I was talking about checks related to triggers, or to LIMIT. I think
earlier thread talked about those mandatory check to the core. So may
be we can move those checks into make_modifytable() before calling
the PlanDMLPushdown.
Noticed that. Will do.
BTW, I keep a ForeignScan node pushing down an update to the remote
server, in the updated patches. I have to admit that that seems like
rather a misnomer. So, it might be worth adding a new ForeignUpdate
node, but my concern about that is that if doing so, we would have a lot
of duplicate code in ForeignUpdate and ForeignScan. What do you think
about that?
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Jan 14, 2016 at 2:00 PM, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>
wrote:
On 2016/01/12 20:31, Rushabh Lathia wrote:
On Thu, Jan 7, 2016 at 6:15 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp <mailto:fujita.etsuro@lab.ntt.co.jp>> wrote:
On 2016/01/06 18:58, Rushabh Lathia wrote:
.) What the need of following change ?@@ -833,9 +833,6 @@ appendWhereClause(StringInfo buf,
int nestlevel;
ListCell *lc;- if (params) - *params = NIL; /* initialize result list to empty */ - /* Set up context struct for recursion */ context.root = root; context.foreignrel = baserel; @@ -971,6 +968,63 @@ deparseUpdateSql(StringInfo buf, PlannerInfo *root, }It is needed for deparsePushedDownUpdateSql to store params in both
WHERE clauses and expressions to assign to the target columns
into one params_list list.Hmm sorry but I am still not getting the point, can you provide some
example to explain this ?
Sorry, maybe my explanation was not enough. Consider:
postgres=# create foreign table ft1 (a int, b int) server myserver options
(table_name 't1');
postgres=# insert into ft1 values (0, 0);
postgres=# prepare mt(int, int) as update ft1 set a = $1 where b = $2;
postgres=# explain verbose execute mt(1, 0);
postgres=# explain verbose execute mt(1, 0);
postgres=# explain verbose execute mt(1, 0);
postgres=# explain verbose execute mt(1, 0);
postgres=# explain verbose execute mt(1, 0);After the 5 executions of mt we have
postgres=# explain verbose execute mt(1, 0);
QUERY PLAN------------------------------------------------------------------------------------
Update on public.ft1 (cost=100.00..140.35 rows=12 width=10)
-> Foreign Update on public.ft1 (cost=100.00..140.35 rows=12 width=10)
Remote SQL: UPDATE public.t1 SET a = $1::integer WHERE ((b =
$2::integer))
(3 rows)If we do that initialization in appendWhereClause, we would get a wrong
params_list list and a wrong remote pushed-down query for the last mt() in
deparsePushedDownUpdateSql.
Strange, I am seeing same behaviour with or without that initialization in
appendWhereClause. After the 5 executions of mt I with or without I am
getting following output:
postgres=# explain verbose execute mt(1, 0);
QUERY
PLAN
------------------------------------------------------------------------------------
Update on public.ft2 (cost=100.00..140.35 rows=12 width=10)
-> Foreign Update on public.ft2 (cost=100.00..140.35 rows=12 width=10)
Remote SQL: UPDATE public.t2 SET a = $1::integer WHERE ((b =
$2::integer))
(3 rows)
.) When Tom Lane and Stephen Frost suggested getting the core
code involved,
I thought that we can do the mandatory checks into core it self
and making
completely out of dml_is_pushdown_safe(). Please correct meThe reason why I put that function in postgres_fdw.c is Check point 4:
+ * 4. We can't push an UPDATE down, if any expressions to assign to the target + * columns are unsafe to evaluate on the remote server.Here I was talking about checks related to triggers, or to LIMIT. I think
earlier thread talked about those mandatory check to the core. So may
be we can move those checks into make_modifytable() before calling
the PlanDMLPushdown.Noticed that. Will do.
BTW, I keep a ForeignScan node pushing down an update to the remote
server, in the updated patches. I have to admit that that seems like
rather a misnomer. So, it might be worth adding a new ForeignUpdate node,
but my concern about that is that if doing so, we would have a lot of
duplicate code in ForeignUpdate and ForeignScan. What do you think about
that?
Yes, I noticed that in the patch and I was about to point that out in my
final review. As first review I was mainly focused on the functionality
testing
and other overview things. Another reason I haven't posted that in my
first review round is, I was not quite sure whether we need the
separate new node ForeignUpdate, ForeignDelete and want to duplicate
code? Was also not quite sure about the fact that what we will achieve
by doing that.
So I thought, I will have this open question in my final review comment,
and will take committer's opinion on this. Since you already raised this
question lets take others opinion on this.
Regards,
--
Rushabh Lathia
www.EnterpriseDB.come
On 2016/01/14 21:36, Rushabh Lathia wrote:
On Thu, Jan 14, 2016 at 2:00 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp <mailto:fujita.etsuro@lab.ntt.co.jp>> wrote:
On 2016/01/12 20:31, Rushabh Lathia wrote:
On Thu, Jan 7, 2016 at 6:15 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp
<mailto:fujita.etsuro@lab.ntt.co.jp>
<mailto:fujita.etsuro@lab.ntt.co.jp
<mailto:fujita.etsuro@lab.ntt.co.jp>>> wrote:
On 2016/01/06 18:58, Rushabh Lathia wrote:
.) What the need of following change ?@@ -833,9 +833,6 @@ appendWhereClause(StringInfo buf,
int nestlevel;
ListCell *lc;- if (params) - *params = NIL; /* initialize result list to empty */ - /* Set up context struct for recursion */ context.root = root; context.foreignrel = baserel; @@ -971,6 +968,63 @@ deparseUpdateSql(StringInfo buf, PlannerInfo *root, }
It is needed for deparsePushedDownUpdateSql to store params
in both
WHERE clauses and expressions to assign to the target columns
into one params_list list.
Hmm sorry but I am still not getting the point, can you provide some
example to explain this ?
Sorry, maybe my explanation was not enough. Consider:
postgres=# create foreign table ft1 (a int, b int) server myserver
options (table_name 't1');
postgres=# insert into ft1 values (0, 0);
postgres=# prepare mt(int, int) as update ft1 set a = $1 where b = $2;
postgres=# explain verbose execute mt(1, 0);
postgres=# explain verbose execute mt(1, 0);
postgres=# explain verbose execute mt(1, 0);
postgres=# explain verbose execute mt(1, 0);
postgres=# explain verbose execute mt(1, 0);After the 5 executions of mt we have
postgres=# explain verbose execute mt(1, 0);
QUERY PLAN
------------------------------------------------------------------------------------
Update on public.ft1 (cost=100.00..140.35 rows=12 width=10)
-> Foreign Update on public.ft1 (cost=100.00..140.35 rows=12
width=10)
Remote SQL: UPDATE public.t1 SET a = $1::integer WHERE ((b
= $2::integer))
(3 rows)If we do that initialization in appendWhereClause, we would get a
wrong params_list list and a wrong remote pushed-down query for the
last mt() in deparsePushedDownUpdateSql.
Strange, I am seeing same behaviour with or without that initialization in
appendWhereClause. After the 5 executions of mt I with or without I am
getting following output:postgres=# explain verbose execute mt(1, 0);
QUERY PLAN
------------------------------------------------------------------------------------
Update on public.ft2 (cost=100.00..140.35 rows=12 width=10)
-> Foreign Update on public.ft2 (cost=100.00..140.35 rows=12 width=10)
Remote SQL: UPDATE public.t2 SET a = $1::integer WHERE ((b =
$2::integer))
(3 rows)
Really? With that initialization in appendWhereClause, I got the
following wrong result (note that both parameter numbers are $1):
postgres=# explain verbose execute mt(1, 0);
QUERY PLAN
------------------------------------------------------------------------------------
Update on public.ft1 (cost=100.00..140.35 rows=12 width=10)
-> Foreign Update on public.ft1 (cost=100.00..140.35 rows=12 width=10)
Remote SQL: UPDATE public.t1 SET a = $1::integer WHERE ((b =
$1::integer))
(3 rows)
BTW, I keep a ForeignScan node pushing down an update to the remote
server, in the updated patches. I have to admit that that seems
like rather a misnomer. So, it might be worth adding a new
ForeignUpdate node, but my concern about that is that if doing so,
we would have a lot of duplicate code in ForeignUpdate and
ForeignScan. What do you think about that?
Yes, I noticed that in the patch and I was about to point that out in my
final review. As first review I was mainly focused on the functionality
testing
and other overview things. Another reason I haven't posted that in my
first review round is, I was not quite sure whether we need the
separate new node ForeignUpdate, ForeignDelete and want to duplicate
code? Was also not quite sure about the fact that what we will achieve
by doing that.So I thought, I will have this open question in my final review comment,
and will take committer's opinion on this. Since you already raised this
question lets take others opinion on this.
OK, let's do that.
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/01/08 14:08, Etsuro Fujita wrote:
On 2016/01/07 21:50, Etsuro Fujita wrote:
On 2016/01/06 20:37, Thom Brown wrote:
I've run into an issue:
*# UPDATE master_customers SET id = 22 WHERE id = 16 RETURNING
tableoid::regclass;
ERROR:
CONTEXT: Remote SQL command: UPDATE public.customers SET id = 22
WHERE ((id = 16)) RETURNING NULL
While working on this, I noticed that the existing postgres_fdw system
shows similar behavior, so I changed the subject.IIUC, the reason for that is when the local query specifies "RETURNING
tableoid::regclass", the FDW has fmstate->has_returning=false while the
remote query executed at ModifyTable has "RETURNING NULL", as shown in
the above example; that would cause an abnormal exit in executing the
remote query in postgresExecForeignUpdate, since that the FDW would get
PGRES_TUPLES_OK as a result of the query while the FDW would think that
the right result to get should be PGRES_COMMAND_OK, from the flag
fmstate->has_returning=false.Attached is a patch to fix that.
I added this to the next CF.
https://commitfest.postgresql.org/9/483/
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 12 January 2016 at 11:49, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp> wrote:
On 2016/01/12 20:36, Thom Brown wrote:
On 8 January 2016 at 05:08, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>
wrote:On 2016/01/06 20:37, Thom Brown wrote:
I've run into an issue:
*# UPDATE master_customers SET id = 22 WHERE id = 16 RETURNING
tableoid::regclass;
ERROR:
CONTEXT: Remote SQL command: UPDATE public.customers SET id = 22
WHERE ((id = 16)) RETURNING NULLWhile working on this, I noticed that the existing postgres_fdw system
shows
similar behavior, so I changed the subject.IIUC, the reason for that is when the local query specifies "RETURNING
tableoid::regclass", the FDW has fmstate->has_returning=false while the
remote query executed at ModifyTable has "RETURNING NULL", as shown in
the
above example; that would cause an abnormal exit in executing the remote
query in postgresExecForeignUpdate, since that the FDW would get
PGRES_TUPLES_OK as a result of the query while the FDW would think that
the
right result to get should be PGRES_COMMAND_OK, from the flag
fmstate->has_returning=false.Attached is a patch to fix that.
I can't apply this patch in tandem with FDW DML pushdown patch (either
v2 or v3).That patch is for fixing the similar issue in the existing postgres_fdw
system. So, please apply that patch without the DML pushdown patch. If
that patch is reasonable as a fix for the issue, I'll update the DML
pushdown patch (v3) on top of that patch.
The patch seems to work for me:
Before:
*# UPDATE master_customers SET id = 22 WHERE id = 1 RETURNING
tableoid::regclass;
ERROR:
CONTEXT: Remote SQL command: UPDATE public.customers SET id = $2
WHERE ctid = $1 RETURNING NULL
After:
*# UPDATE master_customers SET id = 22 WHERE id = 1 RETURNING
tableoid::regclass;
tableoid
------------------
remote.customers
(1 row)
UPDATE 1
Thom
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Jan 19, 2016 at 1:59 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
I've run into an issue:
*# UPDATE master_customers SET id = 22 WHERE id = 16 RETURNING
tableoid::regclass;
ERROR:
CONTEXT: Remote SQL command: UPDATE public.customers SET id = 22
WHERE ((id = 16)) RETURNING NULLWhile working on this, I noticed that the existing postgres_fdw system
shows similar behavior, so I changed the subject.IIUC, the reason for that is when the local query specifies "RETURNING
tableoid::regclass", the FDW has fmstate->has_returning=false while the
remote query executed at ModifyTable has "RETURNING NULL", as shown in
the above example; that would cause an abnormal exit in executing the
remote query in postgresExecForeignUpdate, since that the FDW would get
PGRES_TUPLES_OK as a result of the query while the FDW would think that
the right result to get should be PGRES_COMMAND_OK, from the flag
fmstate->has_returning=false.Attached is a patch to fix that.
I added this to the next CF.
Uggh, what a mess. How about passing an additional boolean
"is_returning" to deparseTargetList()? If false, then
deparseTargetList() behaves as now. If false, then
deparseTargetList() doesn't append anything at all if there are no
columns to return, instead of (as at present) adding NULL. On the
other hand, if there are columns to return, then it appends "
RETURNING " before the first column. Then, deparseReturningList could
skip adding RETURNING itself, and just pass true to
deparseTargetList(). The advantage of this approach is that we don't
end up with two copies of the code that have to stay synchronized -
the decision is made inside deparseTargetList(), and
deparseReturningList() accepts the results.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/01/20 3:42, Robert Haas wrote:
On Tue, Jan 19, 2016 at 1:59 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:I've run into an issue:
*# UPDATE master_customers SET id = 22 WHERE id = 16 RETURNING
tableoid::regclass;
ERROR:
CONTEXT: Remote SQL command: UPDATE public.customers SET id = 22
WHERE ((id = 16)) RETURNING NULL
While working on this, I noticed that the existing postgres_fdw system
shows similar behavior, so I changed the subject.IIUC, the reason for that is when the local query specifies "RETURNING
tableoid::regclass", the FDW has fmstate->has_returning=false while the
remote query executed at ModifyTable has "RETURNING NULL", as shown in
the above example; that would cause an abnormal exit in executing the
remote query in postgresExecForeignUpdate, since that the FDW would get
PGRES_TUPLES_OK as a result of the query while the FDW would think that
the right result to get should be PGRES_COMMAND_OK, from the flag
fmstate->has_returning=false.Attached is a patch to fix that.
I added this to the next CF.
Uggh, what a mess. How about passing an additional boolean
"is_returning" to deparseTargetList()? If false, then
deparseTargetList() behaves as now. If false, then
deparseTargetList() doesn't append anything at all if there are no
columns to return, instead of (as at present) adding NULL. On the
other hand, if there are columns to return, then it appends "
RETURNING " before the first column. Then, deparseReturningList could
skip adding RETURNING itself, and just pass true to
deparseTargetList(). The advantage of this approach is that we don't
end up with two copies of the code that have to stay synchronized -
Thanks for the review! I think that is important.
the decision is made inside deparseTargetList(), and
deparseReturningList() accepts the results.
My concern about that is that would make the code in deparseTargetList()
complicated.
Essentially, I think your propossal needs a two-pass algorithm for
deparseTargetList; (1) create an integer List of the columns being
retrieved from the given attrs_used (getRetrievedAttrs()), and (2) print
those columns (printRetrievedAttrs()). How about sharing those two
functions between deparseTargetList and deparseReturningList?:
* In deparseTargetList, perform getRetrievedAttrs(). If
getRetrievedAttrs()!=NIL, perform printRetrievedAttrs(). Otherwise,
print NULL.
* In deparseReturningList, perform getRetrievedAttrs() before adding
RETURNING. If getRetrievedAttrs()!=NIL, print RETURNING and perform
printRetrievedAttrs(). Otherwise, do nothing.
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Jan 15, 2016 at 9:06 AM, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>
wrote:
On 2016/01/14 21:36, Rushabh Lathia wrote:
On Thu, Jan 14, 2016 at 2:00 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp <mailto:fujita.etsuro@lab.ntt.co.jp>> wrote:On 2016/01/12 20:31, Rushabh Lathia wrote:
On Thu, Jan 7, 2016 at 6:15 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp
<mailto:fujita.etsuro@lab.ntt.co.jp>
<mailto:fujita.etsuro@lab.ntt.co.jp<mailto:fujita.etsuro@lab.ntt.co.jp>>> wrote:
On 2016/01/06 18:58, Rushabh Lathia wrote:
.) What the need of following change ?@@ -833,9 +833,6 @@ appendWhereClause(StringInfo buf,
int nestlevel;
ListCell *lc;- if (params) - *params = NIL; /* initialize result list to empty */ - /* Set up context struct for recursion */ context.root = root; context.foreignrel = baserel; @@ -971,6 +968,63 @@ deparseUpdateSql(StringInfo buf, PlannerInfo *root, }It is needed for deparsePushedDownUpdateSql to store params
in both
WHERE clauses and expressions to assign to the target columns
into one params_list list.Hmm sorry but I am still not getting the point, can you provide
some
example to explain this ?Sorry, maybe my explanation was not enough. Consider:
postgres=# create foreign table ft1 (a int, b int) server myserver
options (table_name 't1');
postgres=# insert into ft1 values (0, 0);
postgres=# prepare mt(int, int) as update ft1 set a = $1 where b = $2;
postgres=# explain verbose execute mt(1, 0);
postgres=# explain verbose execute mt(1, 0);
postgres=# explain verbose execute mt(1, 0);
postgres=# explain verbose execute mt(1, 0);
postgres=# explain verbose execute mt(1, 0);After the 5 executions of mt we have
postgres=# explain verbose execute mt(1, 0);
QUERY PLAN------------------------------------------------------------------------------------
Update on public.ft1 (cost=100.00..140.35 rows=12 width=10)
-> Foreign Update on public.ft1 (cost=100.00..140.35 rows=12
width=10)
Remote SQL: UPDATE public.t1 SET a = $1::integer WHERE ((b
= $2::integer))
(3 rows)If we do that initialization in appendWhereClause, we would get a
wrong params_list list and a wrong remote pushed-down query for the
last mt() in deparsePushedDownUpdateSql.Strange, I am seeing same behaviour with or without that initialization in
appendWhereClause. After the 5 executions of mt I with or without I am
getting following output:postgres=# explain verbose execute mt(1, 0);
QUERY PLAN------------------------------------------------------------------------------------
Update on public.ft2 (cost=100.00..140.35 rows=12 width=10)
-> Foreign Update on public.ft2 (cost=100.00..140.35 rows=12
width=10)
Remote SQL: UPDATE public.t2 SET a = $1::integer WHERE ((b =
$2::integer))
(3 rows)Really? With that initialization in appendWhereClause, I got the
following wrong result (note that both parameter numbers are $1):postgres=# explain verbose execute mt(1, 0);
QUERY PLAN------------------------------------------------------------------------------------
Update on public.ft1 (cost=100.00..140.35 rows=12 width=10)
-> Foreign Update on public.ft1 (cost=100.00..140.35 rows=12 width=10)
Remote SQL: UPDATE public.t1 SET a = $1::integer WHERE ((b =
$1::integer))
(3 rows)
Oops sorry. I got the point now.
BTW, I keep a ForeignScan node pushing down an update to the remote
server, in the updated patches. I have to admit that that seems
like rather a misnomer. So, it might be worth adding a new
ForeignUpdate node, but my concern about that is that if doing so,
we would have a lot of duplicate code in ForeignUpdate and
ForeignScan. What do you think about that?Yes, I noticed that in the patch and I was about to point that out in my
final review. As first review I was mainly focused on the functionality
testing
and other overview things. Another reason I haven't posted that in my
first review round is, I was not quite sure whether we need the
separate new node ForeignUpdate, ForeignDelete and want to duplicate
code? Was also not quite sure about the fact that what we will achieve
by doing that.So I thought, I will have this open question in my final review comment,
and will take committer's opinion on this. Since you already raised this
question lets take others opinion on this.OK, let's do that.
Overall I am quite done with the review of this patch. Patch is in good
shape and covered most of the things which been discussed earlier
or been mentioned during review process. Patch pass through the
make check and also includes good test coverage.
Here are couple of things which is still open for discussion:
1)
.) When Tom Lane and Stephen Frost suggested getting the core code involved,
I thought that we can do the mandatory checks into core it self and making
completely out of dml_is_pushdown_safe(). Please correct meThe reason why I put that function in postgres_fdw.c is Check point 4:
+ * 4. We can't push an UPDATE down, if any expressions to assign to the target + * columns are unsafe to evaluate on the remote server.
Here I was talking about checks related to triggers, or to LIMIT. I think
earlier thread talked about those mandatory check to the core. So may
be we can move those checks into make_modifytable() before calling
the PlanDMLPushdown.
This need to handle by the Owner.
2) Decision on whether we need the separate new node ForeignUpdate,
ForeignDelete. In my opinion I really don't see the need of this as we
that will add lot of duplicate. Having said that if committer or someone
else feel like that will make code more clean that is also true,
This need more comments from the committer.
Thanks
Rushabh Lathia
www.EnterpriseDB.com
On Wed, Jan 20, 2016 at 4:50 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
My concern about that is that would make the code in deparseTargetList()
complicated.Essentially, I think your propossal needs a two-pass algorithm for
deparseTargetList; (1) create an integer List of the columns being retrieved
from the given attrs_used (getRetrievedAttrs()), and (2) print those columns
(printRetrievedAttrs()). How about sharing those two functions between
deparseTargetList and deparseReturningList?:
I don't see why we'd need that. I adjusted the code in postgres_fdw
along the lines I had in mind and am attaching the result. It doesn't
look complicated to me, and it passes the regression test you wrote.
By the way, I'm not too sure I understand the need for the core
changes that are part of this patch, and I think that point merits
some discussion. Whenever you change core like this, you're changing
the contract between the FDW and core; it's not just postgres_fdw that
needs updating, but every FDW. So we'd better be pretty sure we need
these changes and they are adequately justified before we think about
putting them into the tree. Are these core changes really needed
here, or can we fix this whole issue in postgres_fdw and leave the
core code alone?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
fdw-foreign-modify-rmh.patchtext/x-patch; charset=US-ASCII; name=fdw-foreign-modify-rmh.patchDownload
diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c
index e59af2c..12a1031 100644
--- a/contrib/postgres_fdw/deparse.c
+++ b/contrib/postgres_fdw/deparse.c
@@ -110,6 +110,7 @@ static void deparseTargetList(StringInfo buf,
PlannerInfo *root,
Index rtindex,
Relation rel,
+ bool is_returning,
Bitmapset *attrs_used,
List **retrieved_attrs);
static void deparseReturningList(StringInfo buf, PlannerInfo *root,
@@ -724,7 +725,7 @@ deparseSelectSql(StringInfo buf,
* Construct SELECT list
*/
appendStringInfoString(buf, "SELECT ");
- deparseTargetList(buf, root, baserel->relid, rel, attrs_used,
+ deparseTargetList(buf, root, baserel->relid, rel, false, attrs_used,
retrieved_attrs);
/*
@@ -738,7 +739,8 @@ deparseSelectSql(StringInfo buf,
/*
* Emit a target list that retrieves the columns specified in attrs_used.
- * This is used for both SELECT and RETURNING targetlists.
+ * This is used for both SELECT and RETURNING targetlists; the is_returning
+ * parameter is true only for a RETURNING targetlist.
*
* The tlist text is appended to buf, and we also create an integer List
* of the columns being retrieved, which is returned to *retrieved_attrs.
@@ -748,6 +750,7 @@ deparseTargetList(StringInfo buf,
PlannerInfo *root,
Index rtindex,
Relation rel,
+ bool is_returning,
Bitmapset *attrs_used,
List **retrieved_attrs)
{
@@ -777,6 +780,8 @@ deparseTargetList(StringInfo buf,
{
if (!first)
appendStringInfoString(buf, ", ");
+ else if (is_returning)
+ appendStringInfoString(buf, " RETURNING ");
first = false;
deparseColumnRef(buf, rtindex, i, root);
@@ -794,6 +799,8 @@ deparseTargetList(StringInfo buf,
{
if (!first)
appendStringInfoString(buf, ", ");
+ else if (is_returning)
+ appendStringInfoString(buf, " RETURNING ");
first = false;
appendStringInfoString(buf, "ctid");
@@ -803,7 +810,7 @@ deparseTargetList(StringInfo buf,
}
/* Don't generate bad syntax if no undropped columns */
- if (first)
+ if (first && !is_returning)
appendStringInfoString(buf, "NULL");
}
@@ -1022,11 +1029,8 @@ deparseReturningList(StringInfo buf, PlannerInfo *root,
}
if (attrs_used != NULL)
- {
- appendStringInfoString(buf, " RETURNING ");
- deparseTargetList(buf, root, rtindex, rel, attrs_used,
+ deparseTargetList(buf, root, rtindex, rel, true, attrs_used,
retrieved_attrs);
- }
else
*retrieved_attrs = NIL;
}
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index b471c67..b0d12ac 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2408,6 +2408,59 @@ SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
1104 | 204 | ddd |
(819 rows)
+EXPLAIN (verbose, costs off)
+INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
+ QUERY PLAN
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Insert on public.ft2
+ Output: (tableoid)::regclass
+ Remote SQL: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6, c7, c8) VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
+ -> Result
+ Output: 9999, 999, NULL::integer, 'foo'::text, NULL::timestamp with time zone, NULL::timestamp without time zone, NULL::character varying, 'ft2 '::character(10), NULL::user_enum
+(5 rows)
+
+INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
+ tableoid
+----------
+ ft2
+(1 row)
+
+EXPLAIN (verbose, costs off)
+UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
+ QUERY PLAN
+-------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Output: (tableoid)::regclass
+ Remote SQL: UPDATE "S 1"."T 1" SET c3 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.ft2
+ Output: c1, c2, NULL::integer, 'bar'::text, c4, c5, c6, c7, c8, ctid
+ Remote SQL: SELECT "C 1", c2, c4, c5, c6, c7, c8, ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
+(6 rows)
+
+UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
+ tableoid
+----------
+ ft2
+(1 row)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
+ QUERY PLAN
+------------------------------------------------------------------------------------
+ Delete on public.ft2
+ Output: (tableoid)::regclass
+ Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1
+ -> Foreign Scan on public.ft2
+ Output: ctid
+ Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
+(6 rows)
+
+DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
+ tableoid
+----------
+ ft2
+(1 row)
+
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
BEGIN
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 73fa9f6..4b13654 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -413,6 +413,15 @@ EXPLAIN (verbose, costs off)
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
+EXPLAIN (verbose, costs off)
+INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
+INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
+EXPLAIN (verbose, costs off)
+UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
+UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
+EXPLAIN (verbose, costs off)
+DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
+DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 46299fc..27051e8 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -308,6 +308,12 @@ ExecInsert(ModifyTableState *mtstate,
/* FDW might have changed tuple */
tuple = ExecMaterializeSlot(slot);
+ /*
+ * AFTER ROW Triggers or RETURNING expressions might reference the
+ * tableoid column, so initialize t_tableOid before evaluating them.
+ */
+ tuple->t_tableOid = RelationGetRelid(resultRelationDesc);
+
newId = InvalidOid;
}
else
@@ -561,6 +567,8 @@ ExecDelete(ItemPointer tupleid,
}
else if (resultRelInfo->ri_FdwRoutine)
{
+ HeapTuple tuple;
+
/*
* delete from foreign table: let the FDW do it
*
@@ -579,6 +587,15 @@ ExecDelete(ItemPointer tupleid,
if (slot == NULL) /* "do nothing" */
return NULL;
+
+ /*
+ * RETURNING expressions might reference the tableoid column, so
+ * initialize t_tableOid before evaluating them.
+ */
+ if (slot->tts_isempty)
+ ExecStoreAllNullTuple(slot);
+ tuple = ExecMaterializeSlot(slot);
+ tuple->t_tableOid = RelationGetRelid(resultRelationDesc);
}
else
{
@@ -838,6 +855,12 @@ ExecUpdate(ItemPointer tupleid,
/* FDW might have changed tuple */
tuple = ExecMaterializeSlot(slot);
+
+ /*
+ * AFTER ROW Triggers or RETURNING expressions might reference the
+ * tableoid column, so initialize t_tableOid before evaluating them.
+ */
+ tuple->t_tableOid = RelationGetRelid(resultRelationDesc);
}
else
{
On 2016/01/21 5:06, Robert Haas wrote:
On Wed, Jan 20, 2016 at 4:50 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:My concern about that is that would make the code in deparseTargetList()
complicated.Essentially, I think your propossal needs a two-pass algorithm for
deparseTargetList; (1) create an integer List of the columns being retrieved
from the given attrs_used (getRetrievedAttrs()), and (2) print those columns
(printRetrievedAttrs()). How about sharing those two functions between
deparseTargetList and deparseReturningList?:
I don't see why we'd need that. I adjusted the code in postgres_fdw
along the lines I had in mind and am attaching the result. It doesn't
look complicated to me, and it passes the regression test you wrote.
Thanks for the patch! From the patch, I correctly understand what you
proposed. Good idea!
By the way, I'm not too sure I understand the need for the core
changes that are part of this patch, and I think that point merits
some discussion. Whenever you change core like this, you're changing
the contract between the FDW and core; it's not just postgres_fdw that
needs updating, but every FDW. So we'd better be pretty sure we need
these changes and they are adequately justified before we think about
putting them into the tree. Are these core changes really needed
here, or can we fix this whole issue in postgres_fdw and leave the
core code alone?
Well, if we think it is the FDW's responsibility to insert a valid value
for tableoid in the returned slot during ExecForeignInsert,
ExecForeignUpdate or ExecForeignDelete, we don't need those core
changes. However, I think it would be better that that's done by
ModifyTable in the same way as ForeignScan does in ForeignNext, IMO.
That eliminates the need for postgres_fdw or any other FDW to do that
business in the callback routines.
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/01/20 19:57, Rushabh Lathia wrote:
Overall I am quite done with the review of this patch. Patch is in good
shape and covered most of the things which been discussed earlier
or been mentioned during review process. Patch pass through the
make check and also includes good test coverage.
Thanks for the review!
Here are couple of things which is still open for discussion:
1)
.) When Tom Lane and Stephen Frost suggested getting the core
code involved,
I thought that we can do the mandatory checks into core it self
and making
completely out of dml_is_pushdown_safe(). Please correct me
The reason why I put that function in postgres_fdw.c is Check point 4:
+ * 4. We can't push an UPDATE down, if any expressions to assign to the target + * columns are unsafe to evaluate on the remote server.
Here I was talking about checks related to triggers, or to LIMIT. I think
earlier thread talked about those mandatory check to the core. So may
be we can move those checks into make_modifytable() before calling
the PlanDMLPushdown.This need to handle by the Owner.
Done. For that, I modified relation_has_row_triggers a bit, renamed it
to has_row_triggers (more shortly), and moved it to plancat.c. And I
merged dml_is_pushdown_safe with postgresPlanDMLPushdown, and revised
that callback routine a bit. Attached is an updated version of the
patch created on top of Robert's version of the patch [1]/messages/by-id/CA+TgmoZ40j2uC5aC1NXu03oj4CrVOLkS15XX+PTFP-1U-8zR1Q@mail.gmail.com, which fixes
handling of RETURNING tableoid in updating foreign tables.
2) Decision on whether we need the separate new node ForeignUpdate,
ForeignDelete. In my opinion I really don't see the need of this as we
that will add lot of duplicate. Having said that if committer or someone
else feel like that will make code more clean that is also true,This need more comments from the committer.
I agree with you.
Other changes:
* In previous version, I assumed that PlanDMLPushdown sets fsSystemCol
to true when rewriting the ForeignScan plan node so as to push down an
UPDATE/DELETE to the remote server, in order to initialize t_tableOid
for the scan tuple in ForeignNext. The reason is that I created the
patch so that the scan tuple is provided to the local query's RETURNING
computation, which might see the tableoid column. In this version,
however, I modified the patch so that the tableoid value is inserted by
ModifyTable. This eliminates the need for postgres_fdw (or any other
FDW) to set fsSystemCol to true in PlanDMLPushdown.
* Add set_transmission_modes/reset_transmission_modes to
deparsePushedDownUpdateSql.
* Revise comments a bit further.
* Revise docs, including a fix for a wrong copy-and-paste.
Best regards,
Etsuro Fujita
[1]: /messages/by-id/CA+TgmoZ40j2uC5aC1NXu03oj4CrVOLkS15XX+PTFP-1U-8zR1Q@mail.gmail.com
/messages/by-id/CA+TgmoZ40j2uC5aC1NXu03oj4CrVOLkS15XX+PTFP-1U-8zR1Q@mail.gmail.com
Attachments:
fdw-dml-pushdown-v4.patchapplication/x-patch; name=fdw-dml-pushdown-v4.patchDownload
*** a/contrib/postgres_fdw/deparse.c
--- b/contrib/postgres_fdw/deparse.c
***************
*** 823,829 **** deparseTargetList(StringInfo buf,
*
* If params is not NULL, it receives a list of Params and other-relation Vars
* used in the clauses; these values must be transmitted to the remote server
! * as parameter values.
*
* If params is NULL, we're generating the query for EXPLAIN purposes,
* so Params and other-relation Vars should be replaced by dummy values.
--- 823,829 ----
*
* If params is not NULL, it receives a list of Params and other-relation Vars
* used in the clauses; these values must be transmitted to the remote server
! * as parameter values. Caller is responsible for initializing it to empty.
*
* If params is NULL, we're generating the query for EXPLAIN purposes,
* so Params and other-relation Vars should be replaced by dummy values.
***************
*** 840,848 **** appendWhereClause(StringInfo buf,
int nestlevel;
ListCell *lc;
- if (params)
- *params = NIL; /* initialize result list to empty */
-
/* Set up context struct for recursion */
context.root = root;
context.foreignrel = baserel;
--- 840,845 ----
***************
*** 978,983 **** deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 975,1044 ----
}
/*
+ * deparse remote UPDATE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ deparse_expr_cxt context;
+ int nestlevel;
+ bool first;
+ ListCell *lc;
+
+ if (params_list)
+ *params_list = NIL; /* initialize result list to empty */
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = params_list;
+
+ appendStringInfoString(buf, "UPDATE ");
+ deparseRelation(buf, rel);
+ appendStringInfoString(buf, " SET ");
+
+ /* Make sure any constants in the exprs are printed portably */
+ nestlevel = set_transmission_modes();
+
+ first = true;
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(targetlist, attnum);
+
+ if (!first)
+ appendStringInfoString(buf, ", ");
+ first = false;
+
+ deparseColumnRef(buf, rtindex, attnum, root);
+ appendStringInfoString(buf, " = ");
+ deparseExpr((Expr *) tle->expr, &context);
+ }
+
+ reset_transmission_modes(nestlevel);
+
+ if (remote_conds)
+ appendWhereClause(buf, root, baserel, remote_conds,
+ true, params_list);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* deparse remote DELETE statement
*
* The statement text is appended to buf, and we also create an integer List
***************
*** 1000,1005 **** deparseDeleteSql(StringInfo buf, PlannerInfo *root,
--- 1061,1097 ----
}
/*
+ * deparse remote DELETE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+
+ if (params_list)
+ *params_list = NIL; /* initialize result list to empty */
+
+ appendStringInfoString(buf, "DELETE FROM ");
+ deparseRelation(buf, rel);
+
+ if (remote_conds)
+ appendWhereClause(buf, root, baserel, remote_conds,
+ true, params_list);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* Add a RETURNING clause, if needed, to an INSERT/UPDATE/DELETE.
*/
static void
*** a/contrib/postgres_fdw/expected/postgres_fdw.out
--- b/contrib/postgres_fdw/expected/postgres_fdw.out
***************
*** 1314,1320 **** INSERT INTO ft2 (c1,c2,c3)
--- 1314,1339 ----
(3 rows)
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 300), c3 = (c3 || '_update3'::text) WHERE ((("C 1" % 10) = 3))
+ (3 rows)
+
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Output: c1, c2, c3, c4, c5, c6, c7, c8
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 400), c3 = (c3 || '_update7'::text) WHERE ((("C 1" % 10) = 7)) RETURNING "C 1", c2, c3, c4, c5, c6, c7, c8
+ (4 rows)
+
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
c1 | c2 | c3 | c4 | c5 | c6 | c7 | c8
------+-----+--------------------+------------------------------+--------------------------+----+------------+-----
***************
*** 1424,1430 **** UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
--- 1443,1449 ----
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
***************
*** 1445,1460 **** UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
! QUERY PLAN
! ----------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1 RETURNING "C 1", c4
! -> Foreign Scan on public.ft2
! Output: ctid
! Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) FOR UPDATE
! (6 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
--- 1464,1477 ----
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
! QUERY PLAN
! --------------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
! -> Foreign Delete on public.ft2
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) RETURNING "C 1", c4
! (4 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
***************
*** 1565,1571 **** DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
--- 1582,1588 ----
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
***************
*** 2427,2441 **** INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
! QUERY PLAN
! -------------------------------------------------------------------------------------------------------------------
Update on public.ft2
Output: (tableoid)::regclass
! Remote SQL: UPDATE "S 1"."T 1" SET c3 = $2 WHERE ctid = $1
! -> Foreign Scan on public.ft2
! Output: c1, c2, NULL::integer, 'bar'::text, c4, c5, c6, c7, c8, ctid
! Remote SQL: SELECT "C 1", c2, c4, c5, c6, c7, c8, ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
! (6 rows)
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
--- 2444,2456 ----
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
! QUERY PLAN
! ------------------------------------------------------------------------------------
Update on public.ft2
Output: (tableoid)::regclass
! -> Foreign Update on public.ft2
! Remote SQL: UPDATE "S 1"."T 1" SET c3 = 'bar'::text WHERE (("C 1" = 9999))
! (4 rows)
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
***************
*** 2445,2459 **** UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
! QUERY PLAN
! ------------------------------------------------------------------------------------
Delete on public.ft2
Output: (tableoid)::regclass
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1
! -> Foreign Scan on public.ft2
! Output: ctid
! Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
! (6 rows)
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
--- 2460,2472 ----
EXPLAIN (verbose, costs off)
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
! QUERY PLAN
! --------------------------------------------------------------------
Delete on public.ft2
Output: (tableoid)::regclass
! -> Foreign Delete on public.ft2
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE (("C 1" = 9999))
! (4 rows)
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
***************
*** 2607,2613 **** CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 2620,2626 ----
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 2766,2772 **** savepoint s3;
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 2779,2785 ----
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (-2) WHERE ((c2 = 42)) AND (("C 1" = 10))
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 2906,2912 **** CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2positive;
-- But inconsistent check constraints provide inconsistent results
ALTER FOREIGN TABLE ft1 ADD CONSTRAINT ft1_c2negative CHECK (c2 < 0);
--- 2919,2925 ----
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2positive;
-- But inconsistent check constraints provide inconsistent results
ALTER FOREIGN TABLE ft1 ADD CONSTRAINT ft1_c2negative CHECK (c2 < 0);
***************
*** 3299,3304 **** NOTICE: NEW: (13,"test triggered !")
--- 3312,3510 ----
(0,27)
(1 row)
+ -- cleanup
+ DROP TRIGGER trig_row_before ON rem1;
+ DROP TRIGGER trig_row_after ON rem1;
+ DROP TRIGGER trig_local_before ON loc1;
+ -- Test DML pushdown functionality
+ -- Test with statement-level triggers
+ CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_stmt_before ON rem1;
+ CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_stmt_after ON rem1;
+ -- Test with row-level ON INSERT triggers
+ CREATE TRIGGER trig_row_before_insert
+ BEFORE INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_before_insert ON rem1;
+ CREATE TRIGGER trig_row_after_insert
+ AFTER INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_after_insert ON rem1;
+ -- Test with row-level ON UPDATE triggers
+ CREATE TRIGGER trig_row_before_update
+ BEFORE UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_before_update ON rem1;
+ CREATE TRIGGER trig_row_after_update
+ AFTER UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+ -------------------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_after_update ON rem1;
+ -- Test with row-level ON DELETE triggers
+ CREATE TRIGGER trig_row_before_delete
+ BEFORE DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ DROP TRIGGER trig_row_before_delete ON rem1;
+ CREATE TRIGGER trig_row_after_delete
+ AFTER DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+ ------------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ DROP TRIGGER trig_row_after_delete ON rem1;
-- ===================================================================
-- test inheritance features
-- ===================================================================
***************
*** 3768,3773 **** fetch from c;
--- 3974,4029 ----
update bar set f2 = null where current of c;
ERROR: WHERE CURRENT OF is not supported for this table type
rollback;
+ explain (verbose, costs off)
+ delete from foo where f1 < 5 returning *;
+ QUERY PLAN
+ --------------------------------------------------------------------------------
+ Delete on public.foo
+ Output: foo.f1, foo.f2
+ Delete on public.foo
+ Foreign Delete on public.foo2
+ -> Index Scan using i_foo_f1 on public.foo
+ Output: foo.ctid
+ Index Cond: (foo.f1 < 5)
+ -> Foreign Delete on public.foo2
+ Remote SQL: DELETE FROM public.loct1 WHERE ((f1 < 5)) RETURNING f1, f2
+ (9 rows)
+
+ delete from foo where f1 < 5 returning *;
+ f1 | f2
+ ----+----
+ 1 | 1
+ 3 | 3
+ 0 | 0
+ 2 | 2
+ 4 | 4
+ (5 rows)
+
+ explain (verbose, costs off)
+ update bar set f2 = f2 + 100 returning *;
+ QUERY PLAN
+ ------------------------------------------------------------------------------
+ Update on public.bar
+ Output: bar.f1, bar.f2
+ Update on public.bar
+ Foreign Update on public.bar2
+ -> Seq Scan on public.bar
+ Output: bar.f1, (bar.f2 + 100), bar.ctid
+ -> Foreign Update on public.bar2
+ Remote SQL: UPDATE public.loct2 SET f2 = (f2 + 100) RETURNING f1, f2
+ (8 rows)
+
+ update bar set f2 = f2 + 100 returning *;
+ f1 | f2
+ ----+-----
+ 1 | 311
+ 2 | 322
+ 6 | 266
+ 3 | 333
+ 4 | 344
+ 7 | 277
+ (6 rows)
+
drop table foo cascade;
NOTICE: drop cascades to foreign table foo2
drop table bar cascade;
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
***************
*** 57,63 **** PG_MODULE_MAGIC;
* planner to executor. Currently we store:
*
* 1) SELECT statement text to be sent to the remote server
! * 2) Integer list of attribute numbers retrieved by the SELECT
*
* These items are indexed with the enum FdwScanPrivateIndex, so an item
* can be fetched with list_nth(). For example, to get the SELECT statement:
--- 57,64 ----
* planner to executor. Currently we store:
*
* 1) SELECT statement text to be sent to the remote server
! * 2) List of restriction clauses that can be executed remotely
! * 3) Integer list of attribute numbers retrieved by the SELECT
*
* These items are indexed with the enum FdwScanPrivateIndex, so an item
* can be fetched with list_nth(). For example, to get the SELECT statement:
***************
*** 67,72 **** enum FdwScanPrivateIndex
--- 68,75 ----
{
/* SQL statement to execute remotely (as a String node) */
FdwScanPrivateSelectSql,
+ /* List of restriction clauses that can be executed remotely */
+ FdwScanPrivateRemoteConds,
/* Integer list of attribute numbers retrieved by the SELECT */
FdwScanPrivateRetrievedAttrs
};
***************
*** 94,99 **** enum FdwModifyPrivateIndex
--- 97,124 ----
};
/*
+ * Similarly, this enum describes what's kept in the fdw_private list for
+ * a ForeignScan node that has pushed down an UPDATE/DELETE to the remote
+ * server. We store:
+ *
+ * 1) UPDATE/DELETE statement text to be sent to the remote server
+ * 2) Boolean flag showing if the remote query has a RETURNING clause
+ * 3) Integer list of attribute numbers retrieved by RETURNING, if any
+ * 4) Boolean flag showing if we set the command es_processed
+ */
+ enum FdwDmlPushdownPrivateIndex
+ {
+ /* SQL statement to execute remotely (as a String node) */
+ FdwDmlPushdownPrivateUpdateSql,
+ /* has-returning flag (as an integer Value node) */
+ FdwDmlPushdownPrivateHasReturning,
+ /* Integer list of attribute numbers retrieved by RETURNING */
+ FdwDmlPushdownPrivateRetrievedAttrs,
+ /* set-processed flag (as an integer Value node) */
+ FdwDmlPushdownPrivateSetProcessed
+ };
+
+ /*
* Execution state of a foreign scan using postgres_fdw.
*/
typedef struct PgFdwScanState
***************
*** 156,161 **** typedef struct PgFdwModifyState
--- 181,217 ----
} PgFdwModifyState;
/*
+ * Execution state of a foreign scan that has pushed down a foreign table
+ * modification to the remote server
+ */
+ typedef struct PgFdwDmlPushdownState
+ {
+ Relation rel; /* relcache entry for the foreign table */
+ AttInMetadata *attinmeta; /* attribute datatype conversion metadata */
+
+ /* extracted fdw_private data */
+ char *query; /* text of UPDATE/DELETE command */
+ bool has_returning; /* is there a RETURNING clause? */
+ List *retrieved_attrs; /* attr numbers retrieved by RETURNING */
+ bool set_processed; /* do we set the command es_processed? */
+
+ /* for remote query execution */
+ PGconn *conn; /* connection for the scan */
+ int numParams; /* number of parameters passed to query */
+ FmgrInfo *param_flinfo; /* output conversion functions for them */
+ List *param_exprs; /* executable expressions for param values */
+ const char **param_values; /* textual values of query parameters */
+
+ /* for storing result tuples */
+ PGresult *result; /* result for query */
+ int num_tuples; /* # of result tuples */
+ int next_tuple; /* index of next one to return */
+
+ /* working memory contexts */
+ MemoryContext temp_cxt; /* context for per-tuple temporary data */
+ } PgFdwDmlPushdownState;
+
+ /*
* Workspace for analyzing a foreign table.
*/
typedef struct PgFdwAnalyzeState
***************
*** 247,252 **** static TupleTableSlot *postgresExecForeignDelete(EState *estate,
--- 303,315 ----
static void postgresEndForeignModify(EState *estate,
ResultRelInfo *resultRelInfo);
static int postgresIsForeignRelUpdatable(Relation rel);
+ static bool postgresPlanDMLPushdown(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+ static void postgresBeginDMLPushdown(ForeignScanState *node, int eflags);
+ static TupleTableSlot *postgresIterateDMLPushdown(ForeignScanState *node);
+ static void postgresEndDMLPushdown(ForeignScanState *node);
static void postgresExplainForeignScan(ForeignScanState *node,
ExplainState *es);
static void postgresExplainForeignModify(ModifyTableState *mtstate,
***************
*** 254,259 **** static void postgresExplainForeignModify(ModifyTableState *mtstate,
--- 317,324 ----
List *fdw_private,
int subplan_index,
ExplainState *es);
+ static void postgresExplainDMLPushdown(ForeignScanState *node,
+ ExplainState *es);
static bool postgresAnalyzeForeignTable(Relation relation,
AcquireSampleRowsFunc *func,
BlockNumber *totalpages);
***************
*** 290,295 **** static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
--- 355,362 ----
TupleTableSlot *slot);
static void store_returning_result(PgFdwModifyState *fmstate,
TupleTableSlot *slot, PGresult *res);
+ static void execute_dml_stmt(ForeignScanState *node);
+ static TupleTableSlot *get_returning_data(ForeignScanState *node);
static int postgresAcquireSampleRowsFunc(Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
***************
*** 332,341 **** postgres_fdw_handler(PG_FUNCTION_ARGS)
--- 399,413 ----
routine->ExecForeignDelete = postgresExecForeignDelete;
routine->EndForeignModify = postgresEndForeignModify;
routine->IsForeignRelUpdatable = postgresIsForeignRelUpdatable;
+ routine->PlanDMLPushdown = postgresPlanDMLPushdown;
+ routine->BeginDMLPushdown = postgresBeginDMLPushdown;
+ routine->IterateDMLPushdown = postgresIterateDMLPushdown;
+ routine->EndDMLPushdown = postgresEndDMLPushdown;
/* Support functions for EXPLAIN */
routine->ExplainForeignScan = postgresExplainForeignScan;
routine->ExplainForeignModify = postgresExplainForeignModify;
+ routine->ExplainDMLPushdown = postgresExplainDMLPushdown;
/* Support functions for ANALYZE */
routine->AnalyzeForeignTable = postgresAnalyzeForeignTable;
***************
*** 1067,1073 **** postgresGetForeignPlan(PlannerInfo *root,
* Build the fdw_private list that will be available to the executor.
* Items in the list must match enum FdwScanPrivateIndex, above.
*/
! fdw_private = list_make2(makeString(sql.data),
retrieved_attrs);
/*
--- 1139,1146 ----
* Build the fdw_private list that will be available to the executor.
* Items in the list must match enum FdwScanPrivateIndex, above.
*/
! fdw_private = list_make3(makeString(sql.data),
! remote_conds,
retrieved_attrs);
/*
***************
*** 1363,1375 **** postgresAddForeignUpdateTargets(Query *parsetree,
/*
* postgresPlanForeignModify
* Plan an insert/update/delete operation on a foreign table
- *
- * Note: currently, the plan tree generated for UPDATE/DELETE will always
- * include a ForeignScan that retrieves ctids (using SELECT FOR UPDATE)
- * and then the ModifyTable node will have to execute individual remote
- * UPDATE/DELETE commands. If there are no local conditions or joins
- * needed, it'd be better to let the scan node do UPDATE/DELETE RETURNING
- * and then do nothing at ModifyTable. Room for future optimization ...
*/
static List *
postgresPlanForeignModify(PlannerInfo *root,
--- 1436,1441 ----
***************
*** 1882,1887 **** postgresIsForeignRelUpdatable(Relation rel)
--- 1948,2286 ----
}
/*
+ * postgresPlanDMLPushdown
+ * Consider pushing down a foreign table modification to the remote server
+ *
+ * Decide whether the table modification is safe to push down to the remote end,
+ * and if so, modify subplan so as to do that.
+ *
+ * Conditions checked here:
+ *
+ * 1. The table modification must be an update or delete.
+ *
+ * 2. It's unsafe to push down the command if there are any local joins needed.
+ *
+ * 3. It's unsafe to push down the command if there are any quals that can't be
+ * evaluated remotely.
+ *
+ * 4. We can't push down an UPDATE, if any expressions to assign to the target
+ * columns are unsafe to evaluate on the remote end.
+ */
+ static bool
+ postgresPlanDMLPushdown(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index)
+ {
+ CmdType operation = plan->operation;
+ Plan *subplan = (Plan *) list_nth(plan->plans, subplan_index);
+ RangeTblEntry *rte = planner_rt_fetch(resultRelation, root);
+ Relation rel;
+ StringInfoData sql;
+ ForeignScan *fscan;
+ List *targetAttrs = NIL;
+ List *remote_conds;
+ List *params_list = NIL;
+ List *returningList = NIL;
+ List *retrieved_attrs = NIL;
+
+ /*
+ * Decide whether the table modification is safe to push down to the remote
+ * server.
+ */
+
+ /* Check point 1 */
+ if (operation == CMD_INSERT)
+ return false;
+
+ /* Check point 2 */
+ if (nodeTag(subplan) != T_ForeignScan)
+ return false;
+
+ /* Check point 3 */
+ if (subplan->qual != NIL)
+ return false;
+
+ /* Check point 4 */
+ if (operation == CMD_UPDATE)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[resultRelation];
+ int col;
+
+ /*
+ * We transmit only columns that were explicitly targets of the UPDATE,
+ * so as to avoid unnecessary data transmission.
+ */
+ col = -1;
+ while ((col = bms_next_member(rte->updatedCols, col)) >= 0)
+ {
+ /* bit numbers are offset by FirstLowInvalidHeapAttributeNumber */
+ AttrNumber attno = col + FirstLowInvalidHeapAttributeNumber;
+ TargetEntry *tle;
+
+ if (attno <= InvalidAttrNumber) /* shouldn't happen */
+ elog(ERROR, "system-column update is not supported");
+
+ tle = get_tle_by_resno(subplan->targetlist, attno);
+
+ if (!is_foreign_expr(root, baserel, (Expr *) tle->expr))
+ return false;
+
+ targetAttrs = lappend_int(targetAttrs, attno);
+ }
+ }
+
+ /*
+ * Ok, modify subplan so as to push down the command to the remote server.
+ */
+ fscan = (ForeignScan *) subplan;
+
+ initStringInfo(&sql);
+
+ /*
+ * Core code already has some lock on each rel being planned, so we can
+ * use NoLock here.
+ */
+ rel = heap_open(rte->relid, NoLock);
+
+ /*
+ * Extract the baserestrictinfo clauses that can be evaluated remotely.
+ */
+ remote_conds = (List *) list_nth(fscan->fdw_private,
+ FdwScanPrivateRemoteConds);
+
+ /*
+ * Extract the relevant RETURNING list if any.
+ */
+ if (plan->returningLists)
+ returningList = (List *) list_nth(plan->returningLists, subplan_index);
+
+ /*
+ * Construct the SQL command string.
+ */
+ switch (operation)
+ {
+ case CMD_UPDATE:
+ deparsePushedDownUpdateSql(&sql, root, resultRelation, rel,
+ ((Plan *) fscan)->targetlist,
+ targetAttrs,
+ remote_conds, ¶ms_list,
+ returningList, &retrieved_attrs);
+ break;
+ case CMD_DELETE:
+ deparsePushedDownDeleteSql(&sql, root, resultRelation, rel,
+ remote_conds, ¶ms_list,
+ returningList, &retrieved_attrs);
+ break;
+ default:
+ elog(ERROR, "unexpected operation: %d", (int) operation);
+ break;
+ }
+
+ /*
+ * Update the operation info.
+ */
+ fscan->operation = operation;
+
+ /*
+ * Update the fdw_exprs list that will be available to the executor.
+ */
+ fscan->fdw_exprs = params_list;
+
+ /*
+ * Update the fdw_private list that will be available to the executor.
+ * Items in the list must match enum FdwDmlPushdownPrivateIndex, above.
+ */
+ fscan->fdw_private = list_make4(makeString(sql.data),
+ makeInteger((retrieved_attrs != NIL)),
+ retrieved_attrs,
+ makeInteger(plan->canSetTag));
+
+ heap_close(rel, NoLock);
+ return true;
+ }
+
+ /*
+ * postgresBeginDMLPushdown
+ * Initiate pushing down a foreign table modification to the remote server
+ */
+ static void
+ postgresBeginDMLPushdown(ForeignScanState *node, int eflags)
+ {
+ ForeignScan *fsplan = (ForeignScan *) node->ss.ps.plan;
+ EState *estate = node->ss.ps.state;
+ PgFdwDmlPushdownState *dpstate;
+ RangeTblEntry *rte;
+ Oid userid;
+ ForeignTable *table;
+ ForeignServer *server;
+ UserMapping *user;
+ int numParams;
+ int i;
+ ListCell *lc;
+
+ /*
+ * Do nothing in EXPLAIN (no ANALYZE) case. node->fdw_state stays NULL.
+ */
+ if (eflags & EXEC_FLAG_EXPLAIN_ONLY)
+ return;
+
+ /*
+ * We'll save private state in node->fdw_state.
+ */
+ dpstate = (PgFdwDmlPushdownState *) palloc0(sizeof(PgFdwDmlPushdownState));
+ node->fdw_state = (void *) dpstate;
+
+ /*
+ * Identify which user to do the remote access as. This should match what
+ * ExecCheckRTEPerms() does.
+ */
+ rte = rt_fetch(fsplan->scan.scanrelid, estate->es_range_table);
+ userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
+
+ /* Get info about foreign table. */
+ dpstate->rel = node->ss.ss_currentRelation;
+ table = GetForeignTable(RelationGetRelid(dpstate->rel));
+ server = GetForeignServer(table->serverid);
+ user = GetUserMapping(userid, server->serverid);
+
+ /*
+ * Get connection to the foreign server. Connection manager will
+ * establish new connection if necessary.
+ */
+ dpstate->conn = GetConnection(server, user, false);
+
+ /* Initialize state variable */
+ dpstate->num_tuples = -1; /* -1 means not set yet */
+
+ /* Get private info created by planner functions. */
+ dpstate->query = strVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateUpdateSql));
+ dpstate->has_returning = intVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateHasReturning));
+ dpstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateRetrievedAttrs);
+ dpstate->set_processed = intVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateSetProcessed));
+
+ /* Create context for per-tuple temp workspace. */
+ dpstate->temp_cxt = AllocSetContextCreate(estate->es_query_cxt,
+ "postgres_fdw temporary data",
+ ALLOCSET_SMALL_MINSIZE,
+ ALLOCSET_SMALL_INITSIZE,
+ ALLOCSET_SMALL_MAXSIZE);
+
+ /* Prepare for input conversion of RETURNING results. */
+ if (dpstate->has_returning)
+ dpstate->attinmeta = TupleDescGetAttInMetadata(RelationGetDescr(dpstate->rel));
+
+ /* Prepare for output conversion of parameters used in remote query. */
+ numParams = list_length(fsplan->fdw_exprs);
+ dpstate->numParams = numParams;
+ dpstate->param_flinfo = (FmgrInfo *) palloc0(sizeof(FmgrInfo) * numParams);
+
+ i = 0;
+ foreach(lc, fsplan->fdw_exprs)
+ {
+ Node *param_expr = (Node *) lfirst(lc);
+ Oid typefnoid;
+ bool isvarlena;
+
+ getTypeOutputInfo(exprType(param_expr), &typefnoid, &isvarlena);
+ fmgr_info(typefnoid, &dpstate->param_flinfo[i]);
+ i++;
+ }
+
+ /*
+ * Prepare remote-parameter expressions for evaluation. (Note: in
+ * practice, we expect that all these expressions will be just Params, so
+ * we could possibly do something more efficient than using the full
+ * expression-eval machinery for this. But probably there would be little
+ * benefit, and it'd require postgres_fdw to know more than is desirable
+ * about Param evaluation.)
+ */
+ dpstate->param_exprs = (List *)
+ ExecInitExpr((Expr *) fsplan->fdw_exprs,
+ (PlanState *) node);
+
+ /*
+ * Allocate buffer for text form of query parameters, if any.
+ */
+ if (numParams > 0)
+ dpstate->param_values = (const char **) palloc0(numParams * sizeof(char *));
+ else
+ dpstate->param_values = NULL;
+ }
+
+ /*
+ * postgresIterateDMLPushdown
+ * Execute pushing down a foreign table modification to the remote server
+ */
+ static TupleTableSlot *
+ postgresIterateDMLPushdown(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ EState *estate = node->ss.ps.state;
+ ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
+
+ /*
+ * If this is the first call after Begin, execute the statement.
+ */
+ if (dpstate->num_tuples == -1)
+ execute_dml_stmt(node);
+
+ /*
+ * If the local query doesn't specify RETURNING, just clear tuple slot.
+ */
+ if (!resultRelInfo->ri_projectReturning)
+ {
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+ Instrumentation *instr = node->ss.ps.instrument;
+
+ Assert(!dpstate->has_returning);
+
+ /* Increment the command es_processed count if necessary. */
+ if (dpstate->set_processed)
+ estate->es_processed += dpstate->num_tuples;
+
+ /* Increment the tuple count for EXPLAIN ANALYZE if necessary. */
+ if (instr)
+ instr->tuplecount += dpstate->num_tuples;
+
+ return ExecClearTuple(slot);
+ }
+
+ /*
+ * Get the next RETURNING tuple.
+ */
+ return get_returning_data(node);
+ }
+
+ /*
+ * postgresEndDMLPushdown
+ * Finish pushing down a foreign table modification to the remote server
+ */
+ static void
+ postgresEndDMLPushdown(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+
+ /* if dpstate is NULL, we are in EXPLAIN; nothing to do */
+ if (dpstate == NULL)
+ return;
+
+ /* Release PGresult */
+ if (dpstate->result)
+ PQclear(dpstate->result);
+
+ /* Release remote connection */
+ ReleaseConnection(dpstate->conn);
+ dpstate->conn = NULL;
+
+ /* MemoryContexts will be deleted automatically. */
+ }
+
+ /*
* postgresExplainForeignScan
* Produce extra output for EXPLAIN of a ForeignScan on a foreign table
*/
***************
*** 1919,1924 **** postgresExplainForeignModify(ModifyTableState *mtstate,
--- 2318,2342 ----
}
}
+ /*
+ * postgresExplainDMLPushdown
+ * Produce extra output for EXPLAIN of a ForeignScan on a foreign table
+ * that has pushed down an UPDATE/DELETE to the remote server
+ */
+ static void
+ postgresExplainDMLPushdown(ForeignScanState *node, ExplainState *es)
+ {
+ List *fdw_private;
+ char *sql;
+
+ if (es->verbose)
+ {
+ fdw_private = ((ForeignScan *) node->ss.ps.plan)->fdw_private;
+ sql = strVal(list_nth(fdw_private, FdwDmlPushdownPrivateUpdateSql));
+ ExplainPropertyText("Remote SQL", sql, es);
+ }
+ }
+
/*
* estimate_path_cost_size
***************
*** 2535,2540 **** store_returning_result(PgFdwModifyState *fmstate,
--- 2953,3087 ----
}
/*
+ * Execute a pushed-down UPDATE/DELETE statement.
+ */
+ static void
+ execute_dml_stmt(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ ExprContext *econtext = node->ss.ps.ps_ExprContext;
+ int numParams = dpstate->numParams;
+ const char **values = dpstate->param_values;
+
+ /*
+ * Construct array of query parameter values in text format.
+ */
+ if (numParams > 0)
+ {
+ int nestlevel;
+ int i;
+ ListCell *lc;
+
+ nestlevel = set_transmission_modes();
+
+ i = 0;
+ foreach(lc, dpstate->param_exprs)
+ {
+ ExprState *expr_state = (ExprState *) lfirst(lc);
+ Datum expr_value;
+ bool isNull;
+
+ /* Evaluate the parameter expression */
+ expr_value = ExecEvalExpr(expr_state, econtext, &isNull, NULL);
+
+ /*
+ * Get string representation of each parameter value by invoking
+ * type-specific output function, unless the value is null.
+ */
+ if (isNull)
+ values[i] = NULL;
+ else
+ values[i] = OutputFunctionCall(&dpstate->param_flinfo[i],
+ expr_value);
+ i++;
+ }
+
+ reset_transmission_modes(nestlevel);
+ }
+
+ /*
+ * Notice that we pass NULL for paramTypes, thus forcing the remote server
+ * to infer types for all parameters. Since we explicitly cast every
+ * parameter (see deparse.c), the "inference" is trivial and will produce
+ * the desired result. This allows us to avoid assuming that the remote
+ * server has the same OIDs we do for the parameters' types.
+ *
+ * We don't use a PG_TRY block here, so be careful not to throw error
+ * without releasing the PGresult.
+ */
+ dpstate->result = PQexecParams(dpstate->conn, dpstate->query,
+ numParams, NULL, values, NULL, NULL, 0);
+ if (PQresultStatus(dpstate->result) !=
+ (dpstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
+ pgfdw_report_error(ERROR, dpstate->result, dpstate->conn, true,
+ dpstate->query);
+
+ /* Get the number of rows affected. */
+ if (dpstate->has_returning)
+ dpstate->num_tuples = PQntuples(dpstate->result);
+ else
+ dpstate->num_tuples = atoi(PQcmdTuples(dpstate->result));
+ }
+
+ /*
+ * Get the result of a RETURNING clause.
+ */
+ static TupleTableSlot *
+ get_returning_data(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ EState *estate = node->ss.ps.state;
+ ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+
+ Assert(resultRelInfo->ri_projectReturning);
+
+ /* If we didn't get any tuples, must be end of data. */
+ if (dpstate->next_tuple >= dpstate->num_tuples)
+ return ExecClearTuple(slot);
+
+ /* Increment the command es_processed count if necessary. */
+ if (dpstate->set_processed)
+ estate->es_processed += 1;
+
+ /*
+ * Store a RETURNING tuple. Note that if the local query is of the form
+ * e.g., UPDATE/DELETE .. RETURNING 1, we have has_returning=false, so
+ * just emit a dummy tuple in that case.
+ */
+ if (!dpstate->has_returning)
+ ExecStoreAllNullTuple(slot);
+ else
+ {
+ PG_TRY();
+ {
+ HeapTuple newtup;
+
+ newtup = make_tuple_from_result_row(dpstate->result,
+ dpstate->next_tuple,
+ dpstate->rel,
+ dpstate->attinmeta,
+ dpstate->retrieved_attrs,
+ dpstate->temp_cxt);
+ ExecStoreTuple(newtup, slot, InvalidBuffer, false);
+ }
+ PG_CATCH();
+ {
+ if (dpstate->result)
+ PQclear(dpstate->result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ }
+ dpstate->next_tuple++;
+
+ /* Make slot available for evaluation of the local query RETURNING list. */
+ resultRelInfo->ri_projectReturning->pi_exprContext->ecxt_scantuple = slot;
+
+ return slot;
+ }
+
+ /*
* postgresAnalyzeForeignTable
* Test whether analyzing this foreign table is supported
*/
*** a/contrib/postgres_fdw/postgres_fdw.h
--- b/contrib/postgres_fdw/postgres_fdw.h
***************
*** 103,112 **** extern void deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 103,126 ----
Index rtindex, Relation rel,
List *targetAttrs, List *returningList,
List **retrieved_attrs);
+ extern void deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseDeleteSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *returningList,
List **retrieved_attrs);
+ extern void deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseAnalyzeSizeSql(StringInfo buf, Relation rel);
extern void deparseAnalyzeSql(StringInfo buf, Relation rel,
List **retrieved_attrs);
*** a/contrib/postgres_fdw/sql/postgres_fdw.sql
--- b/contrib/postgres_fdw/sql/postgres_fdw.sql
***************
*** 399,416 **** INSERT INTO ft2 (c1,c2,c3) SELECT c1+1000,c2+100, c3 || c3 FROM ft2 LIMIT 20;
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
EXPLAIN (verbose, costs off)
--- 399,420 ----
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
EXPLAIN (verbose, costs off)
***************
*** 737,742 **** UPDATE rem1 SET f2 = 'testo';
--- 741,830 ----
-- Test returning a system attribute
INSERT INTO rem1(f2) VALUES ('test') RETURNING ctid;
+ -- cleanup
+ DROP TRIGGER trig_row_before ON rem1;
+ DROP TRIGGER trig_row_after ON rem1;
+ DROP TRIGGER trig_local_before ON loc1;
+
+
+ -- Test DML pushdown functionality
+
+ -- Test with statement-level triggers
+ CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_stmt_before ON rem1;
+
+ CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_stmt_after ON rem1;
+
+ -- Test with row-level ON INSERT triggers
+ CREATE TRIGGER trig_row_before_insert
+ BEFORE INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_before_insert ON rem1;
+
+ CREATE TRIGGER trig_row_after_insert
+ AFTER INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_after_insert ON rem1;
+
+ -- Test with row-level ON UPDATE triggers
+ CREATE TRIGGER trig_row_before_update
+ BEFORE UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_before_update ON rem1;
+
+ CREATE TRIGGER trig_row_after_update
+ AFTER UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_after_update ON rem1;
+
+ -- Test with row-level ON DELETE triggers
+ CREATE TRIGGER trig_row_before_delete
+ BEFORE DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ DROP TRIGGER trig_row_before_delete ON rem1;
+
+ CREATE TRIGGER trig_row_after_delete
+ AFTER DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ DROP TRIGGER trig_row_after_delete ON rem1;
+
-- ===================================================================
-- test inheritance features
-- ===================================================================
***************
*** 868,873 **** fetch from c;
--- 956,968 ----
update bar set f2 = null where current of c;
rollback;
+ explain (verbose, costs off)
+ delete from foo where f1 < 5 returning *;
+ delete from foo where f1 < 5 returning *;
+ explain (verbose, costs off)
+ update bar set f2 = f2 + 100 returning *;
+ update bar set f2 = f2 + 100 returning *;
+
drop table foo cascade;
drop table bar cascade;
drop table loct1;
*** a/doc/src/sgml/fdwhandler.sgml
--- b/doc/src/sgml/fdwhandler.sgml
***************
*** 664,669 **** IsForeignRelUpdatable (Relation rel);
--- 664,807 ----
updatability for display in the <literal>information_schema</> views.)
</para>
+ <para>
+ <programlisting>
+ bool
+ PlanDMLPushdown (PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+ </programlisting>
+
+ Decide whether it is safe to execute a foreign table update directly
+ on the remote server. If so, return <literal>true</> after performing
+ planning actions needed for that. Otherwise, return <literal>false</>.
+ This function is executed just before <function>PlanForeignModify</>
+ is called. If this function succeeds, <function>PlanForeignModify</>
+ won't be executed, and <function>BeginDMLPushdown</>,
+ <function>IterateDMLPushdown</> and <function>EndDMLPushdown</> will
+ be called at the execution stage, instead. Otherwise, the table update
+ will be executed using the table-updating functions described above.
+ The parameters are the same as for <function>PlanForeignModify</>.
+ </para>
+
+ <para>
+ To execute the table update directly on the remote server, this function
+ must rewrite the target subplan with a <structname>ForeignScan</> plan
+ node that executes the table update directly on the remote server. The
+ <structfield>operation</> field of the <structname>ForeignScan</> must
+ be set to the <literal>CmdType</> enumeration appropriately; that is,
+ <literal>CMD_INSERT</> for <command>INSERT</>,
+ <literal>CMD_UPDATE</> for <command>UPDATE</>, and
+ <literal>CMD_DELETE</> for <command>DELETE</>.
+ </para>
+
+ <para>
+ See <xref linkend="fdw-planning"> for additional information.
+ </para>
+
+ <para>
+ If the <function>PlanDMLPushdown</> pointer is set to
+ <literal>NULL</>, no attempts to execute the table update directly on
+ the remote server are taken.
+ </para>
+
+ <para>
+ <programlisting>
+ void
+ BeginDMLPushdown (ForeignScanState *node,
+ int eflags);
+ </programlisting>
+
+ Begin executing a foreign table update directly on the remote server.
+ This is called during executor startup. It should perform any
+ initialization needed prior to the actual table update (that should be
+ done upon the first call to <function>IterateDMLPushdown</>).
+ The <structname>ForeignScanState</> node has already been created, but
+ its <structfield>fdw_state</> field is still NULL. Information about
+ the table to update is accessible through the
+ <structname>ForeignScanState</> node (in particular, from the underlying
+ <structname>ForeignScan</> plan node, which contains any FDW-private
+ information provided by <function>PlanDMLPushdown</>).
+ <literal>eflags</> contains flag bits describing the executor's
+ operating mode for this plan node.
+ </para>
+
+ <para>
+ Note that when <literal>(eflags & EXEC_FLAG_EXPLAIN_ONLY)</> is
+ true, this function should not perform any externally-visible actions;
+ it should only do the minimum required to make the node state valid
+ for <function>ExplainDMLPushdown</> and <function>EndDMLPushdown</>.
+ </para>
+
+ <para>
+ If the <function>BeginDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to execute the table update directly on
+ the remote server will fail with an error message.
+ </para>
+
+ <para>
+ <programlisting>
+ TupleTableSlot *
+ IterateDMLPushdown (ForeignScanState *node);
+ </programlisting>
+
+ When the <command>INSERT</>, <command>UPDATE</> or <command>DELETE</>
+ query doesn't have a <literal>RETURNING</> clause, just return NULL
+ after the actual table update directly executed on the remote server.
+ When the query has the clause, fetch one result containing the data
+ needed for the <literal>RETURNING</> calculation, returning it in a
+ tuple table slot (the node's <structfield>ScanTupleSlot</> should be
+ used for this purpose). The data that was actually inserted, updated
+ or deleted must be provided to
+ <literal>rinfo->ri_projectReturning</>.
+ Return NULL if no more rows are available.
+ Note that this is called in a short-lived memory context that will be
+ reset between invocations. Create a memory context in
+ <function>BeginDMLPushdown</> if you need longer-lived storage, or use
+ the <structfield>es_query_cxt</> of the node's <structname>EState</>.
+ </para>
+
+ <para>
+ The rows returned must match the <structfield>fdw_scan_tlist</> target
+ list if one was supplied, otherwise they must match the row type of the
+ foreign table being updated. If you choose to optimize away fetching
+ columns that are not needed for the <literal>RETURNING</> calculation,
+ you should insert nulls in those column positions, or else generate a
+ <structfield>fdw_scan_tlist</> list with those columns omitted.
+ </para>
+
+ <para>
+ Whether the query has the clause or not, the query's reported row count
+ must be incremented by the FDW itself. When the query doesn't has the
+ clause, the FDW must also increment the row count for the
+ <structname>ForeignScanState</> node in the <command>EXPLAIN ANALYZE</>
+ case.
+ </para>
+
+ <para>
+ If the <function>IterateDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to execute the table update directly on
+ the remote server will fail with an error message.
+ </para>
+
+ <para>
+ <programlisting>
+ void
+ EndDMLPushdown (ForeignScanState *node);
+ </programlisting>
+
+ End the table update and release resources. It is normally not important
+ to release palloc'd memory, but for example open files and connections
+ to remote servers should be cleaned up.
+ </para>
+
+ <para>
+ If the <function>EndDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to execute the table update directly on
+ the remote server will fail with an error message.
+ </para>
+
</sect2>
<sect2 id="fdw-callbacks-row-locking">
***************
*** 848,853 **** ExplainForeignModify (ModifyTableState *mtstate,
--- 986,1014 ----
<command>EXPLAIN</>.
</para>
+ <para>
+ <programlisting>
+ void
+ ExplainDMLPushdown (ForeignScanState *node,
+ ExplainState *es);
+ </programlisting>
+
+ Print additional <command>EXPLAIN</> output for a foreign table update
+ that is executed directly on the remote server.
+ This function can call <function>ExplainPropertyText</> and
+ related functions to add fields to the <command>EXPLAIN</> output.
+ The flag fields in <literal>es</> can be used to determine what to
+ print, and the state of the <structname>ForeignScanState</> node
+ can be inspected to provide run-time statistics in the <command>EXPLAIN
+ ANALYZE</> case.
+ </para>
+
+ <para>
+ If the <function>ExplainDMLPushdown</> pointer is set to
+ <literal>NULL</>, no additional information is printed during
+ <command>EXPLAIN</>.
+ </para>
+
</sect2>
<sect2 id="fdw-callbacks-analyze">
***************
*** 1068,1074 **** GetForeignServerByName(const char *name, bool missing_ok);
<para>
The FDW callback functions <function>GetForeignRelSize</>,
<function>GetForeignPaths</>, <function>GetForeignPlan</>,
! <function>PlanForeignModify</>, and <function>GetForeignJoinPaths</>
must fit into the workings of the <productname>PostgreSQL</> planner.
Here are some notes about what they must do.
</para>
--- 1229,1236 ----
<para>
The FDW callback functions <function>GetForeignRelSize</>,
<function>GetForeignPaths</>, <function>GetForeignPlan</>,
! <function>PlanForeignModify</>, <function>GetForeignJoinPaths</>, and
! <function>PlanDMLPushdown</>
must fit into the workings of the <productname>PostgreSQL</> planner.
Here are some notes about what they must do.
</para>
***************
*** 1228,1236 **** GetForeignServerByName(const char *name, bool missing_ok);
<para>
When planning an <command>UPDATE</> or <command>DELETE</>,
! <function>PlanForeignModify</> can look up the <structname>RelOptInfo</>
! struct for the foreign table and make use of the
! <literal>baserel->fdw_private</> data previously created by the
scan-planning functions. However, in <command>INSERT</> the target
table is not scanned so there is no <structname>RelOptInfo</> for it.
The <structname>List</> returned by <function>PlanForeignModify</> has
--- 1390,1398 ----
<para>
When planning an <command>UPDATE</> or <command>DELETE</>,
! <function>PlanForeignModify</> and <function>PlanDMLPushdown</> can look
! up the <structname>RelOptInfo</> struct for the foreign table and make use
! of the <literal>baserel->fdw_private</> data previously created by the
scan-planning functions. However, in <command>INSERT</> the target
table is not scanned so there is no <structname>RelOptInfo</> for it.
The <structname>List</> returned by <function>PlanForeignModify</> has
*** a/doc/src/sgml/postgres-fdw.sgml
--- b/doc/src/sgml/postgres-fdw.sgml
***************
*** 471,476 ****
--- 471,485 ----
extension that's listed in the foreign server's <literal>extensions</>
option. Operators and functions in such clauses must
be <literal>IMMUTABLE</> as well.
+ For an <command>UPDATE</> or <command>DELETE</> query,
+ <filename>postgres_fdw</> attempts to optimize the query execution by
+ sending the whole query to the remote server if there are no query
+ <literal>WHERE</> clauses that cannot be sent to the remote server,
+ no local joins for the query, and no row-level local <literal>BEFORE</> or
+ <literal>AFTER</> triggers on the target table. In <command>UPDATE</>,
+ expressions to assign to target columns must use only built-in data types,
+ <literal>IMMUTABLE</> operators, or <literal>IMMUTABLE</> functions,
+ to reduce the risk of misexecution of the query.
</para>
<para>
*** a/src/backend/commands/explain.c
--- b/src/backend/commands/explain.c
***************
*** 888,894 **** ExplainNode(PlanState *planstate, List *ancestors,
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
! pname = sname = "Foreign Scan";
break;
case T_CustomScan:
sname = "Custom Scan";
--- 888,916 ----
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
! sname = "Foreign Scan";
! switch (((ForeignScan *) plan)->operation)
! {
! case CMD_SELECT:
! pname = "Foreign Scan";
! operation = "Select";
! break;
! case CMD_INSERT:
! pname = "Foreign Insert";
! operation = "Insert";
! break;
! case CMD_UPDATE:
! pname = "Foreign Update";
! operation = "Update";
! break;
! case CMD_DELETE:
! pname = "Foreign Delete";
! operation = "Delete";
! break;
! default:
! pname = "???";
! break;
! }
break;
case T_CustomScan:
sname = "Custom Scan";
***************
*** 1636,1641 **** show_plan_tlist(PlanState *planstate, List *ancestors, ExplainState *es)
--- 1658,1669 ----
return;
if (IsA(plan, RecursiveUnion))
return;
+ /* Likewise for ForeignScan that has pushed down INSERT/UPDATE/DELETE */
+ if (IsA(plan, ForeignScan) &&
+ (((ForeignScan *) plan)->operation == CMD_INSERT ||
+ ((ForeignScan *) plan)->operation == CMD_UPDATE ||
+ ((ForeignScan *) plan)->operation == CMD_DELETE))
+ return;
/* Set up deparsing context */
context = set_deparse_context_planstate(es->deparse_cxt,
***************
*** 2224,2231 **** show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es)
FdwRoutine *fdwroutine = fsstate->fdwroutine;
/* Let the FDW emit whatever fields it wants */
! if (fdwroutine->ExplainForeignScan != NULL)
! fdwroutine->ExplainForeignScan(fsstate, es);
}
/*
--- 2252,2267 ----
FdwRoutine *fdwroutine = fsstate->fdwroutine;
/* Let the FDW emit whatever fields it wants */
! if (((ForeignScan *) fsstate->ss.ps.plan)->operation != CMD_SELECT)
! {
! if (fdwroutine->ExplainDMLPushdown != NULL)
! fdwroutine->ExplainDMLPushdown(fsstate, es);
! }
! else
! {
! if (fdwroutine->ExplainForeignScan != NULL)
! fdwroutine->ExplainForeignScan(fsstate, es);
! }
}
/*
***************
*** 2611,2618 **** show_modifytable_info(ModifyTableState *mtstate, List *ancestors,
}
}
! /* Give FDW a chance */
! if (fdwroutine && fdwroutine->ExplainForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, j);
--- 2647,2656 ----
}
}
! /* Give FDW a chance if needed */
! if (!resultRelInfo->ri_FdwPushdown &&
! fdwroutine != NULL &&
! fdwroutine->ExplainForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, j);
*** a/src/backend/executor/execMain.c
--- b/src/backend/executor/execMain.c
***************
*** 1011,1020 **** InitPlan(QueryDesc *queryDesc, int eflags)
* CheckValidRowMarkRel.
*/
void
! CheckValidResultRel(Relation resultRel, CmdType operation)
{
TriggerDesc *trigDesc = resultRel->trigdesc;
FdwRoutine *fdwroutine;
switch (resultRel->rd_rel->relkind)
{
--- 1011,1022 ----
* CheckValidRowMarkRel.
*/
void
! CheckValidResultRel(ResultRelInfo *resultRelInfo, CmdType operation)
{
+ Relation resultRel = resultRelInfo->ri_RelationDesc;
TriggerDesc *trigDesc = resultRel->trigdesc;
FdwRoutine *fdwroutine;
+ bool allow_pushdown;
switch (resultRel->rd_rel->relkind)
{
***************
*** 1083,1096 **** CheckValidResultRel(Relation resultRel, CmdType operation)
case RELKIND_FOREIGN_TABLE:
/* Okay only if the FDW supports it */
fdwroutine = GetFdwRoutineForRelation(resultRel, false);
switch (operation)
{
case CMD_INSERT:
! if (fdwroutine->ExecForeignInsert == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("cannot insert into foreign table \"%s\"",
! RelationGetRelationName(resultRel))));
if (fdwroutine->IsForeignRelUpdatable != NULL &&
(fdwroutine->IsForeignRelUpdatable(resultRel) & (1 << CMD_INSERT)) == 0)
ereport(ERROR,
--- 1085,1107 ----
case RELKIND_FOREIGN_TABLE:
/* Okay only if the FDW supports it */
fdwroutine = GetFdwRoutineForRelation(resultRel, false);
+ allow_pushdown = ((fdwroutine->BeginDMLPushdown != NULL) &&
+ (fdwroutine->IterateDMLPushdown != NULL) &&
+ (fdwroutine->EndDMLPushdown != NULL));
switch (operation)
{
case CMD_INSERT:
! if (resultRelInfo->ri_FdwPushdown && !allow_pushdown)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("cannot push down insert on foreign table \"%s\"",
! RelationGetRelationName(resultRel))));
! if (!resultRelInfo->ri_FdwPushdown &&
! fdwroutine->ExecForeignInsert == NULL)
! ereport(ERROR,
! (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("cannot insert into foreign table \"%s\"",
! RelationGetRelationName(resultRel))));
if (fdwroutine->IsForeignRelUpdatable != NULL &&
(fdwroutine->IsForeignRelUpdatable(resultRel) & (1 << CMD_INSERT)) == 0)
ereport(ERROR,
***************
*** 1099,1105 **** CheckValidResultRel(Relation resultRel, CmdType operation)
RelationGetRelationName(resultRel))));
break;
case CMD_UPDATE:
! if (fdwroutine->ExecForeignUpdate == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot update foreign table \"%s\"",
--- 1110,1122 ----
RelationGetRelationName(resultRel))));
break;
case CMD_UPDATE:
! if (resultRelInfo->ri_FdwPushdown && !allow_pushdown)
! ereport(ERROR,
! (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("cannot push down update on foreign table \"%s\"",
! RelationGetRelationName(resultRel))));
! if (!resultRelInfo->ri_FdwPushdown &&
! fdwroutine->ExecForeignUpdate == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot update foreign table \"%s\"",
***************
*** 1112,1118 **** CheckValidResultRel(Relation resultRel, CmdType operation)
RelationGetRelationName(resultRel))));
break;
case CMD_DELETE:
! if (fdwroutine->ExecForeignDelete == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot delete from foreign table \"%s\"",
--- 1129,1141 ----
RelationGetRelationName(resultRel))));
break;
case CMD_DELETE:
! if (resultRelInfo->ri_FdwPushdown && !allow_pushdown)
! ereport(ERROR,
! (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! errmsg("cannot push down delete on foreign table \"%s\"",
! RelationGetRelationName(resultRel))));
! if (!resultRelInfo->ri_FdwPushdown &&
! fdwroutine->ExecForeignDelete == NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot delete from foreign table \"%s\"",
***************
*** 1245,1250 **** InitResultRelInfo(ResultRelInfo *resultRelInfo,
--- 1268,1274 ----
else
resultRelInfo->ri_FdwRoutine = NULL;
resultRelInfo->ri_FdwState = NULL;
+ resultRelInfo->ri_FdwPushdown = false;
resultRelInfo->ri_ConstraintExprs = NULL;
resultRelInfo->ri_junkFilter = NULL;
resultRelInfo->ri_projectReturning = NULL;
*** a/src/backend/executor/nodeForeignscan.c
--- b/src/backend/executor/nodeForeignscan.c
***************
*** 48,54 **** ForeignNext(ForeignScanState *node)
/* Call the Iterate function in short-lived context */
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! slot = node->fdwroutine->IterateForeignScan(node);
MemoryContextSwitchTo(oldcontext);
/*
--- 48,57 ----
/* Call the Iterate function in short-lived context */
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! if (plan->operation != CMD_SELECT)
! slot = node->fdwroutine->IterateDMLPushdown(node);
! else
! slot = node->fdwroutine->IterateForeignScan(node);
MemoryContextSwitchTo(oldcontext);
/*
***************
*** 226,232 **** ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
/*
* Tell the FDW to initialize the scan.
*/
! fdwroutine->BeginForeignScan(scanstate, eflags);
return scanstate;
}
--- 229,238 ----
/*
* Tell the FDW to initialize the scan.
*/
! if (node->operation != CMD_SELECT)
! fdwroutine->BeginDMLPushdown(scanstate, eflags);
! else
! fdwroutine->BeginForeignScan(scanstate, eflags);
return scanstate;
}
***************
*** 240,247 **** ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
void
ExecEndForeignScan(ForeignScanState *node)
{
/* Let the FDW shut down */
! node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
--- 246,258 ----
void
ExecEndForeignScan(ForeignScanState *node)
{
+ ForeignScan *plan = (ForeignScan *) node->ss.ps.plan;
+
/* Let the FDW shut down */
! if (plan->operation != CMD_SELECT)
! node->fdwroutine->EndDMLPushdown(node);
! else
! node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
*** a/src/backend/executor/nodeModifyTable.c
--- b/src/backend/executor/nodeModifyTable.c
***************
*** 138,150 **** ExecCheckPlanOutput(Relation resultRel, List *targetList)
* tupleSlot: slot holding tuple actually inserted/updated/deleted
* planSlot: slot holding tuple returned by top subplan node
*
* Returns a slot holding the result tuple
*/
static TupleTableSlot *
! ExecProcessReturning(ProjectionInfo *projectReturning,
TupleTableSlot *tupleSlot,
TupleTableSlot *planSlot)
{
ExprContext *econtext = projectReturning->pi_exprContext;
/*
--- 138,154 ----
* tupleSlot: slot holding tuple actually inserted/updated/deleted
* planSlot: slot holding tuple returned by top subplan node
*
+ * Note: If tupleSlot is NULL, the FDW should have already provided econtext's
+ * scan tuple.
+ *
* Returns a slot holding the result tuple
*/
static TupleTableSlot *
! ExecProcessReturning(ResultRelInfo *resultRelInfo,
TupleTableSlot *tupleSlot,
TupleTableSlot *planSlot)
{
+ ProjectionInfo *projectReturning = resultRelInfo->ri_projectReturning;
ExprContext *econtext = projectReturning->pi_exprContext;
/*
***************
*** 154,160 **** ExecProcessReturning(ProjectionInfo *projectReturning,
ResetExprContext(econtext);
/* Make tuple and any needed join variables available to ExecProject */
! econtext->ecxt_scantuple = tupleSlot;
econtext->ecxt_outertuple = planSlot;
/* Compute the RETURNING expressions */
--- 158,177 ----
ResetExprContext(econtext);
/* Make tuple and any needed join variables available to ExecProject */
! if (tupleSlot)
! econtext->ecxt_scantuple = tupleSlot;
! else
! {
! HeapTuple tuple;
!
! /*
! * RETURNING expressions might reference the tableoid column, so
! * initialize t_tableOid before evaluating them.
! */
! Assert(!TupIsNull(econtext->ecxt_scantuple));
! tuple = ExecMaterializeSlot(econtext->ecxt_scantuple);
! tuple->t_tableOid = RelationGetRelid(resultRelInfo->ri_RelationDesc);
! }
econtext->ecxt_outertuple = planSlot;
/* Compute the RETURNING expressions */
***************
*** 496,503 **** ExecInsert(ModifyTableState *mtstate,
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo->ri_projectReturning,
! slot, planSlot);
return NULL;
}
--- 513,519 ----
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo, slot, planSlot);
return NULL;
}
***************
*** 738,745 **** ldelete:;
ExecStoreTuple(&deltuple, slot, InvalidBuffer, false);
}
! rslot = ExecProcessReturning(resultRelInfo->ri_projectReturning,
! slot, planSlot);
/*
* Before releasing the target tuple again, make sure rslot has a
--- 754,760 ----
ExecStoreTuple(&deltuple, slot, InvalidBuffer, false);
}
! rslot = ExecProcessReturning(resultRelInfo, slot, planSlot);
/*
* Before releasing the target tuple again, make sure rslot has a
***************
*** 1024,1031 **** lreplace:;
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo->ri_projectReturning,
! slot, planSlot);
return NULL;
}
--- 1039,1045 ----
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo, slot, planSlot);
return NULL;
}
***************
*** 1380,1385 **** ExecModifyTable(ModifyTableState *node)
--- 1394,1414 ----
break;
}
+ /*
+ * If ri_FdwPushdown is true, all we need to do here is compute the
+ * RETURNING expressions.
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ Assert(resultRelInfo->ri_projectReturning);
+
+ /* No need to provide scan tuple to ExecProcessReturning. */
+ slot = ExecProcessReturning(resultRelInfo, NULL, planSlot);
+
+ estate->es_result_relation_info = saved_resultRelInfo;
+ return slot;
+ }
+
EvalPlanQualSetSlot(&node->mt_epqstate, planSlot);
slot = planSlot;
***************
*** 1559,1568 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
{
subplan = (Plan *) lfirst(l);
/*
* Verify result relation is a valid target for the current operation
*/
! CheckValidResultRel(resultRelInfo->ri_RelationDesc, operation);
/*
* If there are indices on the result relation, open them and save
--- 1588,1600 ----
{
subplan = (Plan *) lfirst(l);
+ /* Initialize the FdwPushdown flag */
+ resultRelInfo->ri_FdwPushdown = list_nth_int(node->fdwPushdowns, i);
+
/*
* Verify result relation is a valid target for the current operation
*/
! CheckValidResultRel(resultRelInfo, operation);
/*
* If there are indices on the result relation, open them and save
***************
*** 1583,1589 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
/* Also let FDWs init themselves for foreign-table result rels */
! if (resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, i);
--- 1615,1622 ----
mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
/* Also let FDWs init themselves for foreign-table result rels */
! if (!resultRelInfo->ri_FdwPushdown &&
! resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, i);
***************
*** 1754,1766 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
--- 1787,1811 ----
erm = ExecFindRowMark(estate, rc->rti, false);
/* build ExecAuxRowMark for each subplan */
+ resultRelInfo = mtstate->resultRelInfo;
for (i = 0; i < nplans; i++)
{
ExecAuxRowMark *aerm;
+ /*
+ * ignore subplan if the FDW pushes down the command to the remote
+ * server
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ resultRelInfo++;
+ continue;
+ }
+
subplan = mtstate->mt_plans[i]->plan;
aerm = ExecBuildAuxRowMark(erm, subplan->targetlist);
mtstate->mt_arowmarks[i] = lappend(mtstate->mt_arowmarks[i], aerm);
+ resultRelInfo++;
}
}
***************
*** 1821,1826 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
--- 1866,1881 ----
ExecCheckPlanOutput(resultRelInfo->ri_RelationDesc,
subplan->targetlist);
+ /*
+ * ignore subplan if the FDW pushes down the command to the
+ * remote server
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ resultRelInfo++;
+ continue;
+ }
+
j = ExecInitJunkFilter(subplan->targetlist,
resultRelInfo->ri_RelationDesc->rd_att->tdhasoid,
ExecInitExtraTupleSlot(estate));
***************
*** 1910,1916 **** ExecEndModifyTable(ModifyTableState *node)
{
ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
! if (resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->EndForeignModify != NULL)
resultRelInfo->ri_FdwRoutine->EndForeignModify(node->ps.state,
resultRelInfo);
--- 1965,1972 ----
{
ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
! if (!resultRelInfo->ri_FdwPushdown &&
! resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->EndForeignModify != NULL)
resultRelInfo->ri_FdwRoutine->EndForeignModify(node->ps.state,
resultRelInfo);
*** a/src/backend/nodes/copyfuncs.c
--- b/src/backend/nodes/copyfuncs.c
***************
*** 186,191 **** _copyModifyTable(const ModifyTable *from)
--- 186,192 ----
COPY_NODE_FIELD(withCheckOptionLists);
COPY_NODE_FIELD(returningLists);
COPY_NODE_FIELD(fdwPrivLists);
+ COPY_NODE_FIELD(fdwPushdowns);
COPY_NODE_FIELD(rowMarks);
COPY_SCALAR_FIELD(epqParam);
COPY_SCALAR_FIELD(onConflictAction);
***************
*** 645,650 **** _copyForeignScan(const ForeignScan *from)
--- 646,652 ----
/*
* copy remainder of node
*/
+ COPY_SCALAR_FIELD(operation);
COPY_SCALAR_FIELD(fs_server);
COPY_NODE_FIELD(fdw_exprs);
COPY_NODE_FIELD(fdw_private);
*** a/src/backend/nodes/outfuncs.c
--- b/src/backend/nodes/outfuncs.c
***************
*** 340,345 **** _outModifyTable(StringInfo str, const ModifyTable *node)
--- 340,346 ----
WRITE_NODE_FIELD(withCheckOptionLists);
WRITE_NODE_FIELD(returningLists);
WRITE_NODE_FIELD(fdwPrivLists);
+ WRITE_NODE_FIELD(fdwPushdowns);
WRITE_NODE_FIELD(rowMarks);
WRITE_INT_FIELD(epqParam);
WRITE_ENUM_FIELD(onConflictAction, OnConflictAction);
***************
*** 591,596 **** _outForeignScan(StringInfo str, const ForeignScan *node)
--- 592,598 ----
_outScanInfo(str, (const Scan *) node);
+ WRITE_ENUM_FIELD(operation, CmdType);
WRITE_OID_FIELD(fs_server);
WRITE_NODE_FIELD(fdw_exprs);
WRITE_NODE_FIELD(fdw_private);
*** a/src/backend/nodes/readfuncs.c
--- b/src/backend/nodes/readfuncs.c
***************
*** 1471,1476 **** _readModifyTable(void)
--- 1471,1477 ----
READ_NODE_FIELD(withCheckOptionLists);
READ_NODE_FIELD(returningLists);
READ_NODE_FIELD(fdwPrivLists);
+ READ_NODE_FIELD(fdwPushdowns);
READ_NODE_FIELD(rowMarks);
READ_INT_FIELD(epqParam);
READ_ENUM_FIELD(onConflictAction, OnConflictAction);
*** a/src/backend/optimizer/plan/createplan.c
--- b/src/backend/optimizer/plan/createplan.c
***************
*** 3770,3775 **** make_foreignscan(List *qptlist,
--- 3770,3776 ----
plan->lefttree = outer_plan;
plan->righttree = NULL;
node->scan.scanrelid = scanrelid;
+ node->operation = CMD_SELECT;
/* fs_server will be filled in by create_foreignscan_plan */
node->fs_server = InvalidOid;
node->fdw_exprs = fdw_exprs;
***************
*** 5046,5051 **** make_modifytable(PlannerInfo *root,
--- 5047,5053 ----
Plan *plan = &node->plan;
double total_size;
List *fdw_private_list;
+ List *fdwpushdown_list;
ListCell *subnode;
ListCell *lc;
int i;
***************
*** 5126,5137 **** make_modifytable(PlannerInfo *root,
--- 5128,5141 ----
* construct private plan data, and accumulate it all into a list.
*/
fdw_private_list = NIL;
+ fdwpushdown_list = NIL;
i = 0;
foreach(lc, resultRelations)
{
Index rti = lfirst_int(lc);
FdwRoutine *fdwroutine;
List *fdw_private;
+ bool fdwpushdown;
/*
* If possible, we want to get the FdwRoutine from our RelOptInfo for
***************
*** 5158,5164 **** make_modifytable(PlannerInfo *root,
--- 5162,5181 ----
fdwroutine = NULL;
}
+ /*
+ * If the target relation has any row-level triggers, we can't push
+ * down the command to the remote server.
+ */
if (fdwroutine != NULL &&
+ fdwroutine->PlanDMLPushdown != NULL &&
+ !has_row_triggers(root, rti, operation))
+ fdwpushdown = fdwroutine->PlanDMLPushdown(root, node, rti, i);
+ else
+ fdwpushdown = false;
+ fdwpushdown_list = lappend_int(fdwpushdown_list, fdwpushdown);
+
+ if (!fdwpushdown &&
+ fdwroutine != NULL &&
fdwroutine->PlanForeignModify != NULL)
fdw_private = fdwroutine->PlanForeignModify(root, node, rti, i);
else
***************
*** 5167,5172 **** make_modifytable(PlannerInfo *root,
--- 5184,5190 ----
i++;
}
node->fdwPrivLists = fdw_private_list;
+ node->fdwPushdowns = fdwpushdown_list;
return node;
}
*** a/src/backend/optimizer/util/plancat.c
--- b/src/backend/optimizer/util/plancat.c
***************
*** 1520,1522 **** has_unique_index(RelOptInfo *rel, AttrNumber attno)
--- 1520,1569 ----
}
return false;
}
+
+
+ /*
+ * has_row_triggers
+ *
+ * Detect whether the specified relation has any row-level triggers for event.
+ */
+ bool
+ has_row_triggers(PlannerInfo *root, Index rti, CmdType event)
+ {
+ RangeTblEntry *rte = planner_rt_fetch(rti, root);
+ Relation relation;
+ TriggerDesc *trigDesc;
+ bool result = false;
+
+ /* Assume we already have adequate lock */
+ relation = heap_open(rte->relid, NoLock);
+
+ trigDesc = relation->trigdesc;
+ switch (event)
+ {
+ case CMD_INSERT:
+ if (trigDesc &&
+ (trigDesc->trig_insert_after_row ||
+ trigDesc->trig_insert_before_row))
+ result = true;
+ break;
+ case CMD_UPDATE:
+ if (trigDesc &&
+ (trigDesc->trig_update_after_row ||
+ trigDesc->trig_update_before_row))
+ result = true;
+ break;
+ case CMD_DELETE:
+ if (trigDesc &&
+ (trigDesc->trig_delete_after_row ||
+ trigDesc->trig_delete_before_row))
+ result = true;
+ break;
+ default:
+ elog(ERROR, "unrecognized CmdType: %d", (int) event);
+ break;
+ }
+
+ heap_close(relation, NoLock);
+ return result;
+ }
*** a/src/include/executor/executor.h
--- b/src/include/executor/executor.h
***************
*** 184,190 **** extern void ExecutorEnd(QueryDesc *queryDesc);
extern void standard_ExecutorEnd(QueryDesc *queryDesc);
extern void ExecutorRewind(QueryDesc *queryDesc);
extern bool ExecCheckRTPerms(List *rangeTable, bool ereport_on_violation);
! extern void CheckValidResultRel(Relation resultRel, CmdType operation);
extern void InitResultRelInfo(ResultRelInfo *resultRelInfo,
Relation resultRelationDesc,
Index resultRelationIndex,
--- 184,190 ----
extern void standard_ExecutorEnd(QueryDesc *queryDesc);
extern void ExecutorRewind(QueryDesc *queryDesc);
extern bool ExecCheckRTPerms(List *rangeTable, bool ereport_on_violation);
! extern void CheckValidResultRel(ResultRelInfo *resultRelInfo, CmdType operation);
extern void InitResultRelInfo(ResultRelInfo *resultRelInfo,
Relation resultRelationDesc,
Index resultRelationIndex,
*** a/src/include/foreign/fdwapi.h
--- b/src/include/foreign/fdwapi.h
***************
*** 93,98 **** typedef void (*EndForeignModify_function) (EState *estate,
--- 93,110 ----
typedef int (*IsForeignRelUpdatable_function) (Relation rel);
+ typedef bool (*PlanDMLPushdown_function) (PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+
+ typedef void (*BeginDMLPushdown_function) (ForeignScanState *node,
+ int eflags);
+
+ typedef TupleTableSlot *(*IterateDMLPushdown_function) (ForeignScanState *node);
+
+ typedef void (*EndDMLPushdown_function) (ForeignScanState *node);
+
typedef RowMarkType (*GetForeignRowMarkType_function) (RangeTblEntry *rte,
LockClauseStrength strength);
***************
*** 110,115 **** typedef void (*ExplainForeignModify_function) (ModifyTableState *mtstate,
--- 122,130 ----
int subplan_index,
struct ExplainState *es);
+ typedef void (*ExplainDMLPushdown_function) (ForeignScanState *node,
+ struct ExplainState *es);
+
typedef int (*AcquireSampleRowsFunc) (Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
***************
*** 162,167 **** typedef struct FdwRoutine
--- 177,186 ----
ExecForeignDelete_function ExecForeignDelete;
EndForeignModify_function EndForeignModify;
IsForeignRelUpdatable_function IsForeignRelUpdatable;
+ PlanDMLPushdown_function PlanDMLPushdown;
+ BeginDMLPushdown_function BeginDMLPushdown;
+ IterateDMLPushdown_function IterateDMLPushdown;
+ EndDMLPushdown_function EndDMLPushdown;
/* Functions for SELECT FOR UPDATE/SHARE row locking */
GetForeignRowMarkType_function GetForeignRowMarkType;
***************
*** 171,176 **** typedef struct FdwRoutine
--- 190,196 ----
/* Support functions for EXPLAIN */
ExplainForeignScan_function ExplainForeignScan;
ExplainForeignModify_function ExplainForeignModify;
+ ExplainDMLPushdown_function ExplainDMLPushdown;
/* Support functions for ANALYZE */
AnalyzeForeignTable_function AnalyzeForeignTable;
*** a/src/include/nodes/execnodes.h
--- b/src/include/nodes/execnodes.h
***************
*** 311,316 **** typedef struct JunkFilter
--- 311,317 ----
* TrigInstrument optional runtime measurements for triggers
* FdwRoutine FDW callback functions, if foreign table
* FdwState available to save private state of FDW
+ * FdwPushdown true when the command is pushed down
* WithCheckOptions list of WithCheckOption's to be checked
* WithCheckOptionExprs list of WithCheckOption expr states
* ConstraintExprs array of constraint-checking expr states
***************
*** 334,339 **** typedef struct ResultRelInfo
--- 335,341 ----
Instrumentation *ri_TrigInstrument;
struct FdwRoutine *ri_FdwRoutine;
void *ri_FdwState;
+ bool ri_FdwPushdown;
List *ri_WithCheckOptions;
List *ri_WithCheckOptionExprs;
List **ri_ConstraintExprs;
*** a/src/include/nodes/plannodes.h
--- b/src/include/nodes/plannodes.h
***************
*** 188,193 **** typedef struct ModifyTable
--- 188,194 ----
List *withCheckOptionLists; /* per-target-table WCO lists */
List *returningLists; /* per-target-table RETURNING tlists */
List *fdwPrivLists; /* per-target-table FDW private data lists */
+ List *fdwPushdowns; /* per-target-table FDW pushdown flags */
List *rowMarks; /* PlanRowMarks (non-locking only) */
int epqParam; /* ID of Param for EvalPlanQual re-eval */
OnConflictAction onConflictAction; /* ON CONFLICT action */
***************
*** 530,535 **** typedef struct WorkTableScan
--- 531,537 ----
typedef struct ForeignScan
{
Scan scan;
+ CmdType operation; /* SELECT/INSERT/UPDATE/DELETE */
Oid fs_server; /* OID of foreign server */
List *fdw_exprs; /* expressions that FDW may evaluate */
List *fdw_private; /* private data for FDW */
*** a/src/include/optimizer/plancat.h
--- b/src/include/optimizer/plancat.h
***************
*** 55,58 **** extern Selectivity join_selectivity(PlannerInfo *root,
--- 55,60 ----
JoinType jointype,
SpecialJoinInfo *sjinfo);
+ extern bool has_row_triggers(PlannerInfo *root, Index rti, CmdType event);
+
#endif /* PLANCAT_H */
On 2016/01/19 19:04, Thom Brown wrote:
On 12 January 2016 at 11:49, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp> wrote:
On 2016/01/12 20:36, Thom Brown wrote:
On 8 January 2016 at 05:08, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>
wrote:
On 2016/01/06 20:37, Thom Brown wrote:
I've run into an issue:
*# UPDATE master_customers SET id = 22 WHERE id = 16 RETURNING
tableoid::regclass;
ERROR:
CONTEXT: Remote SQL command: UPDATE public.customers SET id = 22
WHERE ((id = 16)) RETURNING NULL
While working on this, I noticed that the existing postgres_fdw system
shows
similar behavior, so I changed the subject.IIUC, the reason for that is when the local query specifies "RETURNING
tableoid::regclass", the FDW has fmstate->has_returning=false while the
remote query executed at ModifyTable has "RETURNING NULL", as shown in
the
above example; that would cause an abnormal exit in executing the remote
query in postgresExecForeignUpdate, since that the FDW would get
PGRES_TUPLES_OK as a result of the query while the FDW would think that
the
right result to get should be PGRES_COMMAND_OK, from the flag
fmstate->has_returning=false.
Attached is a patch to fix that.
I can't apply this patch in tandem with FDW DML pushdown patch (either
v2 or v3).
That patch is for fixing the similar issue in the existing postgres_fdw
system. So, please apply that patch without the DML pushdown patch. If
that patch is reasonable as a fix for the issue, I'll update the DML
pushdown patch (v3) on top of that patch.
The patch seems to work for me:
Before:
*# UPDATE master_customers SET id = 22 WHERE id = 1 RETURNING
tableoid::regclass;
ERROR:
CONTEXT: Remote SQL command: UPDATE public.customers SET id = $2
WHERE ctid = $1 RETURNING NULLAfter:
*# UPDATE master_customers SET id = 22 WHERE id = 1 RETURNING
tableoid::regclass;
tableoid
------------------
remote.customers
(1 row)UPDATE 1
Thanks for the testing!
I updated the DML pushdown patch on top of Robert's version of this
bugfix patch. Please see
/messages/by-id/56A0A9F0.9090304@lab.ntt.co.jp
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Jan 21, 2016 at 3:20 PM, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>
wrote:
On 2016/01/20 19:57, Rushabh Lathia wrote:
Overall I am quite done with the review of this patch. Patch is in good
shape and covered most of the things which been discussed earlier
or been mentioned during review process. Patch pass through the
make check and also includes good test coverage.Thanks for the review!
Here are couple of things which is still open for discussion:
1)
.) When Tom Lane and Stephen Frost suggested getting the core
code involved,
I thought that we can do the mandatory checks into core it self
and making
completely out of dml_is_pushdown_safe(). Please correct meThe reason why I put that function in postgres_fdw.c is Check point 4:
+ * 4. We can't push an UPDATE down, if any expressions to assign to the target + * columns are unsafe to evaluate on the remote server.Here I was talking about checks related to triggers, or to LIMIT. I think
earlier thread talked about those mandatory check to the core. So may
be we can move those checks into make_modifytable() before calling
the PlanDMLPushdown.This need to handle by the Owner.
Done. For that, I modified relation_has_row_triggers a bit, renamed it to
has_row_triggers (more shortly), and moved it to plancat.c. And I merged
dml_is_pushdown_safe with postgresPlanDMLPushdown, and revised that
callback routine a bit. Attached is an updated version of the patch
created on top of Robert's version of the patch [1], which fixes handling
of RETURNING tableoid in updating foreign tables.
This looks great.
2) Decision on whether we need the separate new node ForeignUpdate,
ForeignDelete. In my opinion I really don't see the need of this as we
that will add lot of duplicate. Having said that if committer or someone
else feel like that will make code more clean that is also true,This need more comments from the committer.
I agree with you.
Other changes:
* In previous version, I assumed that PlanDMLPushdown sets fsSystemCol to
true when rewriting the ForeignScan plan node so as to push down an
UPDATE/DELETE to the remote server, in order to initialize t_tableOid for
the scan tuple in ForeignNext. The reason is that I created the patch so
that the scan tuple is provided to the local query's RETURNING computation,
which might see the tableoid column. In this version, however, I modified
the patch so that the tableoid value is inserted by ModifyTable. This
eliminates the need for postgres_fdw (or any other FDW) to set fsSystemCol
to true in PlanDMLPushdown.* Add set_transmission_modes/reset_transmission_modes to
deparsePushedDownUpdateSql.* Revise comments a bit further.
* Revise docs, including a fix for a wrong copy-and-paste.
Best regards,
Etsuro Fujita[1]
/messages/by-id/CA+TgmoZ40j2uC5aC1NXu03oj4CrVOLkS15XX+PTFP-1U-8zR1Q@mail.gmail.com
Here are couple of comments:
1)
int
IsForeignRelUpdatable (Relation rel);
Documentation for IsForeignUpdatable() need to change as it says:
If the IsForeignRelUpdatable pointer is set to NULL, foreign tables are
assumed
to be insertable, updatable, or deletable if the FDW provides
ExecForeignInsert,
ExecForeignUpdate or ExecForeignDelete respectively.
With introduce of DMLPushdown API now this is no more correct, as even if
FDW don't provide ExecForeignInsert, ExecForeignUpdate or ExecForeignDelete
API
still foreign tables are assumed to be updatable or deletable with
DMLPushdown
API's, right ?
2)
+ /* SQL statement to execute remotely (as a String node) */
+ FdwDmlPushdownPrivateUpdateSql,
FdwDmlPushdownPrivateUpdateSql holds the UPDATE/DELETE query, so name should
be something like FdwDmlPushdownPrivateQuery os FdwDmlPushdownPrivateSql ?
Later I realized that for FdwModifyPrivateIndex too the index name is
FdwModifyPrivateUpdateSql even though its holding any DML query. Not sure
whether we should consider to change this or not ?
Apart from this perform sanity testing on the new patch and things working
as expected.
--
Rushabh Lathia
www.EnterpriseDB.com
On 2016/01/25 17:03, Rushabh Lathia wrote:
Here are couple of comments:
1)
int
IsForeignRelUpdatable (Relation rel);
Documentation for IsForeignUpdatable() need to change as it says:
If the IsForeignRelUpdatable pointer is set to NULL, foreign tables are
assumed
to be insertable, updatable, or deletable if the FDW provides
ExecForeignInsert,
ExecForeignUpdate or ExecForeignDelete respectively.With introduce of DMLPushdown API now this is no more correct, as even if
FDW don't provide ExecForeignInsert, ExecForeignUpdate or
ExecForeignDelete API
still foreign tables are assumed to be updatable or deletable with
DMLPushdown
API's, right ?
That's what I'd like to discuss.
I intentionally leave that as-is, because I think we should determine
the updatability of a foreign table in the current manner. As you
pointed out, even if the FDW doesn't provide eg, ExecForeignUpdate, an
UPDATE on a foreign table could be done using the DML pushdown APIs if
the UPDATE is *pushdown-safe*. However, since all UPDATEs on the
foreign table are not necessarily pushdown-safe, I'm not sure it's a
good idea to assume the table-level updatability if the FDW provides the
DML pushdown callback routines. To keep the existing updatability
decision, I think the FDW should provide the DML pushdown callback
routines together with ExecForeignInsert, ExecForeignUpdate, or
ExecForeignDelete. What do you think about that?
2)
+ /* SQL statement to execute remotely (as a String node) */ + FdwDmlPushdownPrivateUpdateSql,FdwDmlPushdownPrivateUpdateSql holds the UPDATE/DELETE query, so name should
be something like FdwDmlPushdownPrivateQuery os FdwDmlPushdownPrivateSql ?
Later I realized that for FdwModifyPrivateIndex too the index name is
FdwModifyPrivateUpdateSql even though its holding any DML query. Not sure
whether we should consider to change this or not ?
To tell the truth, I imitated FdwModifyPrivateIndex when adding
FdwDmlPushdownPrivateIndex, because I think "UpdateSql" means INSERT,
UPDATE, or DELETE, not just UPDATE. (IsForeignRelUpdatable discussed
above reports not only the updatability but the insertability and
deletability of a foreign table!). So, +1 for leaving that as-is.
Apart from this perform sanity testing on the new patch and things working
as expected.
Thanks for the review!
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Jan 26, 2016 at 4:15 PM, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>
wrote:
On 2016/01/25 17:03, Rushabh Lathia wrote:
Here are couple of comments:
1)
int
IsForeignRelUpdatable (Relation rel);Documentation for IsForeignUpdatable() need to change as it says:
If the IsForeignRelUpdatable pointer is set to NULL, foreign tables are
assumed
to be insertable, updatable, or deletable if the FDW provides
ExecForeignInsert,
ExecForeignUpdate or ExecForeignDelete respectively.With introduce of DMLPushdown API now this is no more correct, as even if
FDW don't provide ExecForeignInsert, ExecForeignUpdate or
ExecForeignDelete API
still foreign tables are assumed to be updatable or deletable with
DMLPushdown
API's, right ?That's what I'd like to discuss.
I intentionally leave that as-is, because I think we should determine the
updatability of a foreign table in the current manner. As you pointed out,
even if the FDW doesn't provide eg, ExecForeignUpdate, an UPDATE on a
foreign table could be done using the DML pushdown APIs if the UPDATE is
*pushdown-safe*. However, since all UPDATEs on the foreign table are not
necessarily pushdown-safe, I'm not sure it's a good idea to assume the
table-level updatability if the FDW provides the DML pushdown callback
routines. To keep the existing updatability decision, I think the FDW
should provide the DML pushdown callback routines together with
ExecForeignInsert, ExecForeignUpdate, or ExecForeignDelete. What do you
think about that?
Sorry but I am not in favour of adding compulsion that FDW should provide
the DML pushdown callback routines together with existing ExecForeignInsert,
ExecForeignUpdate or ExecForeignDelete APIs.
May be we should change the documentation in such way, that explains
a) If FDW PlanDMLPushdown is NULL, then check for ExecForeignInsert,
ExecForeignUpdate or ExecForeignDelete APIs
b) If FDW PlanDMLPushdown is non-NULL and plan is not pushable
check for ExecForeignInsert, ExecForeignUpdate or ExecForeignDelete APIs
c) If FDW PlanDMLPushdown is non-NULL and plan is pushable
check for DMLPushdown APIs.
Does this sounds wired ?
2)
+ /* SQL statement to execute remotely (as a String node) */ + FdwDmlPushdownPrivateUpdateSql,FdwDmlPushdownPrivateUpdateSql holds the UPDATE/DELETE query, so name
should
be something like FdwDmlPushdownPrivateQuery os FdwDmlPushdownPrivateSql ?Later I realized that for FdwModifyPrivateIndex too the index name is
FdwModifyPrivateUpdateSql even though its holding any DML query. Not sure
whether we should consider to change this or not ?To tell the truth, I imitated FdwModifyPrivateIndex when adding
FdwDmlPushdownPrivateIndex, because I think "UpdateSql" means INSERT,
UPDATE, or DELETE, not just UPDATE. (IsForeignRelUpdatable discussed above
reports not only the updatability but the insertability and deletability of
a foreign table!). So, +1 for leaving that as-is.
Make sense for now.
Apart from this perform sanity testing on the new patch and things working
as expected.
Thanks for the review!
Best regards,
Etsuro Fujita
--
Rushabh Lathia
On 2016/01/26 22:57, Rushabh Lathia wrote:
On Tue, Jan 26, 2016 at 4:15 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp <mailto:fujita.etsuro@lab.ntt.co.jp>> wrote:On 2016/01/25 17:03, Rushabh Lathia wrote:
int
IsForeignRelUpdatable (Relation rel);
Documentation for IsForeignUpdatable() need to change as it says:
If the IsForeignRelUpdatable pointer is set to NULL, foreign
tables are
assumed
to be insertable, updatable, or deletable if the FDW provides
ExecForeignInsert,
ExecForeignUpdate or ExecForeignDelete respectively.With introduce of DMLPushdown API now this is no more correct,
as even if
FDW don't provide ExecForeignInsert, ExecForeignUpdate or
ExecForeignDelete API
still foreign tables are assumed to be updatable or deletable with
DMLPushdown
API's, right ?
That's what I'd like to discuss.
I intentionally leave that as-is, because I think we should
determine the updatability of a foreign table in the current
manner. As you pointed out, even if the FDW doesn't provide eg,
ExecForeignUpdate, an UPDATE on a foreign table could be done using
the DML pushdown APIs if the UPDATE is *pushdown-safe*. However,
since all UPDATEs on the foreign table are not necessarily
pushdown-safe, I'm not sure it's a good idea to assume the
table-level updatability if the FDW provides the DML pushdown
callback routines. To keep the existing updatability decision, I
think the FDW should provide the DML pushdown callback routines
together with ExecForeignInsert, ExecForeignUpdate, or
ExecForeignDelete. What do you think about that?
Sorry but I am not in favour of adding compulsion that FDW should provide
the DML pushdown callback routines together with existing ExecForeignInsert,
ExecForeignUpdate or ExecForeignDelete APIs.May be we should change the documentation in such way, that explains
a) If FDW PlanDMLPushdown is NULL, then check for ExecForeignInsert,
ExecForeignUpdate or ExecForeignDelete APIs
b) If FDW PlanDMLPushdown is non-NULL and plan is not pushable
check for ExecForeignInsert, ExecForeignUpdate or ExecForeignDelete APIs
c) If FDW PlanDMLPushdown is non-NULL and plan is pushable
check for DMLPushdown APIs.Does this sounds wired ?
Yeah, but I think that that would be what is done during executor
startup (see CheckValidResultRel()), while what the documentation is
saying is about relation_is_updatable(); that is, how to decide the
updatability of a given foreign table, not how the executor processes an
individual INSERT/UPDATE/DELETE on a updatable foreign table. So, I'm
not sure it's a good idea to modify the documentation in such a way.
BTW, I have added the description about that check partially. I added
to the PlanDMLPushdown documentation:
+ If this function succeeds, <function>PlanForeignModify</>
+ won't be executed, and <function>BeginDMLPushdown</>,
+ <function>IterateDMLPushdown</> and <function>EndDMLPushdown</> will
+ be called at the execution stage, instead.
And for example, I added to the BeginDMLPushdown documentation:
+ If the <function>BeginDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to execute the table update directly on
+ the remote server will fail with an error message.
However, I agree that we should add a documentation note about the
compulsion somewhere. Maybe something like this:
The FDW should provide DML pushdown callback routines together with
table-updating callback routines described above. Even if the callback
routines are provided, the updatability of a foreign table is determined
based on the presence of ExecForeignInsert, ExecForeignUpdate or
ExecForeignDelete if the IsForeignRelUpdatable pointer is set to NULL.
What's your opinion?
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/01/27 12:20, Etsuro Fujita wrote:
On 2016/01/26 22:57, Rushabh Lathia wrote:
On Tue, Jan 26, 2016 at 4:15 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp <mailto:fujita.etsuro@lab.ntt.co.jp>> wrote:On 2016/01/25 17:03, Rushabh Lathia wrote:
int
IsForeignRelUpdatable (Relation rel);
Documentation for IsForeignUpdatable() need to change as it says:
If the IsForeignRelUpdatable pointer is set to NULL, foreign
tables are
assumed
to be insertable, updatable, or deletable if the FDW provides
ExecForeignInsert,
ExecForeignUpdate or ExecForeignDelete respectively.With introduce of DMLPushdown API now this is no more correct,
as even if
FDW don't provide ExecForeignInsert, ExecForeignUpdate or
ExecForeignDelete API
still foreign tables are assumed to be updatable or deletable
with
DMLPushdown
API's, right ?
That's what I'd like to discuss.
I intentionally leave that as-is, because I think we should
determine the updatability of a foreign table in the current
manner. As you pointed out, even if the FDW doesn't provide eg,
ExecForeignUpdate, an UPDATE on a foreign table could be done using
the DML pushdown APIs if the UPDATE is *pushdown-safe*. However,
since all UPDATEs on the foreign table are not necessarily
pushdown-safe, I'm not sure it's a good idea to assume the
table-level updatability if the FDW provides the DML pushdown
callback routines. To keep the existing updatability decision, I
think the FDW should provide the DML pushdown callback routines
together with ExecForeignInsert, ExecForeignUpdate, or
ExecForeignDelete. What do you think about that?
Sorry but I am not in favour of adding compulsion that FDW should provide
the DML pushdown callback routines together with existing
ExecForeignInsert,
ExecForeignUpdate or ExecForeignDelete APIs.May be we should change the documentation in such way, that explains
a) If FDW PlanDMLPushdown is NULL, then check for ExecForeignInsert,
ExecForeignUpdate or ExecForeignDelete APIs
b) If FDW PlanDMLPushdown is non-NULL and plan is not pushable
check for ExecForeignInsert, ExecForeignUpdate or ExecForeignDelete APIs
c) If FDW PlanDMLPushdown is non-NULL and plan is pushable
check for DMLPushdown APIs.Does this sounds wired ?
Yeah, but I think that that would be what is done during executor
startup (see CheckValidResultRel()), while what the documentation is
saying is about relation_is_updatable(); that is, how to decide the
updatability of a given foreign table, not how the executor processes an
individual INSERT/UPDATE/DELETE on a updatable foreign table. So, I'm
not sure it's a good idea to modify the documentation in such a way.
However, I agree that we should add a documentation note about the
compulsion somewhere. Maybe something like this:The FDW should provide DML pushdown callback routines together with
table-updating callback routines described above. Even if the callback
routines are provided, the updatability of a foreign table is determined
based on the presence of ExecForeignInsert, ExecForeignUpdate or
ExecForeignDelete if the IsForeignRelUpdatable pointer is set to NULL.
On second thought, I think it might be okay to assume the presence of
PlanDMLPushdown, BeginDMLPushdown, IterateDMLPushdown, and
EndDMLPushdown is also sufficient for the insertablity, updatability,
and deletability of a foreign table, if the IsForeignRelUpdatable
pointer is set to NULL. How about modifying the documentation like this:
If the IsForeignRelUpdatable pointer is set to NULL, foreign tables are
assumed to be insertable, updatable, or deletable if the FDW provides
ExecForeignInsert, ExecForeignUpdate, or ExecForeignDelete respectively,
or if the FDW provides PlanDMLPushdown, BeginDMLPushdown,
IterateDMLPushdown, and EndDMLPushdown described below.
Of course, we also need to modify relation_is_updatable() accordingly.
What's your opinion?
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Jan 27, 2016 at 2:50 PM, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>
wrote:
On 2016/01/27 12:20, Etsuro Fujita wrote:
On 2016/01/26 22:57, Rushabh Lathia wrote:
On Tue, Jan 26, 2016 at 4:15 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp <mailto:fujita.etsuro@lab.ntt.co.jp>>
wrote:On 2016/01/25 17:03, Rushabh Lathia wrote:
int
IsForeignRelUpdatable (Relation rel);
Documentation for IsForeignUpdatable() need to change as it says:
If the IsForeignRelUpdatable pointer is set to NULL, foreign
tables are
assumed
to be insertable, updatable, or deletable if the FDW provides
ExecForeignInsert,
ExecForeignUpdate or ExecForeignDelete respectively.With introduce of DMLPushdown API now this is no more correct,
as even if
FDW don't provide ExecForeignInsert, ExecForeignUpdate or
ExecForeignDelete API
still foreign tables are assumed to be updatable or deletable
with
DMLPushdown
API's, right ?That's what I'd like to discuss.
I intentionally leave that as-is, because I think we should
determine the updatability of a foreign table in the current
manner. As you pointed out, even if the FDW doesn't provide eg,
ExecForeignUpdate, an UPDATE on a foreign table could be done using
the DML pushdown APIs if the UPDATE is *pushdown-safe*. However,
since all UPDATEs on the foreign table are not necessarily
pushdown-safe, I'm not sure it's a good idea to assume the
table-level updatability if the FDW provides the DML pushdown
callback routines. To keep the existing updatability decision, I
think the FDW should provide the DML pushdown callback routines
together with ExecForeignInsert, ExecForeignUpdate, or
ExecForeignDelete. What do you think about that?Sorry but I am not in favour of adding compulsion that FDW should provide
the DML pushdown callback routines together with existing
ExecForeignInsert,
ExecForeignUpdate or ExecForeignDelete APIs.May be we should change the documentation in such way, that explains
a) If FDW PlanDMLPushdown is NULL, then check for ExecForeignInsert,
ExecForeignUpdate or ExecForeignDelete APIs
b) If FDW PlanDMLPushdown is non-NULL and plan is not pushable
check for ExecForeignInsert, ExecForeignUpdate or ExecForeignDelete APIs
c) If FDW PlanDMLPushdown is non-NULL and plan is pushable
check for DMLPushdown APIs.Does this sounds wired ?
Yeah, but I think that that would be what is done during executor
startup (see CheckValidResultRel()), while what the documentation is
saying is about relation_is_updatable(); that is, how to decide the
updatability of a given foreign table, not how the executor processes an
individual INSERT/UPDATE/DELETE on a updatable foreign table. So, I'm
not sure it's a good idea to modify the documentation in such a way.However, I agree that we should add a documentation note about the
compulsion somewhere. Maybe something like this:
The FDW should provide DML pushdown callback routines together with
table-updating callback routines described above. Even if the callback
routines are provided, the updatability of a foreign table is determined
based on the presence of ExecForeignInsert, ExecForeignUpdate or
ExecForeignDelete if the IsForeignRelUpdatable pointer is set to NULL.On second thought, I think it might be okay to assume the presence of
PlanDMLPushdown, BeginDMLPushdown, IterateDMLPushdown, and EndDMLPushdown
is also sufficient for the insertablity, updatability, and deletability of
a foreign table, if the IsForeignRelUpdatable pointer is set to NULL. How
about modifying the documentation like this:If the IsForeignRelUpdatable pointer is set to NULL, foreign tables are
assumed to be insertable, updatable, or deletable if the FDW provides
ExecForeignInsert, ExecForeignUpdate, or ExecForeignDelete respectively, or
if the FDW provides PlanDMLPushdown, BeginDMLPushdown, IterateDMLPushdown,
and EndDMLPushdown described below.Of course, we also need to modify relation_is_updatable() accordingly.
What's your opinion?
If I understood correctly, above documentation means, that if FDW have
DMLPushdown APIs that is enough. But in reality thats not the case, we need
ExecForeignInsert, ExecForeignUpdate, or ExecForeignDelete in case DML is
not pushable.
And here fact is DMLPushdown APIs are optional for FDW, so that if FDW
don't have DMLPushdown APIs they can still very well perform the DML
operations using ExecForeignInsert, ExecForeignUpdate, or
ExecForeignDelete. So documentation should be like:
If the IsForeignRelUpdatable pointer is set to NULL, foreign tables are
assumed to be insertable, updatable, or deletable if the FDW provides
ExecForeignInsert, ExecForeignUpdate, or ExecForeignDelete respectively,
If FDW provides DMLPushdown APIs and the DML are pushable to the foreign
server, then FDW still needs ExecForeignInsert, ExecForeignUpdate, or
ExecForeignDelete for the non-pushable DML operation.
What's your opinion ?
Best regards,
Etsuro Fujita
--
Rushabh Lathia
On Thu, Jan 21, 2016 at 4:05 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
By the way, I'm not too sure I understand the need for the core
changes that are part of this patch, and I think that point merits
some discussion. Whenever you change core like this, you're changing
the contract between the FDW and core; it's not just postgres_fdw that
needs updating, but every FDW. So we'd better be pretty sure we need
these changes and they are adequately justified before we think about
putting them into the tree. Are these core changes really needed
here, or can we fix this whole issue in postgres_fdw and leave the
core code alone?Well, if we think it is the FDW's responsibility to insert a valid value for
tableoid in the returned slot during ExecForeignInsert, ExecForeignUpdate or
ExecForeignDelete, we don't need those core changes. However, I think it
would be better that that's done by ModifyTable in the same way as
ForeignScan does in ForeignNext, IMO. That eliminates the need for
postgres_fdw or any other FDW to do that business in the callback routines.
I'm not necessarily opposed to the core changes, but I want to
understand better what complexity they are avoiding. Can you send a
version of this patch that only touches postgres_fdw, so I can
compare?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/01/27 21:23, Rushabh Lathia wrote:
If I understood correctly, above documentation means, that if FDW have
DMLPushdown APIs that is enough. But in reality thats not the case, we
need ExecForeignInsert, ExecForeignUpdate, or ExecForeignDelete in case
DML is not pushable.And here fact is DMLPushdown APIs are optional for FDW, so that if FDW
don't have DMLPushdown APIs they can still very well perform the DML
operations using ExecForeignInsert, ExecForeignUpdate, or
ExecForeignDelete.
I agree with you. I guess I was wrong. sorry.
So documentation should be like:
If the IsForeignRelUpdatable pointer is set to NULL, foreign tables are
assumed to be insertable, updatable, or deletable if the FDW provides
ExecForeignInsert, ExecForeignUpdate, or ExecForeignDelete respectively,If FDW provides DMLPushdown APIs and the DML are pushable to the foreign
server, then FDW still needs ExecForeignInsert, ExecForeignUpdate, or
ExecForeignDelete for the non-pushable DML operation.What's your opinion ?
I agree that we should add this to the documentation, too.
BTW, if I understand correctly, I think we should also modify
relation_is_updatabale() accordingly. Am I right?
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Jan 28, 2016 at 11:33 AM, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp
wrote:
On 2016/01/27 21:23, Rushabh Lathia wrote:
If I understood correctly, above documentation means, that if FDW have
DMLPushdown APIs that is enough. But in reality thats not the case, we
need ExecForeignInsert, ExecForeignUpdate, or ExecForeignDelete in case
DML is not pushable.And here fact is DMLPushdown APIs are optional for FDW, so that if FDW
don't have DMLPushdown APIs they can still very well perform the DML
operations using ExecForeignInsert, ExecForeignUpdate, or
ExecForeignDelete.I agree with you. I guess I was wrong. sorry.
So documentation should be like:
If the IsForeignRelUpdatable pointer is set to NULL, foreign tables are
assumed to be insertable, updatable, or deletable if the FDW provides
ExecForeignInsert, ExecForeignUpdate, or ExecForeignDelete respectively,If FDW provides DMLPushdown APIs and the DML are pushable to the foreign
server, then FDW still needs ExecForeignInsert, ExecForeignUpdate, or
ExecForeignDelete for the non-pushable DML operation.What's your opinion ?
I agree that we should add this to the documentation, too.
BTW, if I understand correctly, I think we should also modify
relation_is_updatabale() accordingly. Am I right?
Yep, we need to modify relation_is_updatable().
Best regards,
Etsuro Fujita
--
Rushabh Lathia
On 2016/01/28 15:20, Rushabh Lathia wrote:
On Thu, Jan 28, 2016 at 11:33 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp <mailto:fujita.etsuro@lab.ntt.co.jp>> wrote:On 2016/01/27 21:23, Rushabh Lathia wrote:
If I understood correctly, above documentation means, that if
FDW have
DMLPushdown APIs that is enough. But in reality thats not the
case, we
need ExecForeignInsert, ExecForeignUpdate, or ExecForeignDelete
in case
DML is not pushable.And here fact is DMLPushdown APIs are optional for FDW, so that
if FDW
don't have DMLPushdown APIs they can still very well perform the DML
operations using ExecForeignInsert, ExecForeignUpdate, or
ExecForeignDelete.
I agree with you. I guess I was wrong. sorry.
So documentation should be like:
If the IsForeignRelUpdatable pointer is set to NULL, foreign
tables are
assumed to be insertable, updatable, or deletable if the FDW
provides
ExecForeignInsert, ExecForeignUpdate, or ExecForeignDelete
respectively,If FDW provides DMLPushdown APIs and the DML are pushable to the
foreign
server, then FDW still needs ExecForeignInsert,
ExecForeignUpdate, or
ExecForeignDelete for the non-pushable DML operation.What's your opinion ?
I agree that we should add this to the documentation, too.
BTW, if I understand correctly, I think we should also modify
relation_is_updatabale() accordingly. Am I right?
Yep, we need to modify relation_is_updatable().
OK, will do.
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/01/28 12:58, Robert Haas wrote:
On Thu, Jan 21, 2016 at 4:05 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:By the way, I'm not too sure I understand the need for the core
changes that are part of this patch, and I think that point merits
some discussion. Whenever you change core like this, you're changing
the contract between the FDW and core; it's not just postgres_fdw that
needs updating, but every FDW. So we'd better be pretty sure we need
these changes and they are adequately justified before we think about
putting them into the tree. Are these core changes really needed
here, or can we fix this whole issue in postgres_fdw and leave the
core code alone?
Well, if we think it is the FDW's responsibility to insert a valid value for
tableoid in the returned slot during ExecForeignInsert, ExecForeignUpdate or
ExecForeignDelete, we don't need those core changes. However, I think it
would be better that that's done by ModifyTable in the same way as
ForeignScan does in ForeignNext, IMO. That eliminates the need for
postgres_fdw or any other FDW to do that business in the callback routines.
I'm not necessarily opposed to the core changes, but I want to
understand better what complexity they are avoiding. Can you send a
version of this patch that only touches postgres_fdw, so I can
compare?
Attached is that version of the patch.
I think that postgres_fdw might be able to insert a tableoid value in
the returned slot in e.g., postgresExecForeignInsert if AFTER ROW
Triggers or RETURNING expressions reference that value, but I didn't do
anything about that.
Best regards,
Etsuro Fujita
Attachments:
fdw-foreign-modify-rmh-efujita.patchapplication/x-patch; name=fdw-foreign-modify-rmh-efujita.patchDownload
*** a/contrib/postgres_fdw/deparse.c
--- b/contrib/postgres_fdw/deparse.c
***************
*** 110,115 **** static void deparseTargetList(StringInfo buf,
--- 110,116 ----
PlannerInfo *root,
Index rtindex,
Relation rel,
+ bool is_returning,
Bitmapset *attrs_used,
List **retrieved_attrs);
static void deparseReturningList(StringInfo buf, PlannerInfo *root,
***************
*** 724,730 **** deparseSelectSql(StringInfo buf,
* Construct SELECT list
*/
appendStringInfoString(buf, "SELECT ");
! deparseTargetList(buf, root, baserel->relid, rel, attrs_used,
retrieved_attrs);
/*
--- 725,731 ----
* Construct SELECT list
*/
appendStringInfoString(buf, "SELECT ");
! deparseTargetList(buf, root, baserel->relid, rel, false, attrs_used,
retrieved_attrs);
/*
***************
*** 738,744 **** deparseSelectSql(StringInfo buf,
/*
* Emit a target list that retrieves the columns specified in attrs_used.
! * This is used for both SELECT and RETURNING targetlists.
*
* The tlist text is appended to buf, and we also create an integer List
* of the columns being retrieved, which is returned to *retrieved_attrs.
--- 739,746 ----
/*
* Emit a target list that retrieves the columns specified in attrs_used.
! * This is used for both SELECT and RETURNING targetlists; the is_returning
! * parameter is true only for a RETURNING targetlist.
*
* The tlist text is appended to buf, and we also create an integer List
* of the columns being retrieved, which is returned to *retrieved_attrs.
***************
*** 748,753 **** deparseTargetList(StringInfo buf,
--- 750,756 ----
PlannerInfo *root,
Index rtindex,
Relation rel,
+ bool is_returning,
Bitmapset *attrs_used,
List **retrieved_attrs)
{
***************
*** 777,782 **** deparseTargetList(StringInfo buf,
--- 780,787 ----
{
if (!first)
appendStringInfoString(buf, ", ");
+ else if (is_returning)
+ appendStringInfoString(buf, " RETURNING ");
first = false;
deparseColumnRef(buf, rtindex, i, root);
***************
*** 794,799 **** deparseTargetList(StringInfo buf,
--- 799,806 ----
{
if (!first)
appendStringInfoString(buf, ", ");
+ else if (is_returning)
+ appendStringInfoString(buf, " RETURNING ");
first = false;
appendStringInfoString(buf, "ctid");
***************
*** 803,809 **** deparseTargetList(StringInfo buf,
}
/* Don't generate bad syntax if no undropped columns */
! if (first)
appendStringInfoString(buf, "NULL");
}
--- 810,816 ----
}
/* Don't generate bad syntax if no undropped columns */
! if (first && !is_returning)
appendStringInfoString(buf, "NULL");
}
***************
*** 1022,1032 **** deparseReturningList(StringInfo buf, PlannerInfo *root,
}
if (attrs_used != NULL)
! {
! appendStringInfoString(buf, " RETURNING ");
! deparseTargetList(buf, root, rtindex, rel, attrs_used,
retrieved_attrs);
- }
else
*retrieved_attrs = NIL;
}
--- 1029,1036 ----
}
if (attrs_used != NULL)
! deparseTargetList(buf, root, rtindex, rel, true, attrs_used,
retrieved_attrs);
else
*retrieved_attrs = NIL;
}
*** a/contrib/postgres_fdw/expected/postgres_fdw.out
--- b/contrib/postgres_fdw/expected/postgres_fdw.out
***************
*** 2408,2413 **** SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
--- 2408,2466 ----
1104 | 204 | ddd |
(819 rows)
+ EXPLAIN (verbose, costs off)
+ INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
+ QUERY PLAN
+ -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Insert on public.ft2
+ Output: (tableoid)::regclass
+ Remote SQL: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6, c7, c8) VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
+ -> Result
+ Output: 9999, 999, NULL::integer, 'foo'::text, NULL::timestamp with time zone, NULL::timestamp without time zone, NULL::character varying, 'ft2 '::character(10), NULL::user_enum
+ (5 rows)
+
+ INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
+ tableoid
+ ----------
+ ft2
+ (1 row)
+
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
+ QUERY PLAN
+ -------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Output: (tableoid)::regclass
+ Remote SQL: UPDATE "S 1"."T 1" SET c3 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.ft2
+ Output: c1, c2, NULL::integer, 'bar'::text, c4, c5, c6, c7, c8, ctid
+ Remote SQL: SELECT "C 1", c2, c4, c5, c6, c7, c8, ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
+ (6 rows)
+
+ UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
+ tableoid
+ ----------
+ ft2
+ (1 row)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
+ QUERY PLAN
+ ------------------------------------------------------------------------------------
+ Delete on public.ft2
+ Output: (tableoid)::regclass
+ Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1
+ -> Foreign Scan on public.ft2
+ Output: ctid
+ Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
+ (6 rows)
+
+ DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
+ tableoid
+ ----------
+ ft2
+ (1 row)
+
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
BEGIN
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
***************
*** 1614,1619 **** postgresExecForeignInsert(EState *estate,
--- 1614,1620 ----
const char **p_values;
PGresult *res;
int n_rows;
+ HeapTuple tuple;
/* Set up the prepared statement on the remote server, if we didn't yet */
if (!fmstate->p_name)
***************
*** 1654,1659 **** postgresExecForeignInsert(EState *estate,
--- 1655,1667 ----
MemoryContextReset(fmstate->temp_cxt);
+ /*
+ * AFTER ROW Triggers or RETURNING expressions might reference the tableoid
+ * column, so initialize t_tableOid before evaluating them.
+ */
+ tuple = ExecMaterializeSlot(slot);
+ tuple->t_tableOid = RelationGetRelid(resultRelInfo->ri_RelationDesc);
+
/* Return NULL if nothing was inserted on the remote end */
return (n_rows > 0) ? slot : NULL;
}
***************
*** 1674,1679 **** postgresExecForeignUpdate(EState *estate,
--- 1682,1688 ----
const char **p_values;
PGresult *res;
int n_rows;
+ HeapTuple tuple;
/* Set up the prepared statement on the remote server, if we didn't yet */
if (!fmstate->p_name)
***************
*** 1724,1729 **** postgresExecForeignUpdate(EState *estate,
--- 1733,1745 ----
MemoryContextReset(fmstate->temp_cxt);
+ /*
+ * AFTER ROW Triggers or RETURNING expressions might reference the tableoid
+ * column, so initialize t_tableOid before evaluating them.
+ */
+ tuple = ExecMaterializeSlot(slot);
+ tuple->t_tableOid = RelationGetRelid(resultRelInfo->ri_RelationDesc);
+
/* Return NULL if nothing was updated on the remote end */
return (n_rows > 0) ? slot : NULL;
}
***************
*** 1744,1749 **** postgresExecForeignDelete(EState *estate,
--- 1760,1766 ----
const char **p_values;
PGresult *res;
int n_rows;
+ HeapTuple tuple;
/* Set up the prepared statement on the remote server, if we didn't yet */
if (!fmstate->p_name)
***************
*** 1794,1799 **** postgresExecForeignDelete(EState *estate,
--- 1811,1825 ----
MemoryContextReset(fmstate->temp_cxt);
+ /*
+ * RETURNING expressions might reference the tableoid column, so initialize
+ * t_tableOid before evaluating them.
+ */
+ if (slot->tts_isempty)
+ ExecStoreAllNullTuple(slot);
+ tuple = ExecMaterializeSlot(slot);
+ tuple->t_tableOid = RelationGetRelid(resultRelInfo->ri_RelationDesc);
+
/* Return NULL if nothing was deleted on the remote end */
return (n_rows > 0) ? slot : NULL;
}
*** a/contrib/postgres_fdw/sql/postgres_fdw.sql
--- b/contrib/postgres_fdw/sql/postgres_fdw.sql
***************
*** 413,418 **** EXPLAIN (verbose, costs off)
--- 413,427 ----
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
+ EXPLAIN (verbose, costs off)
+ INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
+ INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
+ UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
+ EXPLAIN (verbose, costs off)
+ DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
+ DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
On 2016/01/28 15:20, Rushabh Lathia wrote:
On Thu, Jan 28, 2016 at 11:33 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp <mailto:fujita.etsuro@lab.ntt.co.jp>> wrote:On 2016/01/27 21:23, Rushabh Lathia wrote:
If I understood correctly, above documentation means, that if
FDW have
DMLPushdown APIs that is enough. But in reality thats not the
case, we
need ExecForeignInsert, ExecForeignUpdate, or ExecForeignDelete
in case
DML is not pushable.And here fact is DMLPushdown APIs are optional for FDW, so that
if FDW
don't have DMLPushdown APIs they can still very well perform the DML
operations using ExecForeignInsert, ExecForeignUpdate, or
ExecForeignDelete.
So documentation should be like:
If the IsForeignRelUpdatable pointer is set to NULL, foreign
tables are
assumed to be insertable, updatable, or deletable if the FDW
provides
ExecForeignInsert, ExecForeignUpdate, or ExecForeignDelete
respectively,If FDW provides DMLPushdown APIs and the DML are pushable to the
foreign
server, then FDW still needs ExecForeignInsert,
ExecForeignUpdate, or
ExecForeignDelete for the non-pushable DML operation.What's your opinion ?
I agree that we should add this to the documentation, too.
I added docs to the IsForeignRelUpdatable documentation. Also, a brief
introductory remark has been added at the beginning of the DML pushdown
APIs' documentation.
BTW, if I understand correctly, I think we should also modify
relation_is_updatabale() accordingly. Am I right?
Yep, we need to modify relation_is_updatable().
I thought I'd modify that function in the same way as
CheckValidResultRel(), but I noticed that we cannot do that, because we
don't have any information on whether each update is pushed down to the
remote server by PlanDMLPushdown, during relation_is_updatabale(). So,
I left that function as-is. relation_is_updatabale() is just used for
display in the information_schema views, so ISTM that that function is
fine as-is. (As for CheckValidResultRel(), I revised it so as to check
the presence of DML pushdown APIs after checking the existing APIs if
the given command will be pushed down. The reason is because we assume
the presence of the existing APIs, anyway.)
I revised other docs and some comments, mostly for consistency.
Attached is an updated version of the patch, which has been created on
top of the updated version of the bugfix patch posted by Robert in [1]/messages/by-id/CA+TgmoZ40j2uC5aC1NXu03oj4CrVOLkS15XX+PTFP-1U-8zR1Q@mail.gmail.com
(attached).
Best regards,
Etsuro Fujita
[1]: /messages/by-id/CA+TgmoZ40j2uC5aC1NXu03oj4CrVOLkS15XX+PTFP-1U-8zR1Q@mail.gmail.com
/messages/by-id/CA+TgmoZ40j2uC5aC1NXu03oj4CrVOLkS15XX+PTFP-1U-8zR1Q@mail.gmail.com
Attachments:
fdw-foreign-modify-rmh-v2.patchapplication/x-patch; name=fdw-foreign-modify-rmh-v2.patchDownload
diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c
index df3d1ee..d778e61 100644
--- a/contrib/postgres_fdw/deparse.c
+++ b/contrib/postgres_fdw/deparse.c
@@ -112,6 +112,7 @@ static void deparseTargetList(StringInfo buf,
PlannerInfo *root,
Index rtindex,
Relation rel,
+ bool is_returning,
Bitmapset *attrs_used,
List **retrieved_attrs);
static void deparseReturningList(StringInfo buf, PlannerInfo *root,
@@ -776,7 +777,7 @@ deparseSelectSql(Bitmapset *attrs_used, List **retrieved_attrs,
* Construct SELECT list
*/
appendStringInfoString(buf, "SELECT ");
- deparseTargetList(buf, root, foreignrel->relid, rel, attrs_used,
+ deparseTargetList(buf, root, foreignrel->relid, rel, false, attrs_used,
retrieved_attrs);
/*
@@ -790,7 +791,8 @@ deparseSelectSql(Bitmapset *attrs_used, List **retrieved_attrs,
/*
* Emit a target list that retrieves the columns specified in attrs_used.
- * This is used for both SELECT and RETURNING targetlists.
+ * This is used for both SELECT and RETURNING targetlists; the is_returning
+ * parameter is true only for a RETURNING targetlist.
*
* The tlist text is appended to buf, and we also create an integer List
* of the columns being retrieved, which is returned to *retrieved_attrs.
@@ -800,6 +802,7 @@ deparseTargetList(StringInfo buf,
PlannerInfo *root,
Index rtindex,
Relation rel,
+ bool is_returning,
Bitmapset *attrs_used,
List **retrieved_attrs)
{
@@ -829,6 +832,8 @@ deparseTargetList(StringInfo buf,
{
if (!first)
appendStringInfoString(buf, ", ");
+ else if (is_returning)
+ appendStringInfoString(buf, " RETURNING ");
first = false;
deparseColumnRef(buf, rtindex, i, root);
@@ -846,6 +851,8 @@ deparseTargetList(StringInfo buf,
{
if (!first)
appendStringInfoString(buf, ", ");
+ else if (is_returning)
+ appendStringInfoString(buf, " RETURNING ");
first = false;
appendStringInfoString(buf, "ctid");
@@ -855,7 +862,7 @@ deparseTargetList(StringInfo buf,
}
/* Don't generate bad syntax if no undropped columns */
- if (first)
+ if (first && !is_returning)
appendStringInfoString(buf, "NULL");
}
@@ -1113,11 +1120,8 @@ deparseReturningList(StringInfo buf, PlannerInfo *root,
}
if (attrs_used != NULL)
- {
- appendStringInfoString(buf, " RETURNING ");
- deparseTargetList(buf, root, rtindex, rel, attrs_used,
+ deparseTargetList(buf, root, rtindex, rel, true, attrs_used,
retrieved_attrs);
- }
else
*retrieved_attrs = NIL;
}
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index 2390e61..f90ab5a 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2408,6 +2408,59 @@ SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
1104 | 204 | ddd |
(819 rows)
+EXPLAIN (verbose, costs off)
+INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
+ QUERY PLAN
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Insert on public.ft2
+ Output: (tableoid)::regclass
+ Remote SQL: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6, c7, c8) VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
+ -> Result
+ Output: 9999, 999, NULL::integer, 'foo'::text, NULL::timestamp with time zone, NULL::timestamp without time zone, NULL::character varying, 'ft2 '::character(10), NULL::user_enum
+(5 rows)
+
+INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
+ tableoid
+----------
+ ft2
+(1 row)
+
+EXPLAIN (verbose, costs off)
+UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
+ QUERY PLAN
+-------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Output: (tableoid)::regclass
+ Remote SQL: UPDATE "S 1"."T 1" SET c3 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.ft2
+ Output: c1, c2, NULL::integer, 'bar'::text, c4, c5, c6, c7, c8, ctid
+ Remote SQL: SELECT "C 1", c2, c4, c5, c6, c7, c8, ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
+(6 rows)
+
+UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
+ tableoid
+----------
+ ft2
+(1 row)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
+ QUERY PLAN
+------------------------------------------------------------------------------------
+ Delete on public.ft2
+ Output: (tableoid)::regclass
+ Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1
+ -> Foreign Scan on public.ft2
+ Output: ctid
+ Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
+(6 rows)
+
+DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
+ tableoid
+----------
+ ft2
+(1 row)
+
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
BEGIN
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 5c6ead1..f198e8e 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -413,6 +413,15 @@ EXPLAIN (verbose, costs off)
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
+EXPLAIN (verbose, costs off)
+INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
+INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
+EXPLAIN (verbose, costs off)
+UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
+UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
+EXPLAIN (verbose, costs off)
+DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
+DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
-- Test that trigger on remote table works as expected
CREATE OR REPLACE FUNCTION "S 1".F_BRTRIG() RETURNS trigger AS $$
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 46299fc..27051e8 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -308,6 +308,12 @@ ExecInsert(ModifyTableState *mtstate,
/* FDW might have changed tuple */
tuple = ExecMaterializeSlot(slot);
+ /*
+ * AFTER ROW Triggers or RETURNING expressions might reference the
+ * tableoid column, so initialize t_tableOid before evaluating them.
+ */
+ tuple->t_tableOid = RelationGetRelid(resultRelationDesc);
+
newId = InvalidOid;
}
else
@@ -561,6 +567,8 @@ ExecDelete(ItemPointer tupleid,
}
else if (resultRelInfo->ri_FdwRoutine)
{
+ HeapTuple tuple;
+
/*
* delete from foreign table: let the FDW do it
*
@@ -579,6 +587,15 @@ ExecDelete(ItemPointer tupleid,
if (slot == NULL) /* "do nothing" */
return NULL;
+
+ /*
+ * RETURNING expressions might reference the tableoid column, so
+ * initialize t_tableOid before evaluating them.
+ */
+ if (slot->tts_isempty)
+ ExecStoreAllNullTuple(slot);
+ tuple = ExecMaterializeSlot(slot);
+ tuple->t_tableOid = RelationGetRelid(resultRelationDesc);
}
else
{
@@ -838,6 +855,12 @@ ExecUpdate(ItemPointer tupleid,
/* FDW might have changed tuple */
tuple = ExecMaterializeSlot(slot);
+
+ /*
+ * AFTER ROW Triggers or RETURNING expressions might reference the
+ * tableoid column, so initialize t_tableOid before evaluating them.
+ */
+ tuple->t_tableOid = RelationGetRelid(resultRelationDesc);
}
else
{
fdw-dml-pushdown-v5.patchapplication/x-patch; name=fdw-dml-pushdown-v5.patchDownload
*** a/contrib/postgres_fdw/deparse.c
--- b/contrib/postgres_fdw/deparse.c
***************
*** 1069,1074 **** deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 1069,1134 ----
}
/*
+ * deparse remote UPDATE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ deparse_expr_cxt context;
+ int nestlevel;
+ bool first;
+ ListCell *lc;
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = params_list;
+
+ appendStringInfoString(buf, "UPDATE ");
+ deparseRelation(buf, rel);
+ appendStringInfoString(buf, " SET ");
+
+ /* Make sure any constants in the exprs are printed portably */
+ nestlevel = set_transmission_modes();
+
+ first = true;
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(targetlist, attnum);
+
+ if (!first)
+ appendStringInfoString(buf, ", ");
+ first = false;
+
+ deparseColumnRef(buf, rtindex, attnum, root);
+ appendStringInfoString(buf, " = ");
+ deparseExpr((Expr *) tle->expr, &context);
+ }
+
+ reset_transmission_modes(nestlevel);
+
+ if (remote_conds)
+ appendWhereClause(remote_conds, &context);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* deparse remote DELETE statement
*
* The statement text is appended to buf, and we also create an integer List
***************
*** 1091,1096 **** deparseDeleteSql(StringInfo buf, PlannerInfo *root,
--- 1151,1190 ----
}
/*
+ * deparse remote DELETE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ deparse_expr_cxt context;
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = params_list;
+
+ appendStringInfoString(buf, "DELETE FROM ");
+ deparseRelation(buf, rel);
+
+ if (remote_conds)
+ appendWhereClause(remote_conds, &context);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* Add a RETURNING clause, if needed, to an INSERT/UPDATE/DELETE.
*/
static void
*** a/contrib/postgres_fdw/expected/postgres_fdw.out
--- b/contrib/postgres_fdw/expected/postgres_fdw.out
***************
*** 1314,1320 **** INSERT INTO ft2 (c1,c2,c3)
--- 1314,1339 ----
(3 rows)
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 300), c3 = (c3 || '_update3'::text) WHERE ((("C 1" % 10) = 3))
+ (3 rows)
+
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Output: c1, c2, c3, c4, c5, c6, c7, c8
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 400), c3 = (c3 || '_update7'::text) WHERE ((("C 1" % 10) = 7)) RETURNING "C 1", c2, c3, c4, c5, c6, c7, c8
+ (4 rows)
+
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
c1 | c2 | c3 | c4 | c5 | c6 | c7 | c8
------+-----+--------------------+------------------------------+--------------------------+----+------------+-----
***************
*** 1424,1430 **** UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
--- 1443,1449 ----
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
***************
*** 1445,1460 **** UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
! QUERY PLAN
! ----------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1 RETURNING "C 1", c4
! -> Foreign Scan on public.ft2
! Output: ctid
! Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) FOR UPDATE
! (6 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
--- 1464,1477 ----
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
! QUERY PLAN
! --------------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
! -> Foreign Delete on public.ft2
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) RETURNING "C 1", c4
! (4 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
***************
*** 1565,1571 **** DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
--- 1582,1588 ----
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
***************
*** 2426,2441 **** INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
(1 row)
EXPLAIN (verbose, costs off)
! UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
! QUERY PLAN
! -------------------------------------------------------------------------------------------------------------------
Update on public.ft2
Output: (tableoid)::regclass
! Remote SQL: UPDATE "S 1"."T 1" SET c3 = $2 WHERE ctid = $1
! -> Foreign Scan on public.ft2
! Output: c1, c2, NULL::integer, 'bar'::text, c4, c5, c6, c7, c8, ctid
! Remote SQL: SELECT "C 1", c2, c4, c5, c6, c7, c8, ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
! (6 rows)
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
--- 2443,2456 ----
(1 row)
EXPLAIN (verbose, costs off)
! UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
! QUERY PLAN
! ------------------------------------------------------------------------------------
Update on public.ft2
Output: (tableoid)::regclass
! -> Foreign Update on public.ft2
! Remote SQL: UPDATE "S 1"."T 1" SET c3 = 'bar'::text WHERE (("C 1" = 9999))
! (4 rows)
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
***************
*** 2444,2459 **** UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
(1 row)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
! QUERY PLAN
! ------------------------------------------------------------------------------------
Delete on public.ft2
Output: (tableoid)::regclass
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1
! -> Foreign Scan on public.ft2
! Output: ctid
! Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
! (6 rows)
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
--- 2459,2472 ----
(1 row)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
! QUERY PLAN
! --------------------------------------------------------------------
Delete on public.ft2
Output: (tableoid)::regclass
! -> Foreign Delete on public.ft2
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE (("C 1" = 9999))
! (4 rows)
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
***************
*** 2607,2613 **** CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 2620,2626 ----
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 2766,2772 **** savepoint s3;
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 2779,2785 ----
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (-2) WHERE ((c2 = 42)) AND (("C 1" = 10))
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 2906,2912 **** CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2positive;
-- But inconsistent check constraints provide inconsistent results
ALTER FOREIGN TABLE ft1 ADD CONSTRAINT ft1_c2negative CHECK (c2 < 0);
--- 2919,2925 ----
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2positive;
-- But inconsistent check constraints provide inconsistent results
ALTER FOREIGN TABLE ft1 ADD CONSTRAINT ft1_c2negative CHECK (c2 < 0);
***************
*** 3299,3304 **** NOTICE: NEW: (13,"test triggered !")
--- 3312,3510 ----
(0,27)
(1 row)
+ -- cleanup
+ DROP TRIGGER trig_row_before ON rem1;
+ DROP TRIGGER trig_row_after ON rem1;
+ DROP TRIGGER trig_local_before ON loc1;
+ -- Test DML pushdown functionality
+ -- Test with statement-level triggers
+ CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_stmt_before ON rem1;
+ CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_stmt_after ON rem1;
+ -- Test with row-level ON INSERT triggers
+ CREATE TRIGGER trig_row_before_insert
+ BEFORE INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_before_insert ON rem1;
+ CREATE TRIGGER trig_row_after_insert
+ AFTER INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_after_insert ON rem1;
+ -- Test with row-level ON UPDATE triggers
+ CREATE TRIGGER trig_row_before_update
+ BEFORE UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_before_update ON rem1;
+ CREATE TRIGGER trig_row_after_update
+ AFTER UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+ -------------------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_after_update ON rem1;
+ -- Test with row-level ON DELETE triggers
+ CREATE TRIGGER trig_row_before_delete
+ BEFORE DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ DROP TRIGGER trig_row_before_delete ON rem1;
+ CREATE TRIGGER trig_row_after_delete
+ AFTER DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+ ------------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ DROP TRIGGER trig_row_after_delete ON rem1;
-- ===================================================================
-- test inheritance features
-- ===================================================================
***************
*** 3768,3773 **** fetch from c;
--- 3974,4029 ----
update bar set f2 = null where current of c;
ERROR: WHERE CURRENT OF is not supported for this table type
rollback;
+ explain (verbose, costs off)
+ delete from foo where f1 < 5 returning *;
+ QUERY PLAN
+ --------------------------------------------------------------------------------
+ Delete on public.foo
+ Output: foo.f1, foo.f2
+ Delete on public.foo
+ Foreign Delete on public.foo2
+ -> Index Scan using i_foo_f1 on public.foo
+ Output: foo.ctid
+ Index Cond: (foo.f1 < 5)
+ -> Foreign Delete on public.foo2
+ Remote SQL: DELETE FROM public.loct1 WHERE ((f1 < 5)) RETURNING f1, f2
+ (9 rows)
+
+ delete from foo where f1 < 5 returning *;
+ f1 | f2
+ ----+----
+ 1 | 1
+ 3 | 3
+ 0 | 0
+ 2 | 2
+ 4 | 4
+ (5 rows)
+
+ explain (verbose, costs off)
+ update bar set f2 = f2 + 100 returning *;
+ QUERY PLAN
+ ------------------------------------------------------------------------------
+ Update on public.bar
+ Output: bar.f1, bar.f2
+ Update on public.bar
+ Foreign Update on public.bar2
+ -> Seq Scan on public.bar
+ Output: bar.f1, (bar.f2 + 100), bar.ctid
+ -> Foreign Update on public.bar2
+ Remote SQL: UPDATE public.loct2 SET f2 = (f2 + 100) RETURNING f1, f2
+ (8 rows)
+
+ update bar set f2 = f2 + 100 returning *;
+ f1 | f2
+ ----+-----
+ 1 | 311
+ 2 | 322
+ 6 | 266
+ 3 | 333
+ 4 | 344
+ 7 | 277
+ (6 rows)
+
drop table foo cascade;
NOTICE: drop cascades to foreign table foo2
drop table bar cascade;
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
***************
*** 57,63 **** PG_MODULE_MAGIC;
* planner to executor. Currently we store:
*
* 1) SELECT statement text to be sent to the remote server
! * 2) Integer list of attribute numbers retrieved by the SELECT
*
* These items are indexed with the enum FdwScanPrivateIndex, so an item
* can be fetched with list_nth(). For example, to get the SELECT statement:
--- 57,64 ----
* planner to executor. Currently we store:
*
* 1) SELECT statement text to be sent to the remote server
! * 2) List of restriction clauses that can be executed remotely
! * 3) Integer list of attribute numbers retrieved by the SELECT
*
* These items are indexed with the enum FdwScanPrivateIndex, so an item
* can be fetched with list_nth(). For example, to get the SELECT statement:
***************
*** 67,72 **** enum FdwScanPrivateIndex
--- 68,75 ----
{
/* SQL statement to execute remotely (as a String node) */
FdwScanPrivateSelectSql,
+ /* List of restriction clauses that can be executed remotely */
+ FdwScanPrivateRemoteConds,
/* Integer list of attribute numbers retrieved by the SELECT */
FdwScanPrivateRetrievedAttrs
};
***************
*** 94,99 **** enum FdwModifyPrivateIndex
--- 97,124 ----
};
/*
+ * Similarly, this enum describes what's kept in the fdw_private list for
+ * a ForeignScan node that has pushed down an UPDATE/DELETE to the remote
+ * server. We store:
+ *
+ * 1) UPDATE/DELETE statement text to be sent to the remote server
+ * 2) Boolean flag showing if the remote query has a RETURNING clause
+ * 3) Integer list of attribute numbers retrieved by RETURNING, if any
+ * 4) Boolean flag showing if we set the command es_processed
+ */
+ enum FdwDmlPushdownPrivateIndex
+ {
+ /* SQL statement to execute remotely (as a String node) */
+ FdwDmlPushdownPrivateUpdateSql,
+ /* has-returning flag (as an integer Value node) */
+ FdwDmlPushdownPrivateHasReturning,
+ /* Integer list of attribute numbers retrieved by RETURNING */
+ FdwDmlPushdownPrivateRetrievedAttrs,
+ /* set-processed flag (as an integer Value node) */
+ FdwDmlPushdownPrivateSetProcessed
+ };
+
+ /*
* Execution state of a foreign scan using postgres_fdw.
*/
typedef struct PgFdwScanState
***************
*** 156,161 **** typedef struct PgFdwModifyState
--- 181,217 ----
} PgFdwModifyState;
/*
+ * Execution state of a foreign scan that has pushed down a foreign table
+ * modification to the remote server
+ */
+ typedef struct PgFdwDmlPushdownState
+ {
+ Relation rel; /* relcache entry for the foreign table */
+ AttInMetadata *attinmeta; /* attribute datatype conversion metadata */
+
+ /* extracted fdw_private data */
+ char *query; /* text of UPDATE/DELETE command */
+ bool has_returning; /* is there a RETURNING clause? */
+ List *retrieved_attrs; /* attr numbers retrieved by RETURNING */
+ bool set_processed; /* do we set the command es_processed? */
+
+ /* for remote query execution */
+ PGconn *conn; /* connection for the update */
+ int numParams; /* number of parameters passed to query */
+ FmgrInfo *param_flinfo; /* output conversion functions for them */
+ List *param_exprs; /* executable expressions for param values */
+ const char **param_values; /* textual values of query parameters */
+
+ /* for storing result tuples */
+ PGresult *result; /* result for query */
+ int num_tuples; /* # of result tuples */
+ int next_tuple; /* index of next one to return */
+
+ /* working memory context */
+ MemoryContext temp_cxt; /* context for per-tuple temporary data */
+ } PgFdwDmlPushdownState;
+
+ /*
* Workspace for analyzing a foreign table.
*/
typedef struct PgFdwAnalyzeState
***************
*** 247,252 **** static TupleTableSlot *postgresExecForeignDelete(EState *estate,
--- 303,315 ----
static void postgresEndForeignModify(EState *estate,
ResultRelInfo *resultRelInfo);
static int postgresIsForeignRelUpdatable(Relation rel);
+ static bool postgresPlanDMLPushdown(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+ static void postgresBeginDMLPushdown(ForeignScanState *node, int eflags);
+ static TupleTableSlot *postgresIterateDMLPushdown(ForeignScanState *node);
+ static void postgresEndDMLPushdown(ForeignScanState *node);
static void postgresExplainForeignScan(ForeignScanState *node,
ExplainState *es);
static void postgresExplainForeignModify(ModifyTableState *mtstate,
***************
*** 254,259 **** static void postgresExplainForeignModify(ModifyTableState *mtstate,
--- 317,324 ----
List *fdw_private,
int subplan_index,
ExplainState *es);
+ static void postgresExplainDMLPushdown(ForeignScanState *node,
+ ExplainState *es);
static bool postgresAnalyzeForeignTable(Relation relation,
AcquireSampleRowsFunc *func,
BlockNumber *totalpages);
***************
*** 290,295 **** static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
--- 355,362 ----
TupleTableSlot *slot);
static void store_returning_result(PgFdwModifyState *fmstate,
TupleTableSlot *slot, PGresult *res);
+ static void execute_dml_stmt(ForeignScanState *node);
+ static TupleTableSlot *get_returning_data(ForeignScanState *node);
static int postgresAcquireSampleRowsFunc(Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
***************
*** 332,341 **** postgres_fdw_handler(PG_FUNCTION_ARGS)
--- 399,413 ----
routine->ExecForeignDelete = postgresExecForeignDelete;
routine->EndForeignModify = postgresEndForeignModify;
routine->IsForeignRelUpdatable = postgresIsForeignRelUpdatable;
+ routine->PlanDMLPushdown = postgresPlanDMLPushdown;
+ routine->BeginDMLPushdown = postgresBeginDMLPushdown;
+ routine->IterateDMLPushdown = postgresIterateDMLPushdown;
+ routine->EndDMLPushdown = postgresEndDMLPushdown;
/* Support functions for EXPLAIN */
routine->ExplainForeignScan = postgresExplainForeignScan;
routine->ExplainForeignModify = postgresExplainForeignModify;
+ routine->ExplainDMLPushdown = postgresExplainDMLPushdown;
/* Support functions for ANALYZE */
routine->AnalyzeForeignTable = postgresAnalyzeForeignTable;
***************
*** 1010,1016 **** postgresGetForeignPlan(PlannerInfo *root,
* Build the fdw_private list that will be available to the executor.
* Items in the list must match enum FdwScanPrivateIndex, above.
*/
! fdw_private = list_make2(makeString(sql.data),
retrieved_attrs);
/*
--- 1082,1089 ----
* Build the fdw_private list that will be available to the executor.
* Items in the list must match enum FdwScanPrivateIndex, above.
*/
! fdw_private = list_make3(makeString(sql.data),
! remote_conds,
retrieved_attrs);
/*
***************
*** 1304,1316 **** postgresAddForeignUpdateTargets(Query *parsetree,
/*
* postgresPlanForeignModify
* Plan an insert/update/delete operation on a foreign table
- *
- * Note: currently, the plan tree generated for UPDATE/DELETE will always
- * include a ForeignScan that retrieves ctids (using SELECT FOR UPDATE)
- * and then the ModifyTable node will have to execute individual remote
- * UPDATE/DELETE commands. If there are no local conditions or joins
- * needed, it'd be better to let the scan node do UPDATE/DELETE RETURNING
- * and then do nothing at ModifyTable. Room for future optimization ...
*/
static List *
postgresPlanForeignModify(PlannerInfo *root,
--- 1377,1382 ----
***************
*** 1821,1826 **** postgresIsForeignRelUpdatable(Relation rel)
--- 1887,2223 ----
}
/*
+ * postgresPlanDMLPushdown
+ * Consider pushing down a foreign table modification to the remote server
+ *
+ * Decide whether the table modification is safe to push down to the remote end,
+ * and if so, modify subplan so as to do that.
+ *
+ * Conditions checked here:
+ *
+ * 1. The table modification must be an UPDATE or DELETE.
+ *
+ * 2. It's unsafe to push down the command if there are any local joins needed.
+ *
+ * 3. It's unsafe to push down the command if there are any quals that can't be
+ * evaluated remotely.
+ *
+ * 4. We can't push down an UPDATE, if any expressions to assign to the target
+ * columns are unsafe to evaluate on the remote end.
+ */
+ static bool
+ postgresPlanDMLPushdown(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index)
+ {
+ CmdType operation = plan->operation;
+ Plan *subplan = (Plan *) list_nth(plan->plans, subplan_index);
+ RangeTblEntry *rte = planner_rt_fetch(resultRelation, root);
+ Relation rel;
+ StringInfoData sql;
+ ForeignScan *fscan;
+ List *targetAttrs = NIL;
+ List *remote_conds;
+ List *params_list = NIL;
+ List *returningList = NIL;
+ List *retrieved_attrs = NIL;
+
+ /*
+ * Decide whether the table modification is safe to push down to the remote
+ * server.
+ */
+
+ /* Check point 1 */
+ if (operation == CMD_INSERT)
+ return false;
+
+ /* Check point 2 */
+ if (nodeTag(subplan) != T_ForeignScan)
+ return false;
+
+ /* Check point 3 */
+ if (subplan->qual != NIL)
+ return false;
+
+ /* Check point 4 */
+ if (operation == CMD_UPDATE)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[resultRelation];
+ int col;
+
+ /*
+ * We transmit only columns that were explicitly targets of the UPDATE,
+ * so as to avoid unnecessary data transmission.
+ */
+ col = -1;
+ while ((col = bms_next_member(rte->updatedCols, col)) >= 0)
+ {
+ /* bit numbers are offset by FirstLowInvalidHeapAttributeNumber */
+ AttrNumber attno = col + FirstLowInvalidHeapAttributeNumber;
+ TargetEntry *tle;
+
+ if (attno <= InvalidAttrNumber) /* shouldn't happen */
+ elog(ERROR, "system-column update is not supported");
+
+ tle = get_tle_by_resno(subplan->targetlist, attno);
+
+ if (!is_foreign_expr(root, baserel, (Expr *) tle->expr))
+ return false;
+
+ targetAttrs = lappend_int(targetAttrs, attno);
+ }
+ }
+
+ /*
+ * Ok, modify subplan so as to push down the command to the remote server.
+ */
+ fscan = (ForeignScan *) subplan;
+
+ initStringInfo(&sql);
+
+ /*
+ * Core code already has some lock on each rel being planned, so we can
+ * use NoLock here.
+ */
+ rel = heap_open(rte->relid, NoLock);
+
+ /*
+ * Extract the baserestrictinfo clauses that can be evaluated remotely.
+ */
+ remote_conds = (List *) list_nth(fscan->fdw_private,
+ FdwScanPrivateRemoteConds);
+
+ /*
+ * Extract the relevant RETURNING list if any.
+ */
+ if (plan->returningLists)
+ returningList = (List *) list_nth(plan->returningLists, subplan_index);
+
+ /*
+ * Construct the SQL command string.
+ */
+ switch (operation)
+ {
+ case CMD_UPDATE:
+ deparsePushedDownUpdateSql(&sql, root, resultRelation, rel,
+ ((Plan *) fscan)->targetlist,
+ targetAttrs,
+ remote_conds, ¶ms_list,
+ returningList, &retrieved_attrs);
+ break;
+ case CMD_DELETE:
+ deparsePushedDownDeleteSql(&sql, root, resultRelation, rel,
+ remote_conds, ¶ms_list,
+ returningList, &retrieved_attrs);
+ break;
+ default:
+ elog(ERROR, "unexpected operation: %d", (int) operation);
+ break;
+ }
+
+ /*
+ * Update the operation info.
+ */
+ fscan->operation = operation;
+
+ /*
+ * Update the fdw_exprs list that will be available to the executor.
+ */
+ fscan->fdw_exprs = params_list;
+
+ /*
+ * Update the fdw_private list that will be available to the executor.
+ * Items in the list must match enum FdwDmlPushdownPrivateIndex, above.
+ */
+ fscan->fdw_private = list_make4(makeString(sql.data),
+ makeInteger((retrieved_attrs != NIL)),
+ retrieved_attrs,
+ makeInteger(plan->canSetTag));
+
+ heap_close(rel, NoLock);
+ return true;
+ }
+
+ /*
+ * postgresBeginDMLPushdown
+ * Initiate pushing down a foreign table modification to the remote server
+ */
+ static void
+ postgresBeginDMLPushdown(ForeignScanState *node, int eflags)
+ {
+ ForeignScan *fsplan = (ForeignScan *) node->ss.ps.plan;
+ EState *estate = node->ss.ps.state;
+ PgFdwDmlPushdownState *dpstate;
+ RangeTblEntry *rte;
+ Oid userid;
+ ForeignTable *table;
+ UserMapping *user;
+ int numParams;
+ int i;
+ ListCell *lc;
+
+ /*
+ * Do nothing in EXPLAIN (no ANALYZE) case. node->fdw_state stays NULL.
+ */
+ if (eflags & EXEC_FLAG_EXPLAIN_ONLY)
+ return;
+
+ /*
+ * We'll save private state in node->fdw_state.
+ */
+ dpstate = (PgFdwDmlPushdownState *) palloc0(sizeof(PgFdwDmlPushdownState));
+ node->fdw_state = (void *) dpstate;
+
+ /*
+ * Identify which user to do the remote access as. This should match what
+ * ExecCheckRTEPerms() does.
+ */
+ rte = rt_fetch(fsplan->scan.scanrelid, estate->es_range_table);
+ userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
+
+ /* Get info about foreign table. */
+ dpstate->rel = node->ss.ss_currentRelation;
+ table = GetForeignTable(RelationGetRelid(dpstate->rel));
+ user = GetUserMapping(userid, table->serverid);
+
+ /*
+ * Get connection to the foreign server. Connection manager will
+ * establish new connection if necessary.
+ */
+ dpstate->conn = GetConnection(user, false);
+
+ /* Initialize state variable */
+ dpstate->num_tuples = -1; /* -1 means not set yet */
+
+ /* Get private info created by planner functions. */
+ dpstate->query = strVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateUpdateSql));
+ dpstate->has_returning = intVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateHasReturning));
+ dpstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateRetrievedAttrs);
+ dpstate->set_processed = intVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateSetProcessed));
+
+ /* Create context for per-tuple temp workspace. */
+ dpstate->temp_cxt = AllocSetContextCreate(estate->es_query_cxt,
+ "postgres_fdw temporary data",
+ ALLOCSET_SMALL_MINSIZE,
+ ALLOCSET_SMALL_INITSIZE,
+ ALLOCSET_SMALL_MAXSIZE);
+
+ /* Prepare for input conversion of RETURNING results. */
+ if (dpstate->has_returning)
+ dpstate->attinmeta = TupleDescGetAttInMetadata(RelationGetDescr(dpstate->rel));
+
+ /* Prepare for output conversion of parameters used in remote query. */
+ numParams = list_length(fsplan->fdw_exprs);
+ dpstate->numParams = numParams;
+ dpstate->param_flinfo = (FmgrInfo *) palloc0(sizeof(FmgrInfo) * numParams);
+
+ i = 0;
+ foreach(lc, fsplan->fdw_exprs)
+ {
+ Node *param_expr = (Node *) lfirst(lc);
+ Oid typefnoid;
+ bool isvarlena;
+
+ getTypeOutputInfo(exprType(param_expr), &typefnoid, &isvarlena);
+ fmgr_info(typefnoid, &dpstate->param_flinfo[i]);
+ i++;
+ }
+
+ /*
+ * Prepare remote-parameter expressions for evaluation. (Note: in
+ * practice, we expect that all these expressions will be just Params, so
+ * we could possibly do something more efficient than using the full
+ * expression-eval machinery for this. But probably there would be little
+ * benefit, and it'd require postgres_fdw to know more than is desirable
+ * about Param evaluation.)
+ */
+ dpstate->param_exprs = (List *)
+ ExecInitExpr((Expr *) fsplan->fdw_exprs,
+ (PlanState *) node);
+
+ /*
+ * Allocate buffer for text form of query parameters, if any.
+ */
+ if (numParams > 0)
+ dpstate->param_values = (const char **) palloc0(numParams * sizeof(char *));
+ else
+ dpstate->param_values = NULL;
+ }
+
+ /*
+ * postgresIterateDMLPushdown
+ * Execute pushing down a foreign table modification to the remote server
+ */
+ static TupleTableSlot *
+ postgresIterateDMLPushdown(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ EState *estate = node->ss.ps.state;
+ ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
+
+ /*
+ * If this is the first call after Begin, execute the statement.
+ */
+ if (dpstate->num_tuples == -1)
+ execute_dml_stmt(node);
+
+ /*
+ * If the local query doesn't specify RETURNING, just clear tuple slot.
+ */
+ if (!resultRelInfo->ri_projectReturning)
+ {
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+ Instrumentation *instr = node->ss.ps.instrument;
+
+ Assert(!dpstate->has_returning);
+
+ /* Increment the command es_processed count if necessary. */
+ if (dpstate->set_processed)
+ estate->es_processed += dpstate->num_tuples;
+
+ /* Increment the tuple count for EXPLAIN ANALYZE if necessary. */
+ if (instr)
+ instr->tuplecount += dpstate->num_tuples;
+
+ return ExecClearTuple(slot);
+ }
+
+ /*
+ * Get the next RETURNING tuple.
+ */
+ return get_returning_data(node);
+ }
+
+ /*
+ * postgresEndDMLPushdown
+ * Finish pushing down a foreign table modification to the remote server
+ */
+ static void
+ postgresEndDMLPushdown(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+
+ /* if dpstate is NULL, we are in EXPLAIN; nothing to do */
+ if (dpstate == NULL)
+ return;
+
+ /* Release PGresult */
+ if (dpstate->result)
+ PQclear(dpstate->result);
+
+ /* Release remote connection */
+ ReleaseConnection(dpstate->conn);
+ dpstate->conn = NULL;
+
+ /* MemoryContext will be deleted automatically. */
+ }
+
+ /*
* postgresExplainForeignScan
* Produce extra output for EXPLAIN of a ForeignScan on a foreign table
*/
***************
*** 1858,1863 **** postgresExplainForeignModify(ModifyTableState *mtstate,
--- 2255,2279 ----
}
}
+ /*
+ * postgresExplainDMLPushdown
+ * Produce extra output for EXPLAIN of a ForeignScan on a foreign table
+ * that has pushed down an UPDATE/DELETE to the remote server
+ */
+ static void
+ postgresExplainDMLPushdown(ForeignScanState *node, ExplainState *es)
+ {
+ List *fdw_private;
+ char *sql;
+
+ if (es->verbose)
+ {
+ fdw_private = ((ForeignScan *) node->ss.ps.plan)->fdw_private;
+ sql = strVal(list_nth(fdw_private, FdwDmlPushdownPrivateUpdateSql));
+ ExplainPropertyText("Remote SQL", sql, es);
+ }
+ }
+
/*
* estimate_path_cost_size
***************
*** 2474,2479 **** store_returning_result(PgFdwModifyState *fmstate,
--- 2890,3028 ----
}
/*
+ * Execute a pushed-down UPDATE/DELETE statement.
+ */
+ static void
+ execute_dml_stmt(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ ExprContext *econtext = node->ss.ps.ps_ExprContext;
+ int numParams = dpstate->numParams;
+ const char **values = dpstate->param_values;
+
+ /*
+ * Construct array of query parameter values in text format.
+ */
+ if (numParams > 0)
+ {
+ int nestlevel;
+ int i;
+ ListCell *lc;
+
+ nestlevel = set_transmission_modes();
+
+ i = 0;
+ foreach(lc, dpstate->param_exprs)
+ {
+ ExprState *expr_state = (ExprState *) lfirst(lc);
+ Datum expr_value;
+ bool isNull;
+
+ /* Evaluate the parameter expression */
+ expr_value = ExecEvalExpr(expr_state, econtext, &isNull, NULL);
+
+ /*
+ * Get string representation of each parameter value by invoking
+ * type-specific output function, unless the value is null.
+ */
+ if (isNull)
+ values[i] = NULL;
+ else
+ values[i] = OutputFunctionCall(&dpstate->param_flinfo[i],
+ expr_value);
+ i++;
+ }
+
+ reset_transmission_modes(nestlevel);
+ }
+
+ /*
+ * Notice that we pass NULL for paramTypes, thus forcing the remote server
+ * to infer types for all parameters. Since we explicitly cast every
+ * parameter (see deparse.c), the "inference" is trivial and will produce
+ * the desired result. This allows us to avoid assuming that the remote
+ * server has the same OIDs we do for the parameters' types.
+ *
+ * We don't use a PG_TRY block here, so be careful not to throw error
+ * without releasing the PGresult.
+ */
+ dpstate->result = PQexecParams(dpstate->conn, dpstate->query,
+ numParams, NULL, values, NULL, NULL, 0);
+ if (PQresultStatus(dpstate->result) !=
+ (dpstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
+ pgfdw_report_error(ERROR, dpstate->result, dpstate->conn, true,
+ dpstate->query);
+
+ /* Get the number of rows affected. */
+ if (dpstate->has_returning)
+ dpstate->num_tuples = PQntuples(dpstate->result);
+ else
+ dpstate->num_tuples = atoi(PQcmdTuples(dpstate->result));
+ }
+
+ /*
+ * Get the result of a RETURNING clause.
+ */
+ static TupleTableSlot *
+ get_returning_data(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ EState *estate = node->ss.ps.state;
+ ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+
+ Assert(resultRelInfo->ri_projectReturning);
+
+ /* If we didn't get any tuples, must be end of data. */
+ if (dpstate->next_tuple >= dpstate->num_tuples)
+ return ExecClearTuple(slot);
+
+ /* Increment the command es_processed count if necessary. */
+ if (dpstate->set_processed)
+ estate->es_processed += 1;
+
+ /*
+ * Store a RETURNING tuple. If has_returning is false, just emit a dummy
+ * tuple. (We have has_returning=false if the local query is of the form
+ * UPDATE/DELETE .. RETURNING 1 for example.)
+ */
+ if (!dpstate->has_returning)
+ ExecStoreAllNullTuple(slot);
+ else
+ {
+ /*
+ * On error, be sure to release the PGresult on the way out. Callers
+ * do not have PG_TRY blocks to ensure this happens.
+ */
+ PG_TRY();
+ {
+ HeapTuple newtup;
+
+ newtup = make_tuple_from_result_row(dpstate->result,
+ dpstate->next_tuple,
+ dpstate->rel,
+ dpstate->attinmeta,
+ dpstate->retrieved_attrs,
+ dpstate->temp_cxt);
+ ExecStoreTuple(newtup, slot, InvalidBuffer, false);
+ }
+ PG_CATCH();
+ {
+ if (dpstate->result)
+ PQclear(dpstate->result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ }
+ dpstate->next_tuple++;
+
+ /* Make slot available for evaluation of the local query RETURNING list. */
+ resultRelInfo->ri_projectReturning->pi_exprContext->ecxt_scantuple = slot;
+
+ return slot;
+ }
+
+ /*
* postgresAnalyzeForeignTable
* Test whether analyzing this foreign table is supported
*/
*** a/contrib/postgres_fdw/postgres_fdw.h
--- b/contrib/postgres_fdw/postgres_fdw.h
***************
*** 91,100 **** extern void deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 91,114 ----
Index rtindex, Relation rel,
List *targetAttrs, List *returningList,
List **retrieved_attrs);
+ extern void deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseDeleteSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *returningList,
List **retrieved_attrs);
+ extern void deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseAnalyzeSizeSql(StringInfo buf, Relation rel);
extern void deparseAnalyzeSql(StringInfo buf, Relation rel,
List **retrieved_attrs);
*** a/contrib/postgres_fdw/sql/postgres_fdw.sql
--- b/contrib/postgres_fdw/sql/postgres_fdw.sql
***************
*** 399,426 **** INSERT INTO ft2 (c1,c2,c3) SELECT c1+1000,c2+100, c3 || c3 FROM ft2 LIMIT 20;
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
EXPLAIN (verbose, costs off)
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
! UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
-- Test that trigger on remote table works as expected
--- 399,430 ----
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
EXPLAIN (verbose, costs off)
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
! UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
-- Test that trigger on remote table works as expected
***************
*** 737,742 **** UPDATE rem1 SET f2 = 'testo';
--- 741,830 ----
-- Test returning a system attribute
INSERT INTO rem1(f2) VALUES ('test') RETURNING ctid;
+ -- cleanup
+ DROP TRIGGER trig_row_before ON rem1;
+ DROP TRIGGER trig_row_after ON rem1;
+ DROP TRIGGER trig_local_before ON loc1;
+
+
+ -- Test DML pushdown functionality
+
+ -- Test with statement-level triggers
+ CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_stmt_before ON rem1;
+
+ CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_stmt_after ON rem1;
+
+ -- Test with row-level ON INSERT triggers
+ CREATE TRIGGER trig_row_before_insert
+ BEFORE INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_before_insert ON rem1;
+
+ CREATE TRIGGER trig_row_after_insert
+ AFTER INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_after_insert ON rem1;
+
+ -- Test with row-level ON UPDATE triggers
+ CREATE TRIGGER trig_row_before_update
+ BEFORE UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_before_update ON rem1;
+
+ CREATE TRIGGER trig_row_after_update
+ AFTER UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_after_update ON rem1;
+
+ -- Test with row-level ON DELETE triggers
+ CREATE TRIGGER trig_row_before_delete
+ BEFORE DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ DROP TRIGGER trig_row_before_delete ON rem1;
+
+ CREATE TRIGGER trig_row_after_delete
+ AFTER DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ DROP TRIGGER trig_row_after_delete ON rem1;
+
-- ===================================================================
-- test inheritance features
-- ===================================================================
***************
*** 868,873 **** fetch from c;
--- 956,968 ----
update bar set f2 = null where current of c;
rollback;
+ explain (verbose, costs off)
+ delete from foo where f1 < 5 returning *;
+ delete from foo where f1 < 5 returning *;
+ explain (verbose, costs off)
+ update bar set f2 = f2 + 100 returning *;
+ update bar set f2 = f2 + 100 returning *;
+
drop table foo cascade;
drop table bar cascade;
drop table loct1;
*** a/doc/src/sgml/fdwhandler.sgml
--- b/doc/src/sgml/fdwhandler.sgml
***************
*** 657,669 **** IsForeignRelUpdatable (Relation rel);
<literal>NULL</>, foreign tables are assumed to be insertable, updatable,
or deletable if the FDW provides <function>ExecForeignInsert</>,
<function>ExecForeignUpdate</>, or <function>ExecForeignDelete</>
! respectively. This function is only needed if the FDW supports some
tables that are updatable and some that are not. (Even then, it's
permissible to throw an error in the execution routine instead of
checking in this function. However, this function is used to determine
updatability for display in the <literal>information_schema</> views.)
</para>
</sect2>
<sect2 id="fdw-callbacks-row-locking">
--- 657,819 ----
<literal>NULL</>, foreign tables are assumed to be insertable, updatable,
or deletable if the FDW provides <function>ExecForeignInsert</>,
<function>ExecForeignUpdate</>, or <function>ExecForeignDelete</>
! respectively.
! (If the FDW optimizes a foreign table update on a foreign table using
! <function>PlanDMLPushdown</>, it still needs to provide
! <function>BeginDMLPushdown</>, <function>IterateDMLPushdown</>, and
! <function>EndDMLPushdown</> to execute the optimized update.)
! This function is only needed if the FDW supports some
tables that are updatable and some that are not. (Even then, it's
permissible to throw an error in the execution routine instead of
checking in this function. However, this function is used to determine
updatability for display in the <literal>information_schema</> views.)
</para>
+ <para>
+ If an FDW supports an optimization that executes a foreign table update
+ directly on the remote server, it should provide the following callback
+ functions in addition to the table-updating functions described above:
+ </para>
+
+ <para>
+ <programlisting>
+ bool
+ PlanDMLPushdown (PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+ </programlisting>
+
+ Decide whether it is safe to execute a foreign table update directly
+ on the remote server. If so, return <literal>true</> after performing
+ planning actions needed for that. Otherwise, return <literal>false</>.
+ This function is executed just before <function>PlanForeignModify</>
+ is called. If this function succeeds, <function>PlanForeignModify</>
+ won't be executed, and <function>BeginDMLPushdown</>,
+ <function>IterateDMLPushdown</> and <function>EndDMLPushdown</> will
+ be called at the execution stage, instead. Otherwise, the table update
+ will be executed using the table-updating functions described above.
+ The parameters are the same as for <function>PlanForeignModify</>.
+ </para>
+
+ <para>
+ To execute the table update directly on the remote server, this function
+ must rewrite the target subplan with a <structname>ForeignScan</> plan
+ node that executes the table update directly on the remote server. The
+ <structfield>operation</> field of the <structname>ForeignScan</> must
+ be set to the <literal>CmdType</> enumeration appropriately; that is,
+ <literal>CMD_UPDATE</> for <command>UPDATE</>,
+ <literal>CMD_INSERT</> for <command>INSERT</>, and
+ <literal>CMD_DELETE</> for <command>DELETE</>.
+ </para>
+
+ <para>
+ See <xref linkend="fdw-planning"> for additional information.
+ </para>
+
+ <para>
+ If the <function>PlanDMLPushdown</> pointer is set to
+ <literal>NULL</>, no attempts to execute the table update directly on
+ the remote server are taken.
+ </para>
+
+ <para>
+ <programlisting>
+ void
+ BeginDMLPushdown (ForeignScanState *node,
+ int eflags);
+ </programlisting>
+
+ Begin executing a foreign table update directly on the remote server.
+ This is called during executor startup. It should perform any
+ initialization needed prior to the actual table update (that should be
+ done upon the first call to <function>IterateDMLPushdown</>).
+ The <structname>ForeignScanState</> node has already been created, but
+ its <structfield>fdw_state</> field is still NULL. Information about
+ the table to update is accessible through the
+ <structname>ForeignScanState</> node (in particular, from the underlying
+ <structname>ForeignScan</> plan node, which contains any FDW-private
+ information provided by <function>PlanDMLPushdown</>).
+ <literal>eflags</> contains flag bits describing the executor's
+ operating mode for this plan node.
+ </para>
+
+ <para>
+ Note that when <literal>(eflags & EXEC_FLAG_EXPLAIN_ONLY)</> is
+ true, this function should not perform any externally-visible actions;
+ it should only do the minimum required to make the node state valid
+ for <function>ExplainDMLPushdown</> and <function>EndDMLPushdown</>.
+ </para>
+
+ <para>
+ If the <function>BeginDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to execute the table update directly on
+ the remote server will fail with an error message.
+ </para>
+
+ <para>
+ <programlisting>
+ TupleTableSlot *
+ IterateDMLPushdown (ForeignScanState *node);
+ </programlisting>
+
+ When the <command>INSERT</>, <command>UPDATE</> or <command>DELETE</>
+ query doesn't have a <literal>RETURNING</> clause, just return NULL
+ after the actual table update directly executed on the remote server.
+ When the query has the clause, fetch one result containing the data
+ needed for the <literal>RETURNING</> calculation, returning it in a
+ tuple table slot (the node's <structfield>ScanTupleSlot</> should be
+ used for this purpose). The data that was actually inserted, updated
+ or deleted must be stored in the
+ <literal>es_result_relation_info->ri_projectReturning->pi_exprContext->ecxt_scantuple</>
+ of the node's <structname>EState</>.
+ Return NULL if no more rows are available.
+ Note that this is called in a short-lived memory context that will be
+ reset between invocations. Create a memory context in
+ <function>BeginDMLPushdown</> if you need longer-lived storage, or use
+ the <structfield>es_query_cxt</> of the node's <structname>EState</>.
+ </para>
+
+ <para>
+ The rows returned must match the <structfield>fdw_scan_tlist</> target
+ list if one was supplied, otherwise they must match the row type of the
+ foreign table being updated. If you choose to optimize away fetching
+ columns that are not needed for the <literal>RETURNING</> calculation,
+ you should insert nulls in those column positions, or else generate a
+ <structfield>fdw_scan_tlist</> list with those columns omitted.
+ </para>
+
+ <para>
+ Whether the query has the clause or not, the query's reported row count
+ must be incremented by the FDW itself. When the query doesn't has the
+ clause, the FDW must also increment the row count for the
+ <structname>ForeignScanState</> node in the <command>EXPLAIN ANALYZE</>
+ case.
+ </para>
+
+ <para>
+ If the <function>IterateDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to execute the table update directly on
+ the remote server will fail with an error message.
+ </para>
+
+ <para>
+ <programlisting>
+ void
+ EndDMLPushdown (ForeignScanState *node);
+ </programlisting>
+
+ End the table update and release resources. It is normally not important
+ to release palloc'd memory, but for example open files and connections
+ to remote servers should be cleaned up.
+ </para>
+
+ <para>
+ If the <function>EndDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to execute the table update directly on
+ the remote server will fail with an error message.
+ </para>
+
</sect2>
<sect2 id="fdw-callbacks-row-locking">
***************
*** 848,853 **** ExplainForeignModify (ModifyTableState *mtstate,
--- 998,1026 ----
<command>EXPLAIN</>.
</para>
+ <para>
+ <programlisting>
+ void
+ ExplainDMLPushdown (ForeignScanState *node,
+ ExplainState *es);
+ </programlisting>
+
+ Print additional <command>EXPLAIN</> output for a foreign table update
+ that is executed directly on the remote server.
+ This function can call <function>ExplainPropertyText</> and
+ related functions to add fields to the <command>EXPLAIN</> output.
+ The flag fields in <literal>es</> can be used to determine what to
+ print, and the state of the <structname>ForeignScanState</> node
+ can be inspected to provide run-time statistics in the <command>EXPLAIN
+ ANALYZE</> case.
+ </para>
+
+ <para>
+ If the <function>ExplainDMLPushdown</> pointer is set to
+ <literal>NULL</>, no additional information is printed during
+ <command>EXPLAIN</>.
+ </para>
+
</sect2>
<sect2 id="fdw-callbacks-analyze">
***************
*** 1068,1074 **** GetForeignServerByName(const char *name, bool missing_ok);
<para>
The FDW callback functions <function>GetForeignRelSize</>,
<function>GetForeignPaths</>, <function>GetForeignPlan</>,
! <function>PlanForeignModify</>, and <function>GetForeignJoinPaths</>
must fit into the workings of the <productname>PostgreSQL</> planner.
Here are some notes about what they must do.
</para>
--- 1241,1248 ----
<para>
The FDW callback functions <function>GetForeignRelSize</>,
<function>GetForeignPaths</>, <function>GetForeignPlan</>,
! <function>PlanForeignModify</>, <function>GetForeignJoinPaths</>, and
! <function>PlanDMLPushdown</>
must fit into the workings of the <productname>PostgreSQL</> planner.
Here are some notes about what they must do.
</para>
***************
*** 1228,1234 **** GetForeignServerByName(const char *name, bool missing_ok);
<para>
When planning an <command>UPDATE</> or <command>DELETE</>,
! <function>PlanForeignModify</> can look up the <structname>RelOptInfo</>
struct for the foreign table and make use of the
<literal>baserel->fdw_private</> data previously created by the
scan-planning functions. However, in <command>INSERT</> the target
--- 1402,1409 ----
<para>
When planning an <command>UPDATE</> or <command>DELETE</>,
! <function>PlanForeignModify</> and <function>PlanDMLPushdown</>
! can look up the <structname>RelOptInfo</>
struct for the foreign table and make use of the
<literal>baserel->fdw_private</> data previously created by the
scan-planning functions. However, in <command>INSERT</> the target
*** a/doc/src/sgml/postgres-fdw.sgml
--- b/doc/src/sgml/postgres-fdw.sgml
***************
*** 471,476 ****
--- 471,485 ----
extension that's listed in the foreign server's <literal>extensions</>
option. Operators and functions in such clauses must
be <literal>IMMUTABLE</> as well.
+ For an <command>UPDATE</> or <command>DELETE</> query,
+ <filename>postgres_fdw</> attempts to optimize the query execution by
+ sending the whole query to the remote server if there are no query
+ <literal>WHERE</> clauses that cannot be sent to the remote server,
+ no local joins for the query, and no row-level local <literal>BEFORE</> or
+ <literal>AFTER</> triggers on the target table. In <command>UPDATE</>,
+ expressions to assign to target columns must use only built-in data types,
+ <literal>IMMUTABLE</> operators, or <literal>IMMUTABLE</> functions,
+ to reduce the risk of misexecution of the query.
</para>
<para>
*** a/src/backend/commands/explain.c
--- b/src/backend/commands/explain.c
***************
*** 888,894 **** ExplainNode(PlanState *planstate, List *ancestors,
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
! pname = sname = "Foreign Scan";
break;
case T_CustomScan:
sname = "Custom Scan";
--- 888,916 ----
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
! sname = "Foreign Scan";
! switch (((ForeignScan *) plan)->operation)
! {
! case CMD_SELECT:
! pname = "Foreign Scan";
! operation = "Select";
! break;
! case CMD_INSERT:
! pname = "Foreign Insert";
! operation = "Insert";
! break;
! case CMD_UPDATE:
! pname = "Foreign Update";
! operation = "Update";
! break;
! case CMD_DELETE:
! pname = "Foreign Delete";
! operation = "Delete";
! break;
! default:
! pname = "???";
! break;
! }
break;
case T_CustomScan:
sname = "Custom Scan";
***************
*** 1636,1641 **** show_plan_tlist(PlanState *planstate, List *ancestors, ExplainState *es)
--- 1658,1669 ----
return;
if (IsA(plan, RecursiveUnion))
return;
+ /* Likewise for ForeignScan that has pushed down INSERT/UPDATE/DELETE */
+ if (IsA(plan, ForeignScan) &&
+ (((ForeignScan *) plan)->operation == CMD_INSERT ||
+ ((ForeignScan *) plan)->operation == CMD_UPDATE ||
+ ((ForeignScan *) plan)->operation == CMD_DELETE))
+ return;
/* Set up deparsing context */
context = set_deparse_context_planstate(es->deparse_cxt,
***************
*** 2224,2231 **** show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es)
FdwRoutine *fdwroutine = fsstate->fdwroutine;
/* Let the FDW emit whatever fields it wants */
! if (fdwroutine->ExplainForeignScan != NULL)
! fdwroutine->ExplainForeignScan(fsstate, es);
}
/*
--- 2252,2267 ----
FdwRoutine *fdwroutine = fsstate->fdwroutine;
/* Let the FDW emit whatever fields it wants */
! if (((ForeignScan *) fsstate->ss.ps.plan)->operation != CMD_SELECT)
! {
! if (fdwroutine->ExplainDMLPushdown != NULL)
! fdwroutine->ExplainDMLPushdown(fsstate, es);
! }
! else
! {
! if (fdwroutine->ExplainForeignScan != NULL)
! fdwroutine->ExplainForeignScan(fsstate, es);
! }
}
/*
***************
*** 2611,2618 **** show_modifytable_info(ModifyTableState *mtstate, List *ancestors,
}
}
! /* Give FDW a chance */
! if (fdwroutine && fdwroutine->ExplainForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, j);
--- 2647,2656 ----
}
}
! /* Give FDW a chance if needed */
! if (!resultRelInfo->ri_FdwPushdown &&
! fdwroutine != NULL &&
! fdwroutine->ExplainForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, j);
*** a/src/backend/executor/execMain.c
--- b/src/backend/executor/execMain.c
***************
*** 1011,1018 **** InitPlan(QueryDesc *queryDesc, int eflags)
* CheckValidRowMarkRel.
*/
void
! CheckValidResultRel(Relation resultRel, CmdType operation)
{
TriggerDesc *trigDesc = resultRel->trigdesc;
FdwRoutine *fdwroutine;
--- 1011,1019 ----
* CheckValidRowMarkRel.
*/
void
! CheckValidResultRel(ResultRelInfo *resultRelInfo, CmdType operation)
{
+ Relation resultRel = resultRelInfo->ri_RelationDesc;
TriggerDesc *trigDesc = resultRel->trigdesc;
FdwRoutine *fdwroutine;
***************
*** 1128,1133 **** CheckValidResultRel(Relation resultRel, CmdType operation)
--- 1129,1166 ----
elog(ERROR, "unrecognized CmdType: %d", (int) operation);
break;
}
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ if (fdwroutine->BeginDMLPushdown == NULL ||
+ fdwroutine->IterateDMLPushdown == NULL ||
+ fdwroutine->EndDMLPushdown == NULL)
+ {
+ switch (operation)
+ {
+ case CMD_INSERT:
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot push down insert on foreign table \"%s\"",
+ RelationGetRelationName(resultRel))));
+ break;
+ case CMD_UPDATE:
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot push down update on foreign table \"%s\"",
+ RelationGetRelationName(resultRel))));
+ break;
+ case CMD_DELETE:
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot push down delete on foreign table \"%s\"",
+ RelationGetRelationName(resultRel))));
+ break;
+ default:
+ elog(ERROR, "unrecognized CmdType: %d", (int) operation);
+ break;
+ }
+ }
+ }
break;
default:
ereport(ERROR,
***************
*** 1245,1250 **** InitResultRelInfo(ResultRelInfo *resultRelInfo,
--- 1278,1284 ----
else
resultRelInfo->ri_FdwRoutine = NULL;
resultRelInfo->ri_FdwState = NULL;
+ resultRelInfo->ri_FdwPushdown = false;
resultRelInfo->ri_ConstraintExprs = NULL;
resultRelInfo->ri_junkFilter = NULL;
resultRelInfo->ri_projectReturning = NULL;
*** a/src/backend/executor/nodeForeignscan.c
--- b/src/backend/executor/nodeForeignscan.c
***************
*** 48,54 **** ForeignNext(ForeignScanState *node)
/* Call the Iterate function in short-lived context */
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! slot = node->fdwroutine->IterateForeignScan(node);
MemoryContextSwitchTo(oldcontext);
/*
--- 48,57 ----
/* Call the Iterate function in short-lived context */
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! if (plan->operation != CMD_SELECT)
! slot = node->fdwroutine->IterateDMLPushdown(node);
! else
! slot = node->fdwroutine->IterateForeignScan(node);
MemoryContextSwitchTo(oldcontext);
/*
***************
*** 226,232 **** ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
/*
* Tell the FDW to initialize the scan.
*/
! fdwroutine->BeginForeignScan(scanstate, eflags);
return scanstate;
}
--- 229,238 ----
/*
* Tell the FDW to initialize the scan.
*/
! if (node->operation != CMD_SELECT)
! fdwroutine->BeginDMLPushdown(scanstate, eflags);
! else
! fdwroutine->BeginForeignScan(scanstate, eflags);
return scanstate;
}
***************
*** 240,247 **** ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
void
ExecEndForeignScan(ForeignScanState *node)
{
/* Let the FDW shut down */
! node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
--- 246,258 ----
void
ExecEndForeignScan(ForeignScanState *node)
{
+ ForeignScan *plan = (ForeignScan *) node->ss.ps.plan;
+
/* Let the FDW shut down */
! if (plan->operation != CMD_SELECT)
! node->fdwroutine->EndDMLPushdown(node);
! else
! node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
*** a/src/backend/executor/nodeModifyTable.c
--- b/src/backend/executor/nodeModifyTable.c
***************
*** 138,150 **** ExecCheckPlanOutput(Relation resultRel, List *targetList)
* tupleSlot: slot holding tuple actually inserted/updated/deleted
* planSlot: slot holding tuple returned by top subplan node
*
* Returns a slot holding the result tuple
*/
static TupleTableSlot *
! ExecProcessReturning(ProjectionInfo *projectReturning,
TupleTableSlot *tupleSlot,
TupleTableSlot *planSlot)
{
ExprContext *econtext = projectReturning->pi_exprContext;
/*
--- 138,154 ----
* tupleSlot: slot holding tuple actually inserted/updated/deleted
* planSlot: slot holding tuple returned by top subplan node
*
+ * Note: If tupleSlot is NULL, the FDW should have already provided econtext's
+ * scan tuple.
+ *
* Returns a slot holding the result tuple
*/
static TupleTableSlot *
! ExecProcessReturning(ResultRelInfo *resultRelInfo,
TupleTableSlot *tupleSlot,
TupleTableSlot *planSlot)
{
+ ProjectionInfo *projectReturning = resultRelInfo->ri_projectReturning;
ExprContext *econtext = projectReturning->pi_exprContext;
/*
***************
*** 154,160 **** ExecProcessReturning(ProjectionInfo *projectReturning,
ResetExprContext(econtext);
/* Make tuple and any needed join variables available to ExecProject */
! econtext->ecxt_scantuple = tupleSlot;
econtext->ecxt_outertuple = planSlot;
/* Compute the RETURNING expressions */
--- 158,177 ----
ResetExprContext(econtext);
/* Make tuple and any needed join variables available to ExecProject */
! if (tupleSlot)
! econtext->ecxt_scantuple = tupleSlot;
! else
! {
! HeapTuple tuple;
!
! /*
! * RETURNING expressions might reference the tableoid column, so
! * initialize t_tableOid before evaluating them.
! */
! Assert(!TupIsNull(econtext->ecxt_scantuple));
! tuple = ExecMaterializeSlot(econtext->ecxt_scantuple);
! tuple->t_tableOid = RelationGetRelid(resultRelInfo->ri_RelationDesc);
! }
econtext->ecxt_outertuple = planSlot;
/* Compute the RETURNING expressions */
***************
*** 496,503 **** ExecInsert(ModifyTableState *mtstate,
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo->ri_projectReturning,
! slot, planSlot);
return NULL;
}
--- 513,519 ----
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo, slot, planSlot);
return NULL;
}
***************
*** 738,745 **** ldelete:;
ExecStoreTuple(&deltuple, slot, InvalidBuffer, false);
}
! rslot = ExecProcessReturning(resultRelInfo->ri_projectReturning,
! slot, planSlot);
/*
* Before releasing the target tuple again, make sure rslot has a
--- 754,760 ----
ExecStoreTuple(&deltuple, slot, InvalidBuffer, false);
}
! rslot = ExecProcessReturning(resultRelInfo, slot, planSlot);
/*
* Before releasing the target tuple again, make sure rslot has a
***************
*** 1024,1031 **** lreplace:;
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo->ri_projectReturning,
! slot, planSlot);
return NULL;
}
--- 1039,1045 ----
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo, slot, planSlot);
return NULL;
}
***************
*** 1380,1385 **** ExecModifyTable(ModifyTableState *node)
--- 1394,1414 ----
break;
}
+ /*
+ * If ri_FdwPushdown is true, all we need to do here is compute the
+ * RETURNING expressions.
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ Assert(resultRelInfo->ri_projectReturning);
+
+ /* No need to provide scan tuple to ExecProcessReturning. */
+ slot = ExecProcessReturning(resultRelInfo, NULL, planSlot);
+
+ estate->es_result_relation_info = saved_resultRelInfo;
+ return slot;
+ }
+
EvalPlanQualSetSlot(&node->mt_epqstate, planSlot);
slot = planSlot;
***************
*** 1559,1568 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
{
subplan = (Plan *) lfirst(l);
/*
* Verify result relation is a valid target for the current operation
*/
! CheckValidResultRel(resultRelInfo->ri_RelationDesc, operation);
/*
* If there are indices on the result relation, open them and save
--- 1588,1600 ----
{
subplan = (Plan *) lfirst(l);
+ /* Initialize the FdwPushdown flag */
+ resultRelInfo->ri_FdwPushdown = list_nth_int(node->fdwPushdowns, i);
+
/*
* Verify result relation is a valid target for the current operation
*/
! CheckValidResultRel(resultRelInfo, operation);
/*
* If there are indices on the result relation, open them and save
***************
*** 1583,1589 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
/* Also let FDWs init themselves for foreign-table result rels */
! if (resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, i);
--- 1615,1622 ----
mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
/* Also let FDWs init themselves for foreign-table result rels */
! if (!resultRelInfo->ri_FdwPushdown &&
! resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, i);
***************
*** 1754,1766 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
--- 1787,1811 ----
erm = ExecFindRowMark(estate, rc->rti, false);
/* build ExecAuxRowMark for each subplan */
+ resultRelInfo = mtstate->resultRelInfo;
for (i = 0; i < nplans; i++)
{
ExecAuxRowMark *aerm;
+ /*
+ * ignore subplan if the FDW pushes down the command to the remote
+ * server
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ resultRelInfo++;
+ continue;
+ }
+
subplan = mtstate->mt_plans[i]->plan;
aerm = ExecBuildAuxRowMark(erm, subplan->targetlist);
mtstate->mt_arowmarks[i] = lappend(mtstate->mt_arowmarks[i], aerm);
+ resultRelInfo++;
}
}
***************
*** 1821,1826 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
--- 1866,1881 ----
ExecCheckPlanOutput(resultRelInfo->ri_RelationDesc,
subplan->targetlist);
+ /*
+ * ignore subplan if the FDW pushes down the command to the
+ * remote server
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ resultRelInfo++;
+ continue;
+ }
+
j = ExecInitJunkFilter(subplan->targetlist,
resultRelInfo->ri_RelationDesc->rd_att->tdhasoid,
ExecInitExtraTupleSlot(estate));
***************
*** 1910,1916 **** ExecEndModifyTable(ModifyTableState *node)
{
ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
! if (resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->EndForeignModify != NULL)
resultRelInfo->ri_FdwRoutine->EndForeignModify(node->ps.state,
resultRelInfo);
--- 1965,1972 ----
{
ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
! if (!resultRelInfo->ri_FdwPushdown &&
! resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->EndForeignModify != NULL)
resultRelInfo->ri_FdwRoutine->EndForeignModify(node->ps.state,
resultRelInfo);
*** a/src/backend/nodes/copyfuncs.c
--- b/src/backend/nodes/copyfuncs.c
***************
*** 187,192 **** _copyModifyTable(const ModifyTable *from)
--- 187,193 ----
COPY_NODE_FIELD(withCheckOptionLists);
COPY_NODE_FIELD(returningLists);
COPY_NODE_FIELD(fdwPrivLists);
+ COPY_NODE_FIELD(fdwPushdowns);
COPY_NODE_FIELD(rowMarks);
COPY_SCALAR_FIELD(epqParam);
COPY_SCALAR_FIELD(onConflictAction);
***************
*** 646,651 **** _copyForeignScan(const ForeignScan *from)
--- 647,653 ----
/*
* copy remainder of node
*/
+ COPY_SCALAR_FIELD(operation);
COPY_SCALAR_FIELD(fs_server);
COPY_NODE_FIELD(fdw_exprs);
COPY_NODE_FIELD(fdw_private);
*** a/src/backend/nodes/outfuncs.c
--- b/src/backend/nodes/outfuncs.c
***************
*** 341,346 **** _outModifyTable(StringInfo str, const ModifyTable *node)
--- 341,347 ----
WRITE_NODE_FIELD(withCheckOptionLists);
WRITE_NODE_FIELD(returningLists);
WRITE_NODE_FIELD(fdwPrivLists);
+ WRITE_NODE_FIELD(fdwPushdowns);
WRITE_NODE_FIELD(rowMarks);
WRITE_INT_FIELD(epqParam);
WRITE_ENUM_FIELD(onConflictAction, OnConflictAction);
***************
*** 592,597 **** _outForeignScan(StringInfo str, const ForeignScan *node)
--- 593,599 ----
_outScanInfo(str, (const Scan *) node);
+ WRITE_ENUM_FIELD(operation, CmdType);
WRITE_OID_FIELD(fs_server);
WRITE_NODE_FIELD(fdw_exprs);
WRITE_NODE_FIELD(fdw_private);
*** a/src/backend/nodes/readfuncs.c
--- b/src/backend/nodes/readfuncs.c
***************
*** 1472,1477 **** _readModifyTable(void)
--- 1472,1478 ----
READ_NODE_FIELD(withCheckOptionLists);
READ_NODE_FIELD(returningLists);
READ_NODE_FIELD(fdwPrivLists);
+ READ_NODE_FIELD(fdwPushdowns);
READ_NODE_FIELD(rowMarks);
READ_INT_FIELD(epqParam);
READ_ENUM_FIELD(onConflictAction, OnConflictAction);
*** a/src/backend/optimizer/plan/createplan.c
--- b/src/backend/optimizer/plan/createplan.c
***************
*** 3779,3784 **** make_foreignscan(List *qptlist,
--- 3779,3785 ----
plan->lefttree = outer_plan;
plan->righttree = NULL;
node->scan.scanrelid = scanrelid;
+ node->operation = CMD_SELECT;
/* fs_server will be filled in by create_foreignscan_plan */
node->fs_server = InvalidOid;
node->fdw_exprs = fdw_exprs;
***************
*** 5055,5060 **** make_modifytable(PlannerInfo *root,
--- 5056,5062 ----
Plan *plan = &node->plan;
double total_size;
List *fdw_private_list;
+ List *fdwpushdown_list;
ListCell *subnode;
ListCell *lc;
int i;
***************
*** 5135,5146 **** make_modifytable(PlannerInfo *root,
--- 5137,5150 ----
* construct private plan data, and accumulate it all into a list.
*/
fdw_private_list = NIL;
+ fdwpushdown_list = NIL;
i = 0;
foreach(lc, resultRelations)
{
Index rti = lfirst_int(lc);
FdwRoutine *fdwroutine;
List *fdw_private;
+ bool fdwpushdown;
/*
* If possible, we want to get the FdwRoutine from our RelOptInfo for
***************
*** 5167,5173 **** make_modifytable(PlannerInfo *root,
--- 5171,5190 ----
fdwroutine = NULL;
}
+ /*
+ * If the target relation has any row-level triggers, we can't push
+ * down the command to the remote server.
+ */
if (fdwroutine != NULL &&
+ fdwroutine->PlanDMLPushdown != NULL &&
+ !has_row_triggers(root, rti, operation))
+ fdwpushdown = fdwroutine->PlanDMLPushdown(root, node, rti, i);
+ else
+ fdwpushdown = false;
+ fdwpushdown_list = lappend_int(fdwpushdown_list, fdwpushdown);
+
+ if (!fdwpushdown &&
+ fdwroutine != NULL &&
fdwroutine->PlanForeignModify != NULL)
fdw_private = fdwroutine->PlanForeignModify(root, node, rti, i);
else
***************
*** 5176,5181 **** make_modifytable(PlannerInfo *root,
--- 5193,5199 ----
i++;
}
node->fdwPrivLists = fdw_private_list;
+ node->fdwPushdowns = fdwpushdown_list;
return node;
}
*** a/src/backend/optimizer/util/plancat.c
--- b/src/backend/optimizer/util/plancat.c
***************
*** 1520,1522 **** has_unique_index(RelOptInfo *rel, AttrNumber attno)
--- 1520,1569 ----
}
return false;
}
+
+
+ /*
+ * has_row_triggers
+ *
+ * Detect whether the specified relation has any row-level triggers for event.
+ */
+ bool
+ has_row_triggers(PlannerInfo *root, Index rti, CmdType event)
+ {
+ RangeTblEntry *rte = planner_rt_fetch(rti, root);
+ Relation relation;
+ TriggerDesc *trigDesc;
+ bool result = false;
+
+ /* Assume we already have adequate lock */
+ relation = heap_open(rte->relid, NoLock);
+
+ trigDesc = relation->trigdesc;
+ switch (event)
+ {
+ case CMD_INSERT:
+ if (trigDesc &&
+ (trigDesc->trig_insert_after_row ||
+ trigDesc->trig_insert_before_row))
+ result = true;
+ break;
+ case CMD_UPDATE:
+ if (trigDesc &&
+ (trigDesc->trig_update_after_row ||
+ trigDesc->trig_update_before_row))
+ result = true;
+ break;
+ case CMD_DELETE:
+ if (trigDesc &&
+ (trigDesc->trig_delete_after_row ||
+ trigDesc->trig_delete_before_row))
+ result = true;
+ break;
+ default:
+ elog(ERROR, "unrecognized CmdType: %d", (int) event);
+ break;
+ }
+
+ heap_close(relation, NoLock);
+ return result;
+ }
*** a/src/include/executor/executor.h
--- b/src/include/executor/executor.h
***************
*** 184,190 **** extern void ExecutorEnd(QueryDesc *queryDesc);
extern void standard_ExecutorEnd(QueryDesc *queryDesc);
extern void ExecutorRewind(QueryDesc *queryDesc);
extern bool ExecCheckRTPerms(List *rangeTable, bool ereport_on_violation);
! extern void CheckValidResultRel(Relation resultRel, CmdType operation);
extern void InitResultRelInfo(ResultRelInfo *resultRelInfo,
Relation resultRelationDesc,
Index resultRelationIndex,
--- 184,190 ----
extern void standard_ExecutorEnd(QueryDesc *queryDesc);
extern void ExecutorRewind(QueryDesc *queryDesc);
extern bool ExecCheckRTPerms(List *rangeTable, bool ereport_on_violation);
! extern void CheckValidResultRel(ResultRelInfo *resultRelInfo, CmdType operation);
extern void InitResultRelInfo(ResultRelInfo *resultRelInfo,
Relation resultRelationDesc,
Index resultRelationIndex,
*** a/src/include/foreign/fdwapi.h
--- b/src/include/foreign/fdwapi.h
***************
*** 93,98 **** typedef void (*EndForeignModify_function) (EState *estate,
--- 93,110 ----
typedef int (*IsForeignRelUpdatable_function) (Relation rel);
+ typedef bool (*PlanDMLPushdown_function) (PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+
+ typedef void (*BeginDMLPushdown_function) (ForeignScanState *node,
+ int eflags);
+
+ typedef TupleTableSlot *(*IterateDMLPushdown_function) (ForeignScanState *node);
+
+ typedef void (*EndDMLPushdown_function) (ForeignScanState *node);
+
typedef RowMarkType (*GetForeignRowMarkType_function) (RangeTblEntry *rte,
LockClauseStrength strength);
***************
*** 110,115 **** typedef void (*ExplainForeignModify_function) (ModifyTableState *mtstate,
--- 122,130 ----
int subplan_index,
struct ExplainState *es);
+ typedef void (*ExplainDMLPushdown_function) (ForeignScanState *node,
+ struct ExplainState *es);
+
typedef int (*AcquireSampleRowsFunc) (Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
***************
*** 162,167 **** typedef struct FdwRoutine
--- 177,186 ----
ExecForeignDelete_function ExecForeignDelete;
EndForeignModify_function EndForeignModify;
IsForeignRelUpdatable_function IsForeignRelUpdatable;
+ PlanDMLPushdown_function PlanDMLPushdown;
+ BeginDMLPushdown_function BeginDMLPushdown;
+ IterateDMLPushdown_function IterateDMLPushdown;
+ EndDMLPushdown_function EndDMLPushdown;
/* Functions for SELECT FOR UPDATE/SHARE row locking */
GetForeignRowMarkType_function GetForeignRowMarkType;
***************
*** 171,176 **** typedef struct FdwRoutine
--- 190,196 ----
/* Support functions for EXPLAIN */
ExplainForeignScan_function ExplainForeignScan;
ExplainForeignModify_function ExplainForeignModify;
+ ExplainDMLPushdown_function ExplainDMLPushdown;
/* Support functions for ANALYZE */
AnalyzeForeignTable_function AnalyzeForeignTable;
*** a/src/include/nodes/execnodes.h
--- b/src/include/nodes/execnodes.h
***************
*** 311,316 **** typedef struct JunkFilter
--- 311,317 ----
* TrigInstrument optional runtime measurements for triggers
* FdwRoutine FDW callback functions, if foreign table
* FdwState available to save private state of FDW
+ * FdwPushdown true when the command is pushed down
* WithCheckOptions list of WithCheckOption's to be checked
* WithCheckOptionExprs list of WithCheckOption expr states
* ConstraintExprs array of constraint-checking expr states
***************
*** 334,339 **** typedef struct ResultRelInfo
--- 335,341 ----
Instrumentation *ri_TrigInstrument;
struct FdwRoutine *ri_FdwRoutine;
void *ri_FdwState;
+ bool ri_FdwPushdown;
List *ri_WithCheckOptions;
List *ri_WithCheckOptionExprs;
List **ri_ConstraintExprs;
*** a/src/include/nodes/plannodes.h
--- b/src/include/nodes/plannodes.h
***************
*** 189,194 **** typedef struct ModifyTable
--- 189,195 ----
List *withCheckOptionLists; /* per-target-table WCO lists */
List *returningLists; /* per-target-table RETURNING tlists */
List *fdwPrivLists; /* per-target-table FDW private data lists */
+ List *fdwPushdowns; /* per-target-table FDW pushdown flags */
List *rowMarks; /* PlanRowMarks (non-locking only) */
int epqParam; /* ID of Param for EvalPlanQual re-eval */
OnConflictAction onConflictAction; /* ON CONFLICT action */
***************
*** 531,536 **** typedef struct WorkTableScan
--- 532,538 ----
typedef struct ForeignScan
{
Scan scan;
+ CmdType operation; /* SELECT/INSERT/UPDATE/DELETE */
Oid fs_server; /* OID of foreign server */
List *fdw_exprs; /* expressions that FDW may evaluate */
List *fdw_private; /* private data for FDW */
*** a/src/include/optimizer/plancat.h
--- b/src/include/optimizer/plancat.h
***************
*** 55,58 **** extern Selectivity join_selectivity(PlannerInfo *root,
--- 55,60 ----
JoinType jointype,
SpecialJoinInfo *sjinfo);
+ extern bool has_row_triggers(PlannerInfo *root, Index rti, CmdType event);
+
#endif /* PLANCAT_H */
On Wed, Feb 3, 2016 at 3:31 PM, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>
wrote:
On 2016/01/28 15:20, Rushabh Lathia wrote:
On Thu, Jan 28, 2016 at 11:33 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp <mailto:fujita.etsuro@lab.ntt.co.jp>> wrote:On 2016/01/27 21:23, Rushabh Lathia wrote:
If I understood correctly, above documentation means, that if
FDW have
DMLPushdown APIs that is enough. But in reality thats not the
case, we
need ExecForeignInsert, ExecForeignUpdate, or ExecForeignDelete
in case
DML is not pushable.And here fact is DMLPushdown APIs are optional for FDW, so that
if FDW
don't have DMLPushdown APIs they can still very well perform the
DML
operations using ExecForeignInsert, ExecForeignUpdate, or
ExecForeignDelete.So documentation should be like:
If the IsForeignRelUpdatable pointer is set to NULL, foreign
tables are
assumed to be insertable, updatable, or deletable if the FDW
provides
ExecForeignInsert, ExecForeignUpdate, or ExecForeignDelete
respectively,If FDW provides DMLPushdown APIs and the DML are pushable to the
foreign
server, then FDW still needs ExecForeignInsert,
ExecForeignUpdate, or
ExecForeignDelete for the non-pushable DML operation.What's your opinion ?
I agree that we should add this to the documentation, too.
I added docs to the IsForeignRelUpdatable documentation. Also, a brief
introductory remark has been added at the beginning of the DML pushdown
APIs' documentation.BTW, if I understand correctly, I think we should also modify
relation_is_updatabale() accordingly. Am I right?
Yep, we need to modify relation_is_updatable().
I thought I'd modify that function in the same way as
CheckValidResultRel(), but I noticed that we cannot do that, because we
don't have any information on whether each update is pushed down to the
remote server by PlanDMLPushdown, during relation_is_updatabale(). So, I
left that function as-is. relation_is_updatabale() is just used for
display in the information_schema views, so ISTM that that function is fine
as-is. (As for CheckValidResultRel(), I revised it so as to check the
presence of DML pushdown APIs after checking the existing APIs if the given
command will be pushed down. The reason is because we assume the presence
of the existing APIs, anyway.)I revised other docs and some comments, mostly for consistency.
I just started reviewing this and realized that patch is not getting applied
cleanly on latest source, it having some conflicts. Can you please upload
the correct version of patch.
Attached is an updated version of the patch, which has been created on top
of the updated version of the bugfix patch posted by Robert in [1]
(attached).Best regards,
Etsuro Fujita[1]
/messages/by-id/CA+TgmoZ40j2uC5aC1NXu03oj4CrVOLkS15XX+PTFP-1U-8zR1Q@mail.gmail.com
--
Rushabh Lathia
On Thu, Jan 28, 2016 at 7:36 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
Attached is that version of the patch.
I think that postgres_fdw might be able to insert a tableoid value in the
returned slot in e.g., postgresExecForeignInsert if AFTER ROW Triggers or
RETURNING expressions reference that value, but I didn't do anything about
that.
Thanks. I went with the earlier version, but split it into two commits.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/02/04 19:44, Rushabh Lathia wrote:
I just started reviewing this and realized that patch is not getting applied
cleanly on latest source, it having some conflicts. Can you please upload
the correct version of patch.
I rebased the patch to the latest HEAD. Attached is a rebased version
of the patch. You don't need to apply the patch
fdw-foreign-modify-rmh-v2.patch attached before.
Thanks for the review!
Best regards,
Etsuro Fujita
Attachments:
fdw-dml-pushdown-v6.patchapplication/x-patch; name=fdw-dml-pushdown-v6.patchDownload
*** a/contrib/postgres_fdw/deparse.c
--- b/contrib/postgres_fdw/deparse.c
***************
*** 1069,1074 **** deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 1069,1134 ----
}
/*
+ * deparse remote UPDATE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ deparse_expr_cxt context;
+ int nestlevel;
+ bool first;
+ ListCell *lc;
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = params_list;
+
+ appendStringInfoString(buf, "UPDATE ");
+ deparseRelation(buf, rel);
+ appendStringInfoString(buf, " SET ");
+
+ /* Make sure any constants in the exprs are printed portably */
+ nestlevel = set_transmission_modes();
+
+ first = true;
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(targetlist, attnum);
+
+ if (!first)
+ appendStringInfoString(buf, ", ");
+ first = false;
+
+ deparseColumnRef(buf, rtindex, attnum, root);
+ appendStringInfoString(buf, " = ");
+ deparseExpr((Expr *) tle->expr, &context);
+ }
+
+ reset_transmission_modes(nestlevel);
+
+ if (remote_conds)
+ appendWhereClause(remote_conds, &context);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* deparse remote DELETE statement
*
* The statement text is appended to buf, and we also create an integer List
***************
*** 1091,1096 **** deparseDeleteSql(StringInfo buf, PlannerInfo *root,
--- 1151,1190 ----
}
/*
+ * deparse remote DELETE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ deparse_expr_cxt context;
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = params_list;
+
+ appendStringInfoString(buf, "DELETE FROM ");
+ deparseRelation(buf, rel);
+
+ if (remote_conds)
+ appendWhereClause(remote_conds, &context);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* Add a RETURNING clause, if needed, to an INSERT/UPDATE/DELETE.
*/
static void
*** a/contrib/postgres_fdw/expected/postgres_fdw.out
--- b/contrib/postgres_fdw/expected/postgres_fdw.out
***************
*** 1314,1320 **** INSERT INTO ft2 (c1,c2,c3)
--- 1314,1339 ----
(3 rows)
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 300), c3 = (c3 || '_update3'::text) WHERE ((("C 1" % 10) = 3))
+ (3 rows)
+
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Output: c1, c2, c3, c4, c5, c6, c7, c8
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 400), c3 = (c3 || '_update7'::text) WHERE ((("C 1" % 10) = 7)) RETURNING "C 1", c2, c3, c4, c5, c6, c7, c8
+ (4 rows)
+
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
c1 | c2 | c3 | c4 | c5 | c6 | c7 | c8
------+-----+--------------------+------------------------------+--------------------------+----+------------+-----
***************
*** 1424,1430 **** UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
--- 1443,1449 ----
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
***************
*** 1445,1460 **** UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
! QUERY PLAN
! ----------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1 RETURNING "C 1", c4
! -> Foreign Scan on public.ft2
! Output: ctid
! Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) FOR UPDATE
! (6 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
--- 1464,1477 ----
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
! QUERY PLAN
! --------------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
! -> Foreign Delete on public.ft2
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) RETURNING "C 1", c4
! (4 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
***************
*** 1565,1571 **** DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
--- 1582,1588 ----
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
***************
*** 2426,2441 **** INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
(1 row)
EXPLAIN (verbose, costs off)
! UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
! QUERY PLAN
! -------------------------------------------------------------------------------------------------------------------
Update on public.ft2
Output: (tableoid)::regclass
! Remote SQL: UPDATE "S 1"."T 1" SET c3 = $2 WHERE ctid = $1
! -> Foreign Scan on public.ft2
! Output: c1, c2, NULL::integer, 'bar'::text, c4, c5, c6, c7, c8, ctid
! Remote SQL: SELECT "C 1", c2, c4, c5, c6, c7, c8, ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
! (6 rows)
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
--- 2443,2456 ----
(1 row)
EXPLAIN (verbose, costs off)
! UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
! QUERY PLAN
! ------------------------------------------------------------------------------------
Update on public.ft2
Output: (tableoid)::regclass
! -> Foreign Update on public.ft2
! Remote SQL: UPDATE "S 1"."T 1" SET c3 = 'bar'::text WHERE (("C 1" = 9999))
! (4 rows)
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
***************
*** 2444,2459 **** UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
(1 row)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
! QUERY PLAN
! ------------------------------------------------------------------------------------
Delete on public.ft2
Output: (tableoid)::regclass
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1
! -> Foreign Scan on public.ft2
! Output: ctid
! Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
! (6 rows)
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
--- 2459,2472 ----
(1 row)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
! QUERY PLAN
! --------------------------------------------------------------------
Delete on public.ft2
Output: (tableoid)::regclass
! -> Foreign Delete on public.ft2
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE (("C 1" = 9999))
! (4 rows)
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
***************
*** 2607,2613 **** CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 2620,2626 ----
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 2766,2772 **** savepoint s3;
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 2779,2785 ----
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (-2) WHERE ((c2 = 42)) AND (("C 1" = 10))
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 2906,2912 **** CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2positive;
-- But inconsistent check constraints provide inconsistent results
ALTER FOREIGN TABLE ft1 ADD CONSTRAINT ft1_c2negative CHECK (c2 < 0);
--- 2919,2925 ----
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2positive;
-- But inconsistent check constraints provide inconsistent results
ALTER FOREIGN TABLE ft1 ADD CONSTRAINT ft1_c2negative CHECK (c2 < 0);
***************
*** 3299,3304 **** NOTICE: NEW: (13,"test triggered !")
--- 3312,3510 ----
(0,27)
(1 row)
+ -- cleanup
+ DROP TRIGGER trig_row_before ON rem1;
+ DROP TRIGGER trig_row_after ON rem1;
+ DROP TRIGGER trig_local_before ON loc1;
+ -- Test DML pushdown functionality
+ -- Test with statement-level triggers
+ CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_stmt_before ON rem1;
+ CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_stmt_after ON rem1;
+ -- Test with row-level ON INSERT triggers
+ CREATE TRIGGER trig_row_before_insert
+ BEFORE INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_before_insert ON rem1;
+ CREATE TRIGGER trig_row_after_insert
+ AFTER INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_after_insert ON rem1;
+ -- Test with row-level ON UPDATE triggers
+ CREATE TRIGGER trig_row_before_update
+ BEFORE UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_before_update ON rem1;
+ CREATE TRIGGER trig_row_after_update
+ AFTER UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+ -------------------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_after_update ON rem1;
+ -- Test with row-level ON DELETE triggers
+ CREATE TRIGGER trig_row_before_delete
+ BEFORE DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ DROP TRIGGER trig_row_before_delete ON rem1;
+ CREATE TRIGGER trig_row_after_delete
+ AFTER DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+ ------------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ DROP TRIGGER trig_row_after_delete ON rem1;
-- ===================================================================
-- test inheritance features
-- ===================================================================
***************
*** 3768,3773 **** fetch from c;
--- 3974,4029 ----
update bar set f2 = null where current of c;
ERROR: WHERE CURRENT OF is not supported for this table type
rollback;
+ explain (verbose, costs off)
+ delete from foo where f1 < 5 returning *;
+ QUERY PLAN
+ --------------------------------------------------------------------------------
+ Delete on public.foo
+ Output: foo.f1, foo.f2
+ Delete on public.foo
+ Foreign Delete on public.foo2
+ -> Index Scan using i_foo_f1 on public.foo
+ Output: foo.ctid
+ Index Cond: (foo.f1 < 5)
+ -> Foreign Delete on public.foo2
+ Remote SQL: DELETE FROM public.loct1 WHERE ((f1 < 5)) RETURNING f1, f2
+ (9 rows)
+
+ delete from foo where f1 < 5 returning *;
+ f1 | f2
+ ----+----
+ 1 | 1
+ 3 | 3
+ 0 | 0
+ 2 | 2
+ 4 | 4
+ (5 rows)
+
+ explain (verbose, costs off)
+ update bar set f2 = f2 + 100 returning *;
+ QUERY PLAN
+ ------------------------------------------------------------------------------
+ Update on public.bar
+ Output: bar.f1, bar.f2
+ Update on public.bar
+ Foreign Update on public.bar2
+ -> Seq Scan on public.bar
+ Output: bar.f1, (bar.f2 + 100), bar.ctid
+ -> Foreign Update on public.bar2
+ Remote SQL: UPDATE public.loct2 SET f2 = (f2 + 100) RETURNING f1, f2
+ (8 rows)
+
+ update bar set f2 = f2 + 100 returning *;
+ f1 | f2
+ ----+-----
+ 1 | 311
+ 2 | 322
+ 6 | 266
+ 3 | 333
+ 4 | 344
+ 7 | 277
+ (6 rows)
+
drop table foo cascade;
NOTICE: drop cascades to foreign table foo2
drop table bar cascade;
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
***************
*** 61,66 **** enum FdwScanPrivateIndex
--- 61,68 ----
{
/* SQL statement to execute remotely (as a String node) */
FdwScanPrivateSelectSql,
+ /* List of restriction clauses that can be executed remotely */
+ FdwScanPrivateRemoteConds,
/* Integer list of attribute numbers retrieved by the SELECT */
FdwScanPrivateRetrievedAttrs,
/* Integer representing the desired fetch_size */
***************
*** 90,95 **** enum FdwModifyPrivateIndex
--- 92,119 ----
};
/*
+ * Similarly, this enum describes what's kept in the fdw_private list for
+ * a ForeignScan node that has pushed down an UPDATE/DELETE to the remote
+ * server. We store:
+ *
+ * 1) UPDATE/DELETE statement text to be sent to the remote server
+ * 2) Boolean flag showing if the remote query has a RETURNING clause
+ * 3) Integer list of attribute numbers retrieved by RETURNING, if any
+ * 4) Boolean flag showing if we set the command es_processed
+ */
+ enum FdwDmlPushdownPrivateIndex
+ {
+ /* SQL statement to execute remotely (as a String node) */
+ FdwDmlPushdownPrivateUpdateSql,
+ /* has-returning flag (as an integer Value node) */
+ FdwDmlPushdownPrivateHasReturning,
+ /* Integer list of attribute numbers retrieved by RETURNING */
+ FdwDmlPushdownPrivateRetrievedAttrs,
+ /* set-processed flag (as an integer Value node) */
+ FdwDmlPushdownPrivateSetProcessed
+ };
+
+ /*
* Execution state of a foreign scan using postgres_fdw.
*/
typedef struct PgFdwScanState
***************
*** 154,159 **** typedef struct PgFdwModifyState
--- 178,214 ----
} PgFdwModifyState;
/*
+ * Execution state of a foreign scan that has pushed down a foreign table
+ * modification to the remote server
+ */
+ typedef struct PgFdwDmlPushdownState
+ {
+ Relation rel; /* relcache entry for the foreign table */
+ AttInMetadata *attinmeta; /* attribute datatype conversion metadata */
+
+ /* extracted fdw_private data */
+ char *query; /* text of UPDATE/DELETE command */
+ bool has_returning; /* is there a RETURNING clause? */
+ List *retrieved_attrs; /* attr numbers retrieved by RETURNING */
+ bool set_processed; /* do we set the command es_processed? */
+
+ /* for remote query execution */
+ PGconn *conn; /* connection for the update */
+ int numParams; /* number of parameters passed to query */
+ FmgrInfo *param_flinfo; /* output conversion functions for them */
+ List *param_exprs; /* executable expressions for param values */
+ const char **param_values; /* textual values of query parameters */
+
+ /* for storing result tuples */
+ PGresult *result; /* result for query */
+ int num_tuples; /* # of result tuples */
+ int next_tuple; /* index of next one to return */
+
+ /* working memory context */
+ MemoryContext temp_cxt; /* context for per-tuple temporary data */
+ } PgFdwDmlPushdownState;
+
+ /*
* Workspace for analyzing a foreign table.
*/
typedef struct PgFdwAnalyzeState
***************
*** 245,250 **** static TupleTableSlot *postgresExecForeignDelete(EState *estate,
--- 300,312 ----
static void postgresEndForeignModify(EState *estate,
ResultRelInfo *resultRelInfo);
static int postgresIsForeignRelUpdatable(Relation rel);
+ static bool postgresPlanDMLPushdown(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+ static void postgresBeginDMLPushdown(ForeignScanState *node, int eflags);
+ static TupleTableSlot *postgresIterateDMLPushdown(ForeignScanState *node);
+ static void postgresEndDMLPushdown(ForeignScanState *node);
static void postgresExplainForeignScan(ForeignScanState *node,
ExplainState *es);
static void postgresExplainForeignModify(ModifyTableState *mtstate,
***************
*** 252,257 **** static void postgresExplainForeignModify(ModifyTableState *mtstate,
--- 314,321 ----
List *fdw_private,
int subplan_index,
ExplainState *es);
+ static void postgresExplainDMLPushdown(ForeignScanState *node,
+ ExplainState *es);
static bool postgresAnalyzeForeignTable(Relation relation,
AcquireSampleRowsFunc *func,
BlockNumber *totalpages);
***************
*** 288,293 **** static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
--- 352,359 ----
TupleTableSlot *slot);
static void store_returning_result(PgFdwModifyState *fmstate,
TupleTableSlot *slot, PGresult *res);
+ static void execute_dml_stmt(ForeignScanState *node);
+ static TupleTableSlot *get_returning_data(ForeignScanState *node);
static int postgresAcquireSampleRowsFunc(Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
***************
*** 330,339 **** postgres_fdw_handler(PG_FUNCTION_ARGS)
--- 396,410 ----
routine->ExecForeignDelete = postgresExecForeignDelete;
routine->EndForeignModify = postgresEndForeignModify;
routine->IsForeignRelUpdatable = postgresIsForeignRelUpdatable;
+ routine->PlanDMLPushdown = postgresPlanDMLPushdown;
+ routine->BeginDMLPushdown = postgresBeginDMLPushdown;
+ routine->IterateDMLPushdown = postgresIterateDMLPushdown;
+ routine->EndDMLPushdown = postgresEndDMLPushdown;
/* Support functions for EXPLAIN */
routine->ExplainForeignScan = postgresExplainForeignScan;
routine->ExplainForeignModify = postgresExplainForeignModify;
+ routine->ExplainDMLPushdown = postgresExplainDMLPushdown;
/* Support functions for ANALYZE */
routine->AnalyzeForeignTable = postgresAnalyzeForeignTable;
***************
*** 1010,1016 **** postgresGetForeignPlan(PlannerInfo *root,
* Build the fdw_private list that will be available to the executor.
* Items in the list must match enum FdwScanPrivateIndex, above.
*/
! fdw_private = list_make3(makeString(sql.data),
retrieved_attrs,
makeInteger(fpinfo->fetch_size));
--- 1081,1088 ----
* Build the fdw_private list that will be available to the executor.
* Items in the list must match enum FdwScanPrivateIndex, above.
*/
! fdw_private = list_make4(makeString(sql.data),
! remote_conds,
retrieved_attrs,
makeInteger(fpinfo->fetch_size));
***************
*** 1307,1319 **** postgresAddForeignUpdateTargets(Query *parsetree,
/*
* postgresPlanForeignModify
* Plan an insert/update/delete operation on a foreign table
- *
- * Note: currently, the plan tree generated for UPDATE/DELETE will always
- * include a ForeignScan that retrieves ctids (using SELECT FOR UPDATE)
- * and then the ModifyTable node will have to execute individual remote
- * UPDATE/DELETE commands. If there are no local conditions or joins
- * needed, it'd be better to let the scan node do UPDATE/DELETE RETURNING
- * and then do nothing at ModifyTable. Room for future optimization ...
*/
static List *
postgresPlanForeignModify(PlannerInfo *root,
--- 1379,1384 ----
***************
*** 1824,1829 **** postgresIsForeignRelUpdatable(Relation rel)
--- 1889,2225 ----
}
/*
+ * postgresPlanDMLPushdown
+ * Consider pushing down a foreign table modification to the remote server
+ *
+ * Decide whether the table modification is safe to push down to the remote end,
+ * and if so, modify subplan so as to do that.
+ *
+ * Conditions checked here:
+ *
+ * 1. The table modification must be an UPDATE or DELETE.
+ *
+ * 2. It's unsafe to push down the command if there are any local joins needed.
+ *
+ * 3. It's unsafe to push down the command if there are any quals that can't be
+ * evaluated remotely.
+ *
+ * 4. We can't push down an UPDATE, if any expressions to assign to the target
+ * columns are unsafe to evaluate on the remote end.
+ */
+ static bool
+ postgresPlanDMLPushdown(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index)
+ {
+ CmdType operation = plan->operation;
+ Plan *subplan = (Plan *) list_nth(plan->plans, subplan_index);
+ RangeTblEntry *rte = planner_rt_fetch(resultRelation, root);
+ Relation rel;
+ StringInfoData sql;
+ ForeignScan *fscan;
+ List *targetAttrs = NIL;
+ List *remote_conds;
+ List *params_list = NIL;
+ List *returningList = NIL;
+ List *retrieved_attrs = NIL;
+
+ /*
+ * Decide whether the table modification is safe to push down to the remote
+ * server.
+ */
+
+ /* Check point 1 */
+ if (operation == CMD_INSERT)
+ return false;
+
+ /* Check point 2 */
+ if (nodeTag(subplan) != T_ForeignScan)
+ return false;
+
+ /* Check point 3 */
+ if (subplan->qual != NIL)
+ return false;
+
+ /* Check point 4 */
+ if (operation == CMD_UPDATE)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[resultRelation];
+ int col;
+
+ /*
+ * We transmit only columns that were explicitly targets of the UPDATE,
+ * so as to avoid unnecessary data transmission.
+ */
+ col = -1;
+ while ((col = bms_next_member(rte->updatedCols, col)) >= 0)
+ {
+ /* bit numbers are offset by FirstLowInvalidHeapAttributeNumber */
+ AttrNumber attno = col + FirstLowInvalidHeapAttributeNumber;
+ TargetEntry *tle;
+
+ if (attno <= InvalidAttrNumber) /* shouldn't happen */
+ elog(ERROR, "system-column update is not supported");
+
+ tle = get_tle_by_resno(subplan->targetlist, attno);
+
+ if (!is_foreign_expr(root, baserel, (Expr *) tle->expr))
+ return false;
+
+ targetAttrs = lappend_int(targetAttrs, attno);
+ }
+ }
+
+ /*
+ * Ok, modify subplan so as to push down the command to the remote server.
+ */
+ fscan = (ForeignScan *) subplan;
+
+ initStringInfo(&sql);
+
+ /*
+ * Core code already has some lock on each rel being planned, so we can
+ * use NoLock here.
+ */
+ rel = heap_open(rte->relid, NoLock);
+
+ /*
+ * Extract the baserestrictinfo clauses that can be evaluated remotely.
+ */
+ remote_conds = (List *) list_nth(fscan->fdw_private,
+ FdwScanPrivateRemoteConds);
+
+ /*
+ * Extract the relevant RETURNING list if any.
+ */
+ if (plan->returningLists)
+ returningList = (List *) list_nth(plan->returningLists, subplan_index);
+
+ /*
+ * Construct the SQL command string.
+ */
+ switch (operation)
+ {
+ case CMD_UPDATE:
+ deparsePushedDownUpdateSql(&sql, root, resultRelation, rel,
+ ((Plan *) fscan)->targetlist,
+ targetAttrs,
+ remote_conds, ¶ms_list,
+ returningList, &retrieved_attrs);
+ break;
+ case CMD_DELETE:
+ deparsePushedDownDeleteSql(&sql, root, resultRelation, rel,
+ remote_conds, ¶ms_list,
+ returningList, &retrieved_attrs);
+ break;
+ default:
+ elog(ERROR, "unexpected operation: %d", (int) operation);
+ break;
+ }
+
+ /*
+ * Update the operation info.
+ */
+ fscan->operation = operation;
+
+ /*
+ * Update the fdw_exprs list that will be available to the executor.
+ */
+ fscan->fdw_exprs = params_list;
+
+ /*
+ * Update the fdw_private list that will be available to the executor.
+ * Items in the list must match enum FdwDmlPushdownPrivateIndex, above.
+ */
+ fscan->fdw_private = list_make4(makeString(sql.data),
+ makeInteger((retrieved_attrs != NIL)),
+ retrieved_attrs,
+ makeInteger(plan->canSetTag));
+
+ heap_close(rel, NoLock);
+ return true;
+ }
+
+ /*
+ * postgresBeginDMLPushdown
+ * Initiate pushing down a foreign table modification to the remote server
+ */
+ static void
+ postgresBeginDMLPushdown(ForeignScanState *node, int eflags)
+ {
+ ForeignScan *fsplan = (ForeignScan *) node->ss.ps.plan;
+ EState *estate = node->ss.ps.state;
+ PgFdwDmlPushdownState *dpstate;
+ RangeTblEntry *rte;
+ Oid userid;
+ ForeignTable *table;
+ UserMapping *user;
+ int numParams;
+ int i;
+ ListCell *lc;
+
+ /*
+ * Do nothing in EXPLAIN (no ANALYZE) case. node->fdw_state stays NULL.
+ */
+ if (eflags & EXEC_FLAG_EXPLAIN_ONLY)
+ return;
+
+ /*
+ * We'll save private state in node->fdw_state.
+ */
+ dpstate = (PgFdwDmlPushdownState *) palloc0(sizeof(PgFdwDmlPushdownState));
+ node->fdw_state = (void *) dpstate;
+
+ /*
+ * Identify which user to do the remote access as. This should match what
+ * ExecCheckRTEPerms() does.
+ */
+ rte = rt_fetch(fsplan->scan.scanrelid, estate->es_range_table);
+ userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
+
+ /* Get info about foreign table. */
+ dpstate->rel = node->ss.ss_currentRelation;
+ table = GetForeignTable(RelationGetRelid(dpstate->rel));
+ user = GetUserMapping(userid, table->serverid);
+
+ /*
+ * Get connection to the foreign server. Connection manager will
+ * establish new connection if necessary.
+ */
+ dpstate->conn = GetConnection(user, false);
+
+ /* Initialize state variable */
+ dpstate->num_tuples = -1; /* -1 means not set yet */
+
+ /* Get private info created by planner functions. */
+ dpstate->query = strVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateUpdateSql));
+ dpstate->has_returning = intVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateHasReturning));
+ dpstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateRetrievedAttrs);
+ dpstate->set_processed = intVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateSetProcessed));
+
+ /* Create context for per-tuple temp workspace. */
+ dpstate->temp_cxt = AllocSetContextCreate(estate->es_query_cxt,
+ "postgres_fdw temporary data",
+ ALLOCSET_SMALL_MINSIZE,
+ ALLOCSET_SMALL_INITSIZE,
+ ALLOCSET_SMALL_MAXSIZE);
+
+ /* Prepare for input conversion of RETURNING results. */
+ if (dpstate->has_returning)
+ dpstate->attinmeta = TupleDescGetAttInMetadata(RelationGetDescr(dpstate->rel));
+
+ /* Prepare for output conversion of parameters used in remote query. */
+ numParams = list_length(fsplan->fdw_exprs);
+ dpstate->numParams = numParams;
+ dpstate->param_flinfo = (FmgrInfo *) palloc0(sizeof(FmgrInfo) * numParams);
+
+ i = 0;
+ foreach(lc, fsplan->fdw_exprs)
+ {
+ Node *param_expr = (Node *) lfirst(lc);
+ Oid typefnoid;
+ bool isvarlena;
+
+ getTypeOutputInfo(exprType(param_expr), &typefnoid, &isvarlena);
+ fmgr_info(typefnoid, &dpstate->param_flinfo[i]);
+ i++;
+ }
+
+ /*
+ * Prepare remote-parameter expressions for evaluation. (Note: in
+ * practice, we expect that all these expressions will be just Params, so
+ * we could possibly do something more efficient than using the full
+ * expression-eval machinery for this. But probably there would be little
+ * benefit, and it'd require postgres_fdw to know more than is desirable
+ * about Param evaluation.)
+ */
+ dpstate->param_exprs = (List *)
+ ExecInitExpr((Expr *) fsplan->fdw_exprs,
+ (PlanState *) node);
+
+ /*
+ * Allocate buffer for text form of query parameters, if any.
+ */
+ if (numParams > 0)
+ dpstate->param_values = (const char **) palloc0(numParams * sizeof(char *));
+ else
+ dpstate->param_values = NULL;
+ }
+
+ /*
+ * postgresIterateDMLPushdown
+ * Execute pushing down a foreign table modification to the remote server
+ */
+ static TupleTableSlot *
+ postgresIterateDMLPushdown(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ EState *estate = node->ss.ps.state;
+ ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
+
+ /*
+ * If this is the first call after Begin, execute the statement.
+ */
+ if (dpstate->num_tuples == -1)
+ execute_dml_stmt(node);
+
+ /*
+ * If the local query doesn't specify RETURNING, just clear tuple slot.
+ */
+ if (!resultRelInfo->ri_projectReturning)
+ {
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+ Instrumentation *instr = node->ss.ps.instrument;
+
+ Assert(!dpstate->has_returning);
+
+ /* Increment the command es_processed count if necessary. */
+ if (dpstate->set_processed)
+ estate->es_processed += dpstate->num_tuples;
+
+ /* Increment the tuple count for EXPLAIN ANALYZE if necessary. */
+ if (instr)
+ instr->tuplecount += dpstate->num_tuples;
+
+ return ExecClearTuple(slot);
+ }
+
+ /*
+ * Get the next RETURNING tuple.
+ */
+ return get_returning_data(node);
+ }
+
+ /*
+ * postgresEndDMLPushdown
+ * Finish pushing down a foreign table modification to the remote server
+ */
+ static void
+ postgresEndDMLPushdown(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+
+ /* if dpstate is NULL, we are in EXPLAIN; nothing to do */
+ if (dpstate == NULL)
+ return;
+
+ /* Release PGresult */
+ if (dpstate->result)
+ PQclear(dpstate->result);
+
+ /* Release remote connection */
+ ReleaseConnection(dpstate->conn);
+ dpstate->conn = NULL;
+
+ /* MemoryContext will be deleted automatically. */
+ }
+
+ /*
* postgresExplainForeignScan
* Produce extra output for EXPLAIN of a ForeignScan on a foreign table
*/
***************
*** 1861,1866 **** postgresExplainForeignModify(ModifyTableState *mtstate,
--- 2257,2281 ----
}
}
+ /*
+ * postgresExplainDMLPushdown
+ * Produce extra output for EXPLAIN of a ForeignScan on a foreign table
+ * that has pushed down an UPDATE/DELETE to the remote server
+ */
+ static void
+ postgresExplainDMLPushdown(ForeignScanState *node, ExplainState *es)
+ {
+ List *fdw_private;
+ char *sql;
+
+ if (es->verbose)
+ {
+ fdw_private = ((ForeignScan *) node->ss.ps.plan)->fdw_private;
+ sql = strVal(list_nth(fdw_private, FdwDmlPushdownPrivateUpdateSql));
+ ExplainPropertyText("Remote SQL", sql, es);
+ }
+ }
+
/*
* estimate_path_cost_size
***************
*** 2473,2478 **** store_returning_result(PgFdwModifyState *fmstate,
--- 2888,3026 ----
}
/*
+ * Execute a pushed-down UPDATE/DELETE statement.
+ */
+ static void
+ execute_dml_stmt(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ ExprContext *econtext = node->ss.ps.ps_ExprContext;
+ int numParams = dpstate->numParams;
+ const char **values = dpstate->param_values;
+
+ /*
+ * Construct array of query parameter values in text format.
+ */
+ if (numParams > 0)
+ {
+ int nestlevel;
+ int i;
+ ListCell *lc;
+
+ nestlevel = set_transmission_modes();
+
+ i = 0;
+ foreach(lc, dpstate->param_exprs)
+ {
+ ExprState *expr_state = (ExprState *) lfirst(lc);
+ Datum expr_value;
+ bool isNull;
+
+ /* Evaluate the parameter expression */
+ expr_value = ExecEvalExpr(expr_state, econtext, &isNull, NULL);
+
+ /*
+ * Get string representation of each parameter value by invoking
+ * type-specific output function, unless the value is null.
+ */
+ if (isNull)
+ values[i] = NULL;
+ else
+ values[i] = OutputFunctionCall(&dpstate->param_flinfo[i],
+ expr_value);
+ i++;
+ }
+
+ reset_transmission_modes(nestlevel);
+ }
+
+ /*
+ * Notice that we pass NULL for paramTypes, thus forcing the remote server
+ * to infer types for all parameters. Since we explicitly cast every
+ * parameter (see deparse.c), the "inference" is trivial and will produce
+ * the desired result. This allows us to avoid assuming that the remote
+ * server has the same OIDs we do for the parameters' types.
+ *
+ * We don't use a PG_TRY block here, so be careful not to throw error
+ * without releasing the PGresult.
+ */
+ dpstate->result = PQexecParams(dpstate->conn, dpstate->query,
+ numParams, NULL, values, NULL, NULL, 0);
+ if (PQresultStatus(dpstate->result) !=
+ (dpstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
+ pgfdw_report_error(ERROR, dpstate->result, dpstate->conn, true,
+ dpstate->query);
+
+ /* Get the number of rows affected. */
+ if (dpstate->has_returning)
+ dpstate->num_tuples = PQntuples(dpstate->result);
+ else
+ dpstate->num_tuples = atoi(PQcmdTuples(dpstate->result));
+ }
+
+ /*
+ * Get the result of a RETURNING clause.
+ */
+ static TupleTableSlot *
+ get_returning_data(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ EState *estate = node->ss.ps.state;
+ ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+
+ Assert(resultRelInfo->ri_projectReturning);
+
+ /* If we didn't get any tuples, must be end of data. */
+ if (dpstate->next_tuple >= dpstate->num_tuples)
+ return ExecClearTuple(slot);
+
+ /* Increment the command es_processed count if necessary. */
+ if (dpstate->set_processed)
+ estate->es_processed += 1;
+
+ /*
+ * Store a RETURNING tuple. If has_returning is false, just emit a dummy
+ * tuple. (We have has_returning=false if the local query is of the form
+ * UPDATE/DELETE .. RETURNING 1 for example.)
+ */
+ if (!dpstate->has_returning)
+ ExecStoreAllNullTuple(slot);
+ else
+ {
+ /*
+ * On error, be sure to release the PGresult on the way out. Callers
+ * do not have PG_TRY blocks to ensure this happens.
+ */
+ PG_TRY();
+ {
+ HeapTuple newtup;
+
+ newtup = make_tuple_from_result_row(dpstate->result,
+ dpstate->next_tuple,
+ dpstate->rel,
+ dpstate->attinmeta,
+ dpstate->retrieved_attrs,
+ dpstate->temp_cxt);
+ ExecStoreTuple(newtup, slot, InvalidBuffer, false);
+ }
+ PG_CATCH();
+ {
+ if (dpstate->result)
+ PQclear(dpstate->result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ }
+ dpstate->next_tuple++;
+
+ /* Make slot available for evaluation of the local query RETURNING list. */
+ resultRelInfo->ri_projectReturning->pi_exprContext->ecxt_scantuple = slot;
+
+ return slot;
+ }
+
+ /*
* postgresAnalyzeForeignTable
* Test whether analyzing this foreign table is supported
*/
*** a/contrib/postgres_fdw/postgres_fdw.h
--- b/contrib/postgres_fdw/postgres_fdw.h
***************
*** 93,102 **** extern void deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 93,116 ----
Index rtindex, Relation rel,
List *targetAttrs, List *returningList,
List **retrieved_attrs);
+ extern void deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseDeleteSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *returningList,
List **retrieved_attrs);
+ extern void deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseAnalyzeSizeSql(StringInfo buf, Relation rel);
extern void deparseAnalyzeSql(StringInfo buf, Relation rel,
List **retrieved_attrs);
*** a/contrib/postgres_fdw/sql/postgres_fdw.sql
--- b/contrib/postgres_fdw/sql/postgres_fdw.sql
***************
*** 399,426 **** INSERT INTO ft2 (c1,c2,c3) SELECT c1+1000,c2+100, c3 || c3 FROM ft2 LIMIT 20;
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
EXPLAIN (verbose, costs off)
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
! UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
-- Test that trigger on remote table works as expected
--- 399,430 ----
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
EXPLAIN (verbose, costs off)
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
! UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
-- Test that trigger on remote table works as expected
***************
*** 737,742 **** UPDATE rem1 SET f2 = 'testo';
--- 741,830 ----
-- Test returning a system attribute
INSERT INTO rem1(f2) VALUES ('test') RETURNING ctid;
+ -- cleanup
+ DROP TRIGGER trig_row_before ON rem1;
+ DROP TRIGGER trig_row_after ON rem1;
+ DROP TRIGGER trig_local_before ON loc1;
+
+
+ -- Test DML pushdown functionality
+
+ -- Test with statement-level triggers
+ CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_stmt_before ON rem1;
+
+ CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_stmt_after ON rem1;
+
+ -- Test with row-level ON INSERT triggers
+ CREATE TRIGGER trig_row_before_insert
+ BEFORE INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_before_insert ON rem1;
+
+ CREATE TRIGGER trig_row_after_insert
+ AFTER INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_after_insert ON rem1;
+
+ -- Test with row-level ON UPDATE triggers
+ CREATE TRIGGER trig_row_before_update
+ BEFORE UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_before_update ON rem1;
+
+ CREATE TRIGGER trig_row_after_update
+ AFTER UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_after_update ON rem1;
+
+ -- Test with row-level ON DELETE triggers
+ CREATE TRIGGER trig_row_before_delete
+ BEFORE DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ DROP TRIGGER trig_row_before_delete ON rem1;
+
+ CREATE TRIGGER trig_row_after_delete
+ AFTER DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ DROP TRIGGER trig_row_after_delete ON rem1;
+
-- ===================================================================
-- test inheritance features
-- ===================================================================
***************
*** 868,873 **** fetch from c;
--- 956,968 ----
update bar set f2 = null where current of c;
rollback;
+ explain (verbose, costs off)
+ delete from foo where f1 < 5 returning *;
+ delete from foo where f1 < 5 returning *;
+ explain (verbose, costs off)
+ update bar set f2 = f2 + 100 returning *;
+ update bar set f2 = f2 + 100 returning *;
+
drop table foo cascade;
drop table bar cascade;
drop table loct1;
*** a/doc/src/sgml/fdwhandler.sgml
--- b/doc/src/sgml/fdwhandler.sgml
***************
*** 672,684 **** IsForeignRelUpdatable (Relation rel);
<literal>NULL</>, foreign tables are assumed to be insertable, updatable,
or deletable if the FDW provides <function>ExecForeignInsert</>,
<function>ExecForeignUpdate</>, or <function>ExecForeignDelete</>
! respectively. This function is only needed if the FDW supports some
tables that are updatable and some that are not. (Even then, it's
permissible to throw an error in the execution routine instead of
checking in this function. However, this function is used to determine
updatability for display in the <literal>information_schema</> views.)
</para>
</sect2>
<sect2 id="fdw-callbacks-row-locking">
--- 672,833 ----
<literal>NULL</>, foreign tables are assumed to be insertable, updatable,
or deletable if the FDW provides <function>ExecForeignInsert</>,
<function>ExecForeignUpdate</>, or <function>ExecForeignDelete</>
! respectively.
! (If the FDW optimizes a foreign table update on a foreign table using
! <function>PlanDMLPushdown</>, it still needs to provide
! <function>BeginDMLPushdown</>, <function>IterateDMLPushdown</> and
! <function>EndDMLPushdown</> to execute the optimized update.)
! This function is only needed if the FDW supports some
tables that are updatable and some that are not. (Even then, it's
permissible to throw an error in the execution routine instead of
checking in this function. However, this function is used to determine
updatability for display in the <literal>information_schema</> views.)
</para>
+ <para>
+ If an FDW supports an optimization that executes a foreign table update
+ directly on the remote server, it should provide the following callback
+ functions in addition to the table-updating functions described above:
+ </para>
+
+ <para>
+ <programlisting>
+ bool
+ PlanDMLPushdown (PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+ </programlisting>
+
+ Decide whether it is safe to execute a foreign table update directly
+ on the remote server. If so, return <literal>true</> after performing
+ planning actions needed for that. Otherwise, return <literal>false</>.
+ This optional function is called during query planning.
+ If this function succeeds, <function>BeginDMLPushdown</>,
+ <function>IterateDMLPushdown</> and <function>EndDMLPushdown</> will be
+ called at the execution stage, instead. Otherwise, the table update
+ will be executed using the table-updating functions described above.
+ The parameters are the same as for <function>PlanForeignModify</>.
+ </para>
+
+ <para>
+ To execute the table update directly on the remote server, this function
+ must rewrite the target subplan with a <structname>ForeignScan</> plan
+ node that executes the table update directly on the remote server. The
+ <structfield>operation</> field of the <structname>ForeignScan</> must
+ be set to the <literal>CmdType</> enumeration appropriately; that is,
+ <literal>CMD_UPDATE</> for <command>UPDATE</>,
+ <literal>CMD_INSERT</> for <command>INSERT</>, and
+ <literal>CMD_DELETE</> for <command>DELETE</>.
+ </para>
+
+ <para>
+ See <xref linkend="fdw-planning"> for additional information.
+ </para>
+
+ <para>
+ If the <function>PlanDMLPushdown</> pointer is set to
+ <literal>NULL</>, no attempts to execute the table update directly on
+ the remote server are taken.
+ </para>
+
+ <para>
+ <programlisting>
+ void
+ BeginDMLPushdown (ForeignScanState *node,
+ int eflags);
+ </programlisting>
+
+ Begin executing a foreign table update directly on the remote server.
+ This is called during executor startup. It should perform any
+ initialization needed prior to the actual table update (that should be
+ done upon the first call to <function>IterateDMLPushdown</>).
+ The <structname>ForeignScanState</> node has already been created, but
+ its <structfield>fdw_state</> field is still NULL. Information about
+ the table to update is accessible through the
+ <structname>ForeignScanState</> node (in particular, from the underlying
+ <structname>ForeignScan</> plan node, which contains any FDW-private
+ information provided by <function>PlanDMLPushdown</>).
+ <literal>eflags</> contains flag bits describing the executor's
+ operating mode for this plan node.
+ </para>
+
+ <para>
+ Note that when <literal>(eflags & EXEC_FLAG_EXPLAIN_ONLY)</> is
+ true, this function should not perform any externally-visible actions;
+ it should only do the minimum required to make the node state valid
+ for <function>ExplainDMLPushdown</> and <function>EndDMLPushdown</>.
+ </para>
+
+ <para>
+ If the <function>BeginDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to execute the table update directly on
+ the remote server will fail with an error message.
+ </para>
+
+ <para>
+ <programlisting>
+ TupleTableSlot *
+ IterateDMLPushdown (ForeignScanState *node);
+ </programlisting>
+
+ When the <command>INSERT</>, <command>UPDATE</> or <command>DELETE</>
+ query doesn't have a <literal>RETURNING</> clause, just return NULL
+ after the actual table update directly executed on the remote server.
+ When the query has the clause, fetch one result containing the data
+ needed for the <literal>RETURNING</> calculation, returning it in a
+ tuple table slot (the node's <structfield>ScanTupleSlot</> should be
+ used for this purpose). The data that was actually inserted, updated
+ or deleted must be stored in the
+ <literal>es_result_relation_info->ri_projectReturning->pi_exprContext->ecxt_scantuple</>
+ of the node's <structname>EState</>.
+ Return NULL if no more rows are available.
+ Note that this is called in a short-lived memory context that will be
+ reset between invocations. Create a memory context in
+ <function>BeginDMLPushdown</> if you need longer-lived storage, or use
+ the <structfield>es_query_cxt</> of the node's <structname>EState</>.
+ </para>
+
+ <para>
+ The rows returned must match the <structfield>fdw_scan_tlist</> target
+ list if one was supplied, otherwise they must match the row type of the
+ foreign table being updated. If you choose to optimize away fetching
+ columns that are not needed for the <literal>RETURNING</> calculation,
+ you should insert nulls in those column positions, or else generate a
+ <structfield>fdw_scan_tlist</> list with those columns omitted.
+ </para>
+
+ <para>
+ Whether the query has the clause or not, the query's reported row count
+ must be incremented by the FDW itself. When the query doesn't has the
+ clause, the FDW must also increment the row count for the
+ <structname>ForeignScanState</> node in the <command>EXPLAIN ANALYZE</>
+ case.
+ </para>
+
+ <para>
+ If the <function>IterateDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to execute the table update directly on
+ the remote server will fail with an error message.
+ </para>
+
+ <para>
+ <programlisting>
+ void
+ EndDMLPushdown (ForeignScanState *node);
+ </programlisting>
+
+ End the table update and release resources. It is normally not important
+ to release palloc'd memory, but for example open files and connections
+ to remote servers should be cleaned up.
+ </para>
+
+ <para>
+ If the <function>EndDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to execute the table update directly on
+ the remote server will fail with an error message.
+ </para>
+
</sect2>
<sect2 id="fdw-callbacks-row-locking">
***************
*** 866,871 **** ExplainForeignModify (ModifyTableState *mtstate,
--- 1015,1043 ----
<command>EXPLAIN</>.
</para>
+ <para>
+ <programlisting>
+ void
+ ExplainDMLPushdown (ForeignScanState *node,
+ ExplainState *es);
+ </programlisting>
+
+ Print additional <command>EXPLAIN</> output for a foreign table update
+ that is executed directly on the remote server.
+ This function can call <function>ExplainPropertyText</> and
+ related functions to add fields to the <command>EXPLAIN</> output.
+ The flag fields in <literal>es</> can be used to determine what to
+ print, and the state of the <structname>ForeignScanState</> node
+ can be inspected to provide run-time statistics in the <command>EXPLAIN
+ ANALYZE</> case.
+ </para>
+
+ <para>
+ If the <function>ExplainDMLPushdown</> pointer is set to
+ <literal>NULL</>, no additional information is printed during
+ <command>EXPLAIN</>.
+ </para>
+
</sect2>
<sect2 id="fdw-callbacks-analyze">
***************
*** 1147,1153 **** GetForeignServerByName(const char *name, bool missing_ok);
<para>
The FDW callback functions <function>GetForeignRelSize</>,
<function>GetForeignPaths</>, <function>GetForeignPlan</>,
! <function>PlanForeignModify</>, and <function>GetForeignJoinPaths</>
must fit into the workings of the <productname>PostgreSQL</> planner.
Here are some notes about what they must do.
</para>
--- 1319,1326 ----
<para>
The FDW callback functions <function>GetForeignRelSize</>,
<function>GetForeignPaths</>, <function>GetForeignPlan</>,
! <function>PlanForeignModify</>, <function>GetForeignJoinPaths</>, and
! <function>PlanDMLPushdown</>
must fit into the workings of the <productname>PostgreSQL</> planner.
Here are some notes about what they must do.
</para>
***************
*** 1307,1313 **** GetForeignServerByName(const char *name, bool missing_ok);
<para>
When planning an <command>UPDATE</> or <command>DELETE</>,
! <function>PlanForeignModify</> can look up the <structname>RelOptInfo</>
struct for the foreign table and make use of the
<literal>baserel->fdw_private</> data previously created by the
scan-planning functions. However, in <command>INSERT</> the target
--- 1480,1487 ----
<para>
When planning an <command>UPDATE</> or <command>DELETE</>,
! <function>PlanForeignModify</> and <function>PlanDMLPushdown</>
! can look up the <structname>RelOptInfo</>
struct for the foreign table and make use of the
<literal>baserel->fdw_private</> data previously created by the
scan-planning functions. However, in <command>INSERT</> the target
*** a/doc/src/sgml/postgres-fdw.sgml
--- b/doc/src/sgml/postgres-fdw.sgml
***************
*** 484,489 ****
--- 484,498 ----
extension that's listed in the foreign server's <literal>extensions</>
option. Operators and functions in such clauses must
be <literal>IMMUTABLE</> as well.
+ For an <command>UPDATE</> or <command>DELETE</> query,
+ <filename>postgres_fdw</> attempts to optimize the query execution by
+ sending the whole query to the remote server if there are no query
+ <literal>WHERE</> clauses that cannot be sent to the remote server,
+ no local joins for the query, and no row-level local <literal>BEFORE</> or
+ <literal>AFTER</> triggers on the target table. In <command>UPDATE</>,
+ expressions to assign to target columns must use only built-in data types,
+ <literal>IMMUTABLE</> operators, or <literal>IMMUTABLE</> functions,
+ to reduce the risk of misexecution of the query.
</para>
<para>
*** a/src/backend/commands/explain.c
--- b/src/backend/commands/explain.c
***************
*** 888,894 **** ExplainNode(PlanState *planstate, List *ancestors,
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
! pname = sname = "Foreign Scan";
break;
case T_CustomScan:
sname = "Custom Scan";
--- 888,916 ----
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
! sname = "Foreign Scan";
! switch (((ForeignScan *) plan)->operation)
! {
! case CMD_SELECT:
! pname = "Foreign Scan";
! operation = "Select";
! break;
! case CMD_INSERT:
! pname = "Foreign Insert";
! operation = "Insert";
! break;
! case CMD_UPDATE:
! pname = "Foreign Update";
! operation = "Update";
! break;
! case CMD_DELETE:
! pname = "Foreign Delete";
! operation = "Delete";
! break;
! default:
! pname = "???";
! break;
! }
break;
case T_CustomScan:
sname = "Custom Scan";
***************
*** 1636,1641 **** show_plan_tlist(PlanState *planstate, List *ancestors, ExplainState *es)
--- 1658,1669 ----
return;
if (IsA(plan, RecursiveUnion))
return;
+ /* Likewise for ForeignScan that has pushed down INSERT/UPDATE/DELETE */
+ if (IsA(plan, ForeignScan) &&
+ (((ForeignScan *) plan)->operation == CMD_INSERT ||
+ ((ForeignScan *) plan)->operation == CMD_UPDATE ||
+ ((ForeignScan *) plan)->operation == CMD_DELETE))
+ return;
/* Set up deparsing context */
context = set_deparse_context_planstate(es->deparse_cxt,
***************
*** 2224,2231 **** show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es)
FdwRoutine *fdwroutine = fsstate->fdwroutine;
/* Let the FDW emit whatever fields it wants */
! if (fdwroutine->ExplainForeignScan != NULL)
! fdwroutine->ExplainForeignScan(fsstate, es);
}
/*
--- 2252,2267 ----
FdwRoutine *fdwroutine = fsstate->fdwroutine;
/* Let the FDW emit whatever fields it wants */
! if (((ForeignScan *) fsstate->ss.ps.plan)->operation != CMD_SELECT)
! {
! if (fdwroutine->ExplainDMLPushdown != NULL)
! fdwroutine->ExplainDMLPushdown(fsstate, es);
! }
! else
! {
! if (fdwroutine->ExplainForeignScan != NULL)
! fdwroutine->ExplainForeignScan(fsstate, es);
! }
}
/*
***************
*** 2611,2618 **** show_modifytable_info(ModifyTableState *mtstate, List *ancestors,
}
}
! /* Give FDW a chance */
! if (fdwroutine && fdwroutine->ExplainForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, j);
--- 2647,2656 ----
}
}
! /* Give FDW a chance if needed */
! if (!resultRelInfo->ri_FdwPushdown &&
! fdwroutine != NULL &&
! fdwroutine->ExplainForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, j);
*** a/src/backend/executor/execMain.c
--- b/src/backend/executor/execMain.c
***************
*** 1011,1018 **** InitPlan(QueryDesc *queryDesc, int eflags)
* CheckValidRowMarkRel.
*/
void
! CheckValidResultRel(Relation resultRel, CmdType operation)
{
TriggerDesc *trigDesc = resultRel->trigdesc;
FdwRoutine *fdwroutine;
--- 1011,1019 ----
* CheckValidRowMarkRel.
*/
void
! CheckValidResultRel(ResultRelInfo *resultRelInfo, CmdType operation)
{
+ Relation resultRel = resultRelInfo->ri_RelationDesc;
TriggerDesc *trigDesc = resultRel->trigdesc;
FdwRoutine *fdwroutine;
***************
*** 1128,1133 **** CheckValidResultRel(Relation resultRel, CmdType operation)
--- 1129,1166 ----
elog(ERROR, "unrecognized CmdType: %d", (int) operation);
break;
}
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ if (fdwroutine->BeginDMLPushdown == NULL ||
+ fdwroutine->IterateDMLPushdown == NULL ||
+ fdwroutine->EndDMLPushdown == NULL)
+ {
+ switch (operation)
+ {
+ case CMD_INSERT:
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot push down insert on foreign table \"%s\"",
+ RelationGetRelationName(resultRel))));
+ break;
+ case CMD_UPDATE:
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot push down update on foreign table \"%s\"",
+ RelationGetRelationName(resultRel))));
+ break;
+ case CMD_DELETE:
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot push down delete on foreign table \"%s\"",
+ RelationGetRelationName(resultRel))));
+ break;
+ default:
+ elog(ERROR, "unrecognized CmdType: %d", (int) operation);
+ break;
+ }
+ }
+ }
break;
default:
ereport(ERROR,
***************
*** 1245,1250 **** InitResultRelInfo(ResultRelInfo *resultRelInfo,
--- 1278,1284 ----
else
resultRelInfo->ri_FdwRoutine = NULL;
resultRelInfo->ri_FdwState = NULL;
+ resultRelInfo->ri_FdwPushdown = false;
resultRelInfo->ri_ConstraintExprs = NULL;
resultRelInfo->ri_junkFilter = NULL;
resultRelInfo->ri_projectReturning = NULL;
*** a/src/backend/executor/nodeForeignscan.c
--- b/src/backend/executor/nodeForeignscan.c
***************
*** 48,54 **** ForeignNext(ForeignScanState *node)
/* Call the Iterate function in short-lived context */
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! slot = node->fdwroutine->IterateForeignScan(node);
MemoryContextSwitchTo(oldcontext);
/*
--- 48,57 ----
/* Call the Iterate function in short-lived context */
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! if (plan->operation != CMD_SELECT)
! slot = node->fdwroutine->IterateDMLPushdown(node);
! else
! slot = node->fdwroutine->IterateForeignScan(node);
MemoryContextSwitchTo(oldcontext);
/*
***************
*** 226,232 **** ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
/*
* Tell the FDW to initialize the scan.
*/
! fdwroutine->BeginForeignScan(scanstate, eflags);
return scanstate;
}
--- 229,238 ----
/*
* Tell the FDW to initialize the scan.
*/
! if (node->operation != CMD_SELECT)
! fdwroutine->BeginDMLPushdown(scanstate, eflags);
! else
! fdwroutine->BeginForeignScan(scanstate, eflags);
return scanstate;
}
***************
*** 240,247 **** ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
void
ExecEndForeignScan(ForeignScanState *node)
{
/* Let the FDW shut down */
! node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
--- 246,258 ----
void
ExecEndForeignScan(ForeignScanState *node)
{
+ ForeignScan *plan = (ForeignScan *) node->ss.ps.plan;
+
/* Let the FDW shut down */
! if (plan->operation != CMD_SELECT)
! node->fdwroutine->EndDMLPushdown(node);
! else
! node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
*** a/src/backend/executor/nodeModifyTable.c
--- b/src/backend/executor/nodeModifyTable.c
***************
*** 138,150 **** ExecCheckPlanOutput(Relation resultRel, List *targetList)
* tupleSlot: slot holding tuple actually inserted/updated/deleted
* planSlot: slot holding tuple returned by top subplan node
*
* Returns a slot holding the result tuple
*/
static TupleTableSlot *
! ExecProcessReturning(ProjectionInfo *projectReturning,
TupleTableSlot *tupleSlot,
TupleTableSlot *planSlot)
{
ExprContext *econtext = projectReturning->pi_exprContext;
/*
--- 138,154 ----
* tupleSlot: slot holding tuple actually inserted/updated/deleted
* planSlot: slot holding tuple returned by top subplan node
*
+ * Note: If tupleSlot is NULL, the FDW should have already provided econtext's
+ * scan tuple.
+ *
* Returns a slot holding the result tuple
*/
static TupleTableSlot *
! ExecProcessReturning(ResultRelInfo *resultRelInfo,
TupleTableSlot *tupleSlot,
TupleTableSlot *planSlot)
{
+ ProjectionInfo *projectReturning = resultRelInfo->ri_projectReturning;
ExprContext *econtext = projectReturning->pi_exprContext;
/*
***************
*** 154,160 **** ExecProcessReturning(ProjectionInfo *projectReturning,
ResetExprContext(econtext);
/* Make tuple and any needed join variables available to ExecProject */
! econtext->ecxt_scantuple = tupleSlot;
econtext->ecxt_outertuple = planSlot;
/* Compute the RETURNING expressions */
--- 158,177 ----
ResetExprContext(econtext);
/* Make tuple and any needed join variables available to ExecProject */
! if (tupleSlot)
! econtext->ecxt_scantuple = tupleSlot;
! else
! {
! HeapTuple tuple;
!
! /*
! * RETURNING expressions might reference the tableoid column, so
! * initialize t_tableOid before evaluating them.
! */
! Assert(!TupIsNull(econtext->ecxt_scantuple));
! tuple = ExecMaterializeSlot(econtext->ecxt_scantuple);
! tuple->t_tableOid = RelationGetRelid(resultRelInfo->ri_RelationDesc);
! }
econtext->ecxt_outertuple = planSlot;
/* Compute the RETURNING expressions */
***************
*** 496,503 **** ExecInsert(ModifyTableState *mtstate,
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo->ri_projectReturning,
! slot, planSlot);
return NULL;
}
--- 513,519 ----
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo, slot, planSlot);
return NULL;
}
***************
*** 738,745 **** ldelete:;
ExecStoreTuple(&deltuple, slot, InvalidBuffer, false);
}
! rslot = ExecProcessReturning(resultRelInfo->ri_projectReturning,
! slot, planSlot);
/*
* Before releasing the target tuple again, make sure rslot has a
--- 754,760 ----
ExecStoreTuple(&deltuple, slot, InvalidBuffer, false);
}
! rslot = ExecProcessReturning(resultRelInfo, slot, planSlot);
/*
* Before releasing the target tuple again, make sure rslot has a
***************
*** 1024,1031 **** lreplace:;
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo->ri_projectReturning,
! slot, planSlot);
return NULL;
}
--- 1039,1045 ----
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo, slot, planSlot);
return NULL;
}
***************
*** 1380,1385 **** ExecModifyTable(ModifyTableState *node)
--- 1394,1414 ----
break;
}
+ /*
+ * If ri_FdwPushdown is true, all we need to do here is compute the
+ * RETURNING expressions.
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ Assert(resultRelInfo->ri_projectReturning);
+
+ /* No need to provide scan tuple to ExecProcessReturning. */
+ slot = ExecProcessReturning(resultRelInfo, NULL, planSlot);
+
+ estate->es_result_relation_info = saved_resultRelInfo;
+ return slot;
+ }
+
EvalPlanQualSetSlot(&node->mt_epqstate, planSlot);
slot = planSlot;
***************
*** 1559,1568 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
{
subplan = (Plan *) lfirst(l);
/*
* Verify result relation is a valid target for the current operation
*/
! CheckValidResultRel(resultRelInfo->ri_RelationDesc, operation);
/*
* If there are indices on the result relation, open them and save
--- 1588,1600 ----
{
subplan = (Plan *) lfirst(l);
+ /* Initialize the FdwPushdown flag */
+ resultRelInfo->ri_FdwPushdown = list_nth_int(node->fdwPushdowns, i);
+
/*
* Verify result relation is a valid target for the current operation
*/
! CheckValidResultRel(resultRelInfo, operation);
/*
* If there are indices on the result relation, open them and save
***************
*** 1583,1589 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
/* Also let FDWs init themselves for foreign-table result rels */
! if (resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, i);
--- 1615,1622 ----
mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
/* Also let FDWs init themselves for foreign-table result rels */
! if (!resultRelInfo->ri_FdwPushdown &&
! resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, i);
***************
*** 1754,1766 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
--- 1787,1811 ----
erm = ExecFindRowMark(estate, rc->rti, false);
/* build ExecAuxRowMark for each subplan */
+ resultRelInfo = mtstate->resultRelInfo;
for (i = 0; i < nplans; i++)
{
ExecAuxRowMark *aerm;
+ /*
+ * ignore subplan if the FDW pushes down the command to the remote
+ * server
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ resultRelInfo++;
+ continue;
+ }
+
subplan = mtstate->mt_plans[i]->plan;
aerm = ExecBuildAuxRowMark(erm, subplan->targetlist);
mtstate->mt_arowmarks[i] = lappend(mtstate->mt_arowmarks[i], aerm);
+ resultRelInfo++;
}
}
***************
*** 1821,1826 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
--- 1866,1881 ----
ExecCheckPlanOutput(resultRelInfo->ri_RelationDesc,
subplan->targetlist);
+ /*
+ * ignore subplan if the FDW pushes down the command to the
+ * remote server
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ resultRelInfo++;
+ continue;
+ }
+
j = ExecInitJunkFilter(subplan->targetlist,
resultRelInfo->ri_RelationDesc->rd_att->tdhasoid,
ExecInitExtraTupleSlot(estate));
***************
*** 1910,1916 **** ExecEndModifyTable(ModifyTableState *node)
{
ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
! if (resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->EndForeignModify != NULL)
resultRelInfo->ri_FdwRoutine->EndForeignModify(node->ps.state,
resultRelInfo);
--- 1965,1972 ----
{
ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
! if (!resultRelInfo->ri_FdwPushdown &&
! resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->EndForeignModify != NULL)
resultRelInfo->ri_FdwRoutine->EndForeignModify(node->ps.state,
resultRelInfo);
*** a/src/backend/nodes/copyfuncs.c
--- b/src/backend/nodes/copyfuncs.c
***************
*** 187,192 **** _copyModifyTable(const ModifyTable *from)
--- 187,193 ----
COPY_NODE_FIELD(withCheckOptionLists);
COPY_NODE_FIELD(returningLists);
COPY_NODE_FIELD(fdwPrivLists);
+ COPY_NODE_FIELD(fdwPushdowns);
COPY_NODE_FIELD(rowMarks);
COPY_SCALAR_FIELD(epqParam);
COPY_SCALAR_FIELD(onConflictAction);
***************
*** 646,651 **** _copyForeignScan(const ForeignScan *from)
--- 647,653 ----
/*
* copy remainder of node
*/
+ COPY_SCALAR_FIELD(operation);
COPY_SCALAR_FIELD(fs_server);
COPY_NODE_FIELD(fdw_exprs);
COPY_NODE_FIELD(fdw_private);
*** a/src/backend/nodes/outfuncs.c
--- b/src/backend/nodes/outfuncs.c
***************
*** 341,346 **** _outModifyTable(StringInfo str, const ModifyTable *node)
--- 341,347 ----
WRITE_NODE_FIELD(withCheckOptionLists);
WRITE_NODE_FIELD(returningLists);
WRITE_NODE_FIELD(fdwPrivLists);
+ WRITE_NODE_FIELD(fdwPushdowns);
WRITE_NODE_FIELD(rowMarks);
WRITE_INT_FIELD(epqParam);
WRITE_ENUM_FIELD(onConflictAction, OnConflictAction);
***************
*** 592,597 **** _outForeignScan(StringInfo str, const ForeignScan *node)
--- 593,599 ----
_outScanInfo(str, (const Scan *) node);
+ WRITE_ENUM_FIELD(operation, CmdType);
WRITE_OID_FIELD(fs_server);
WRITE_NODE_FIELD(fdw_exprs);
WRITE_NODE_FIELD(fdw_private);
*** a/src/backend/nodes/readfuncs.c
--- b/src/backend/nodes/readfuncs.c
***************
*** 1472,1477 **** _readModifyTable(void)
--- 1472,1478 ----
READ_NODE_FIELD(withCheckOptionLists);
READ_NODE_FIELD(returningLists);
READ_NODE_FIELD(fdwPrivLists);
+ READ_NODE_FIELD(fdwPushdowns);
READ_NODE_FIELD(rowMarks);
READ_INT_FIELD(epqParam);
READ_ENUM_FIELD(onConflictAction, OnConflictAction);
*** a/src/backend/optimizer/plan/createplan.c
--- b/src/backend/optimizer/plan/createplan.c
***************
*** 3788,3793 **** make_foreignscan(List *qptlist,
--- 3788,3794 ----
plan->lefttree = outer_plan;
plan->righttree = NULL;
node->scan.scanrelid = scanrelid;
+ node->operation = CMD_SELECT;
/* fs_server will be filled in by create_foreignscan_plan */
node->fs_server = InvalidOid;
node->fdw_exprs = fdw_exprs;
***************
*** 5064,5069 **** make_modifytable(PlannerInfo *root,
--- 5065,5071 ----
Plan *plan = &node->plan;
double total_size;
List *fdw_private_list;
+ List *fdwpushdown_list;
ListCell *subnode;
ListCell *lc;
int i;
***************
*** 5144,5155 **** make_modifytable(PlannerInfo *root,
--- 5146,5159 ----
* construct private plan data, and accumulate it all into a list.
*/
fdw_private_list = NIL;
+ fdwpushdown_list = NIL;
i = 0;
foreach(lc, resultRelations)
{
Index rti = lfirst_int(lc);
FdwRoutine *fdwroutine;
List *fdw_private;
+ bool fdwpushdown;
/*
* If possible, we want to get the FdwRoutine from our RelOptInfo for
***************
*** 5176,5182 **** make_modifytable(PlannerInfo *root,
--- 5180,5199 ----
fdwroutine = NULL;
}
+ /*
+ * If the target relation has any row-level triggers, we can't push
+ * down the command to the remote server.
+ */
if (fdwroutine != NULL &&
+ fdwroutine->PlanDMLPushdown != NULL &&
+ !has_row_triggers(root, rti, operation))
+ fdwpushdown = fdwroutine->PlanDMLPushdown(root, node, rti, i);
+ else
+ fdwpushdown = false;
+ fdwpushdown_list = lappend_int(fdwpushdown_list, fdwpushdown);
+
+ if (!fdwpushdown &&
+ fdwroutine != NULL &&
fdwroutine->PlanForeignModify != NULL)
fdw_private = fdwroutine->PlanForeignModify(root, node, rti, i);
else
***************
*** 5185,5190 **** make_modifytable(PlannerInfo *root,
--- 5202,5208 ----
i++;
}
node->fdwPrivLists = fdw_private_list;
+ node->fdwPushdowns = fdwpushdown_list;
return node;
}
*** a/src/backend/optimizer/util/plancat.c
--- b/src/backend/optimizer/util/plancat.c
***************
*** 1520,1522 **** has_unique_index(RelOptInfo *rel, AttrNumber attno)
--- 1520,1569 ----
}
return false;
}
+
+
+ /*
+ * has_row_triggers
+ *
+ * Detect whether the specified relation has any row-level triggers for event.
+ */
+ bool
+ has_row_triggers(PlannerInfo *root, Index rti, CmdType event)
+ {
+ RangeTblEntry *rte = planner_rt_fetch(rti, root);
+ Relation relation;
+ TriggerDesc *trigDesc;
+ bool result = false;
+
+ /* Assume we already have adequate lock */
+ relation = heap_open(rte->relid, NoLock);
+
+ trigDesc = relation->trigdesc;
+ switch (event)
+ {
+ case CMD_INSERT:
+ if (trigDesc &&
+ (trigDesc->trig_insert_after_row ||
+ trigDesc->trig_insert_before_row))
+ result = true;
+ break;
+ case CMD_UPDATE:
+ if (trigDesc &&
+ (trigDesc->trig_update_after_row ||
+ trigDesc->trig_update_before_row))
+ result = true;
+ break;
+ case CMD_DELETE:
+ if (trigDesc &&
+ (trigDesc->trig_delete_after_row ||
+ trigDesc->trig_delete_before_row))
+ result = true;
+ break;
+ default:
+ elog(ERROR, "unrecognized CmdType: %d", (int) event);
+ break;
+ }
+
+ heap_close(relation, NoLock);
+ return result;
+ }
*** a/src/include/executor/executor.h
--- b/src/include/executor/executor.h
***************
*** 184,190 **** extern void ExecutorEnd(QueryDesc *queryDesc);
extern void standard_ExecutorEnd(QueryDesc *queryDesc);
extern void ExecutorRewind(QueryDesc *queryDesc);
extern bool ExecCheckRTPerms(List *rangeTable, bool ereport_on_violation);
! extern void CheckValidResultRel(Relation resultRel, CmdType operation);
extern void InitResultRelInfo(ResultRelInfo *resultRelInfo,
Relation resultRelationDesc,
Index resultRelationIndex,
--- 184,190 ----
extern void standard_ExecutorEnd(QueryDesc *queryDesc);
extern void ExecutorRewind(QueryDesc *queryDesc);
extern bool ExecCheckRTPerms(List *rangeTable, bool ereport_on_violation);
! extern void CheckValidResultRel(ResultRelInfo *resultRelInfo, CmdType operation);
extern void InitResultRelInfo(ResultRelInfo *resultRelInfo,
Relation resultRelationDesc,
Index resultRelationIndex,
*** a/src/include/foreign/fdwapi.h
--- b/src/include/foreign/fdwapi.h
***************
*** 94,99 **** typedef void (*EndForeignModify_function) (EState *estate,
--- 94,111 ----
typedef int (*IsForeignRelUpdatable_function) (Relation rel);
+ typedef bool (*PlanDMLPushdown_function) (PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+
+ typedef void (*BeginDMLPushdown_function) (ForeignScanState *node,
+ int eflags);
+
+ typedef TupleTableSlot *(*IterateDMLPushdown_function) (ForeignScanState *node);
+
+ typedef void (*EndDMLPushdown_function) (ForeignScanState *node);
+
typedef RowMarkType (*GetForeignRowMarkType_function) (RangeTblEntry *rte,
LockClauseStrength strength);
***************
*** 111,116 **** typedef void (*ExplainForeignModify_function) (ModifyTableState *mtstate,
--- 123,131 ----
int subplan_index,
struct ExplainState *es);
+ typedef void (*ExplainDMLPushdown_function) (ForeignScanState *node,
+ struct ExplainState *es);
+
typedef int (*AcquireSampleRowsFunc) (Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
***************
*** 171,176 **** typedef struct FdwRoutine
--- 186,195 ----
ExecForeignDelete_function ExecForeignDelete;
EndForeignModify_function EndForeignModify;
IsForeignRelUpdatable_function IsForeignRelUpdatable;
+ PlanDMLPushdown_function PlanDMLPushdown;
+ BeginDMLPushdown_function BeginDMLPushdown;
+ IterateDMLPushdown_function IterateDMLPushdown;
+ EndDMLPushdown_function EndDMLPushdown;
/* Functions for SELECT FOR UPDATE/SHARE row locking */
GetForeignRowMarkType_function GetForeignRowMarkType;
***************
*** 180,185 **** typedef struct FdwRoutine
--- 199,205 ----
/* Support functions for EXPLAIN */
ExplainForeignScan_function ExplainForeignScan;
ExplainForeignModify_function ExplainForeignModify;
+ ExplainDMLPushdown_function ExplainDMLPushdown;
/* Support functions for ANALYZE */
AnalyzeForeignTable_function AnalyzeForeignTable;
*** a/src/include/nodes/execnodes.h
--- b/src/include/nodes/execnodes.h
***************
*** 311,316 **** typedef struct JunkFilter
--- 311,317 ----
* TrigInstrument optional runtime measurements for triggers
* FdwRoutine FDW callback functions, if foreign table
* FdwState available to save private state of FDW
+ * FdwPushdown true when the command is pushed down
* WithCheckOptions list of WithCheckOption's to be checked
* WithCheckOptionExprs list of WithCheckOption expr states
* ConstraintExprs array of constraint-checking expr states
***************
*** 334,339 **** typedef struct ResultRelInfo
--- 335,341 ----
Instrumentation *ri_TrigInstrument;
struct FdwRoutine *ri_FdwRoutine;
void *ri_FdwState;
+ bool ri_FdwPushdown;
List *ri_WithCheckOptions;
List *ri_WithCheckOptionExprs;
List **ri_ConstraintExprs;
*** a/src/include/nodes/plannodes.h
--- b/src/include/nodes/plannodes.h
***************
*** 189,194 **** typedef struct ModifyTable
--- 189,195 ----
List *withCheckOptionLists; /* per-target-table WCO lists */
List *returningLists; /* per-target-table RETURNING tlists */
List *fdwPrivLists; /* per-target-table FDW private data lists */
+ List *fdwPushdowns; /* per-target-table FDW pushdown flags */
List *rowMarks; /* PlanRowMarks (non-locking only) */
int epqParam; /* ID of Param for EvalPlanQual re-eval */
OnConflictAction onConflictAction; /* ON CONFLICT action */
***************
*** 531,536 **** typedef struct WorkTableScan
--- 532,538 ----
typedef struct ForeignScan
{
Scan scan;
+ CmdType operation; /* SELECT/INSERT/UPDATE/DELETE */
Oid fs_server; /* OID of foreign server */
List *fdw_exprs; /* expressions that FDW may evaluate */
List *fdw_private; /* private data for FDW */
*** a/src/include/optimizer/plancat.h
--- b/src/include/optimizer/plancat.h
***************
*** 55,58 **** extern Selectivity join_selectivity(PlannerInfo *root,
--- 55,60 ----
JoinType jointype,
SpecialJoinInfo *sjinfo);
+ extern bool has_row_triggers(PlannerInfo *root, Index rti, CmdType event);
+
#endif /* PLANCAT_H */
On 2016/02/05 12:28, Robert Haas wrote:
On Thu, Jan 28, 2016 at 7:36 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:Attached is that version of the patch.
I think that postgres_fdw might be able to insert a tableoid value in the
returned slot in e.g., postgresExecForeignInsert if AFTER ROW Triggers or
RETURNING expressions reference that value, but I didn't do anything about
that.
Thanks. I went with the earlier version, but split it into two commits.
Thank you!
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Feb 3, 2016 at 3:31 PM, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>
wrote:
On 2016/01/28 15:20, Rushabh Lathia wrote:
On Thu, Jan 28, 2016 at 11:33 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp <mailto:fujita.etsuro@lab.ntt.co.jp>> wrote:On 2016/01/27 21:23, Rushabh Lathia wrote:
If I understood correctly, above documentation means, that if
FDW have
DMLPushdown APIs that is enough. But in reality thats not the
case, we
need ExecForeignInsert, ExecForeignUpdate, or ExecForeignDelete
in case
DML is not pushable.And here fact is DMLPushdown APIs are optional for FDW, so that
if FDW
don't have DMLPushdown APIs they can still very well perform the
DML
operations using ExecForeignInsert, ExecForeignUpdate, or
ExecForeignDelete.So documentation should be like:
If the IsForeignRelUpdatable pointer is set to NULL, foreign
tables are
assumed to be insertable, updatable, or deletable if the FDW
provides
ExecForeignInsert, ExecForeignUpdate, or ExecForeignDelete
respectively,If FDW provides DMLPushdown APIs and the DML are pushable to the
foreign
server, then FDW still needs ExecForeignInsert,
ExecForeignUpdate, or
ExecForeignDelete for the non-pushable DML operation.What's your opinion ?
I agree that we should add this to the documentation, too.
I added docs to the IsForeignRelUpdatable documentation. Also, a brief
introductory remark has been added at the beginning of the DML pushdown
APIs' documentation.BTW, if I understand correctly, I think we should also modify
relation_is_updatabale() accordingly. Am I right?
Yep, we need to modify relation_is_updatable().
I thought I'd modify that function in the same way as
CheckValidResultRel(), but I noticed that we cannot do that, because we
don't have any information on whether each update is pushed down to the
remote server by PlanDMLPushdown, during relation_is_updatabale().
Sorry I didn't get you here. Can't resultRelInfo->ri_FdwPushdown gives
information update whether update is pushed down safe or not ? What my
concern here is, lets say resultRelInfo->ri_FdwPushdown marked as true
(PlanDMLPushdown return true), but later into CheckValidResultRel() it
found out that missing BeginDMLPushdown, IterateDMLPushdown and
EndDMLPushdown APIs and it will end up with error.
What I think CheckValidResultRel() should do is, if
resultRelInfo->ri_FdwPushdown is true then check for the DMLPushdown API
and if it doesn't find those API then check for traditional APIs
(ExecForeignInsert, ExecForeignUpdate or ExecForeignDelete). And when it
doesn't find both then it should return an error.
I changed CheckValidResultRel(), where
1) Don't throw an error if resultRelInfo->ri_FdwPushdown is true and
DMLPushdown APIs are missing as query can still perform operation with
traditional ExecForeign APIs. So in this situation just marked
resultRelInfo->ri_FdwPushdown to false.
(Wondering can we add the checks for DMLPushdown APIs into PlanDMLPushdown
as additional check? Means PlanDMLPushdown should return true only if FDW
provides the BeginDMLPushdown & IterateDMLPushdown & EndDMLPushdown APIs ?
What you say ?)
2) Don't throw an error if resultRelInfo->ri_FdwPushdown is true and
DMLPushdown APIs is present but ExecForeign APIs are missing.
3) Throw an error if resultRelInfo->ri_FdwPushdown is false and ExecForeign
APIs are missing.
Attaching is the WIP patch here, do share your thought.
(need to apply on top of V6 patch)
So, I left that function as-is. relation_is_updatabale() is just used for
display in the information_schema views, so ISTM that that function is fine
as-is. (As for CheckValidResultRel(), I revised it so as to check the
presence of DML pushdown APIs after checking the existing APIs if the given
command will be pushed down. The reason is because we assume the presence
of the existing APIs, anyway.)
I revised other docs and some comments, mostly for consistency.
Attached is an updated version of the patch, which has been created on top
of the updated version of the bugfix patch posted by Robert in [1]
(attached).Best regards,
Etsuro Fujita[1]
/messages/by-id/CA+TgmoZ40j2uC5aC1NXu03oj4CrVOLkS15XX+PTFP-1U-8zR1Q@mail.gmail.com
--
Rushabh Lathia
Attachments:
checkvisiblerel.patchtext/x-diff; charset=US-ASCII; name=checkvisiblerel.patchDownload
diff --git a/src/backend/executor/.execMain.c.swp b/src/backend/executor/.execMain.c.swp
index 6f54fd9..8a29cc6 100644
Binary files a/src/backend/executor/.execMain.c.swp and b/src/backend/executor/.execMain.c.swp differ
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index eb24051..04d90ea 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1082,86 +1082,88 @@ CheckValidResultRel(ResultRelInfo *resultRelInfo, CmdType operation)
RelationGetRelationName(resultRel))));
break;
case RELKIND_FOREIGN_TABLE:
- /* Okay only if the FDW supports it */
- fdwroutine = GetFdwRoutineForRelation(resultRel, false);
- switch (operation)
- {
- case CMD_INSERT:
- if (fdwroutine->ExecForeignInsert == NULL)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("cannot insert into foreign table \"%s\"",
- RelationGetRelationName(resultRel))));
- if (fdwroutine->IsForeignRelUpdatable != NULL &&
- (fdwroutine->IsForeignRelUpdatable(resultRel) & (1 << CMD_INSERT)) == 0)
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("foreign table \"%s\" does not allow inserts",
- RelationGetRelationName(resultRel))));
- break;
- case CMD_UPDATE:
- if (fdwroutine->ExecForeignUpdate == NULL)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("cannot update foreign table \"%s\"",
- RelationGetRelationName(resultRel))));
- if (fdwroutine->IsForeignRelUpdatable != NULL &&
- (fdwroutine->IsForeignRelUpdatable(resultRel) & (1 << CMD_UPDATE)) == 0)
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("foreign table \"%s\" does not allow updates",
- RelationGetRelationName(resultRel))));
- break;
- case CMD_DELETE:
- if (fdwroutine->ExecForeignDelete == NULL)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("cannot delete from foreign table \"%s\"",
- RelationGetRelationName(resultRel))));
- if (fdwroutine->IsForeignRelUpdatable != NULL &&
- (fdwroutine->IsForeignRelUpdatable(resultRel) & (1 << CMD_DELETE)) == 0)
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("foreign table \"%s\" does not allow deletes",
- RelationGetRelationName(resultRel))));
- break;
- default:
- elog(ERROR, "unrecognized CmdType: %d", (int) operation);
- break;
- }
- if (resultRelInfo->ri_FdwPushdown)
{
- if (fdwroutine->BeginDMLPushdown == NULL ||
- fdwroutine->IterateDMLPushdown == NULL ||
- fdwroutine->EndDMLPushdown == NULL)
+ /*
+ * Mark DMLPushdownOk if DML is pushed down safe and FDW provides
+ * the DMLPushdown APIs. Later if ExecForeignInsert or
+ * ExecForeignUpdate or ExecForeignDelete is missing and
+ * DMLPushdownOk is false, then only throw an error.
+ */
+ bool DMLPushdownOk = false;
+ /* Okay only if the FDW supports it */
+ fdwroutine = GetFdwRoutineForRelation(resultRel, false);
+ if (resultRelInfo->ri_FdwPushdown)
{
- switch (operation)
+ if (fdwroutine->BeginDMLPushdown == NULL ||
+ fdwroutine->IterateDMLPushdown == NULL ||
+ fdwroutine->EndDMLPushdown == NULL)
{
- case CMD_INSERT:
+ switch (operation)
+ {
+ case CMD_INSERT:
+ case CMD_UPDATE:
+ case CMD_DELETE:
+ DMLPushdownOk = false;
+ break;
+ default:
+ elog(ERROR, "unrecognized CmdType: %d", (int) operation);
+ break;
+ }
+ }
+ else
+ DMLPushdownOk = true;
+ }
+
+ switch (operation)
+ {
+ case CMD_INSERT:
+ if (fdwroutine->ExecForeignInsert == NULL &&
+ !DMLPushdownOk)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("cannot push down insert on foreign table \"%s\"",
+ errmsg("cannot insert into foreign table \"%s\"",
RelationGetRelationName(resultRel))));
- break;
- case CMD_UPDATE:
+ if (fdwroutine->IsForeignRelUpdatable != NULL &&
+ (fdwroutine->IsForeignRelUpdatable(resultRel) & (1 << CMD_INSERT)) == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("foreign table \"%s\" does not allow inserts",
+ RelationGetRelationName(resultRel))));
+ break;
+ case CMD_UPDATE:
+ if (fdwroutine->ExecForeignUpdate == NULL &&
+ !DMLPushdownOk)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("cannot push down update on foreign table \"%s\"",
+ errmsg("cannot update foreign table \"%s\"",
RelationGetRelationName(resultRel))));
- break;
- case CMD_DELETE:
+ if (fdwroutine->IsForeignRelUpdatable != NULL &&
+ (fdwroutine->IsForeignRelUpdatable(resultRel) & (1 << CMD_UPDATE)) == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("foreign table \"%s\" does not allow updates",
+ RelationGetRelationName(resultRel))));
+ break;
+ case CMD_DELETE:
+ if (fdwroutine->ExecForeignDelete == NULL &&
+ !DMLPushdownOk)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("cannot push down delete on foreign table \"%s\"",
+ errmsg("cannot delete from foreign table \"%s\"",
RelationGetRelationName(resultRel))));
- break;
- default:
- elog(ERROR, "unrecognized CmdType: %d", (int) operation);
- break;
- }
+ if (fdwroutine->IsForeignRelUpdatable != NULL &&
+ (fdwroutine->IsForeignRelUpdatable(resultRel) & (1 << CMD_DELETE)) == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("foreign table \"%s\" does not allow deletes",
+ RelationGetRelationName(resultRel))));
+ break;
+ default:
+ elog(ERROR, "unrecognized CmdType: %d", (int) operation);
+ break;
}
+ break;
}
- break;
default:
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
On Fri, Feb 5, 2016 at 4:46 PM, Rushabh Lathia <rushabh.lathia@gmail.com>
wrote:
On Wed, Feb 3, 2016 at 3:31 PM, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp
wrote:
On 2016/01/28 15:20, Rushabh Lathia wrote:
On Thu, Jan 28, 2016 at 11:33 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp <mailto:fujita.etsuro@lab.ntt.co.jp>>
wrote:On 2016/01/27 21:23, Rushabh Lathia wrote:
If I understood correctly, above documentation means, that if
FDW have
DMLPushdown APIs that is enough. But in reality thats not the
case, we
need ExecForeignInsert, ExecForeignUpdate, or ExecForeignDelete
in case
DML is not pushable.And here fact is DMLPushdown APIs are optional for FDW, so that
if FDW
don't have DMLPushdown APIs they can still very well perform the
DML
operations using ExecForeignInsert, ExecForeignUpdate, or
ExecForeignDelete.So documentation should be like:
If the IsForeignRelUpdatable pointer is set to NULL, foreign
tables are
assumed to be insertable, updatable, or deletable if the FDW
provides
ExecForeignInsert, ExecForeignUpdate, or ExecForeignDelete
respectively,If FDW provides DMLPushdown APIs and the DML are pushable to the
foreign
server, then FDW still needs ExecForeignInsert,
ExecForeignUpdate, or
ExecForeignDelete for the non-pushable DML operation.What's your opinion ?
I agree that we should add this to the documentation, too.
I added docs to the IsForeignRelUpdatable documentation. Also, a brief
introductory remark has been added at the beginning of the DML pushdown
APIs' documentation.BTW, if I understand correctly, I think we should also modify
relation_is_updatabale() accordingly. Am I right?
Yep, we need to modify relation_is_updatable().
I thought I'd modify that function in the same way as
CheckValidResultRel(), but I noticed that we cannot do that, because we
don't have any information on whether each update is pushed down to the
remote server by PlanDMLPushdown, during relation_is_updatabale().Sorry I didn't get you here. Can't resultRelInfo->ri_FdwPushdown gives
information update whether update is pushed down safe or not ? What my
concern here is, lets say resultRelInfo->ri_FdwPushdown marked as true
(PlanDMLPushdown return true), but later into CheckValidResultRel() it
found out that missing BeginDMLPushdown, IterateDMLPushdown and
EndDMLPushdown APIs and it will end up with error.What I think CheckValidResultRel() should do is, if
resultRelInfo->ri_FdwPushdown is true then check for the DMLPushdown API
and if it doesn't find those API then check for traditional APIs
(ExecForeignInsert, ExecForeignUpdate or ExecForeignDelete). And when it
doesn't find both then it should return an error.I changed CheckValidResultRel(), where
1) Don't throw an error if resultRelInfo->ri_FdwPushdown is true and
DMLPushdown APIs are missing as query can still perform operation with
traditional ExecForeign APIs. So in this situation just marked
resultRelInfo->ri_FdwPushdown to false.(Wondering can we add the checks for DMLPushdown APIs into PlanDMLPushdown
as additional check? Means PlanDMLPushdown should return true only if FDW
provides the BeginDMLPushdown & IterateDMLPushdown & EndDMLPushdown APIs ?
What you say ?)
On another thought, we should not give responsibility to check for the APIs
to the FDW. So may be we should call PlanDMLPushdown only if
BeginDMLPushdown & IterateDMLPushdown & EndDMLPushdown APIs are present
into FDW. That means prepare DMLPushdown plan only when all the required
APIs are available with FDW. This will also reduce the changes into
CheckValidResultRel().
Thanks Ashutosh Bapat for healthy discussion.
PFA patch.
2) Don't throw an error if resultRelInfo->ri_FdwPushdown is true and
DMLPushdown APIs is present but ExecForeign APIs are missing.
3) Throw an error if resultRelInfo->ri_FdwPushdown is false and
ExecForeign APIs are missing.Attaching is the WIP patch here, do share your thought.
(need to apply on top of V6 patch)So, I left that function as-is. relation_is_updatabale() is just used for
display in the information_schema views, so ISTM that that function is fine
as-is. (As for CheckValidResultRel(), I revised it so as to check the
presence of DML pushdown APIs after checking the existing APIs if the given
command will be pushed down. The reason is because we assume the presence
of the existing APIs, anyway.)I revised other docs and some comments, mostly for consistency.
Attached is an updated version of the patch, which has been created on
top of the updated version of the bugfix patch posted by Robert in [1]
(attached).Best regards,
Etsuro Fujita[1]
/messages/by-id/CA+TgmoZ40j2uC5aC1NXu03oj4CrVOLkS15XX+PTFP-1U-8zR1Q@mail.gmail.com--
Rushabh Lathia
--
Rushabh Lathia
Attachments:
fdw-dml-pushdown-v7.patchtext/x-diff; charset=US-ASCII; name=fdw-dml-pushdown-v7.patchDownload
diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c
index d778e61..157de59 100644
--- a/contrib/postgres_fdw/deparse.c
+++ b/contrib/postgres_fdw/deparse.c
@@ -1069,6 +1069,66 @@ deparseUpdateSql(StringInfo buf, PlannerInfo *root,
}
/*
+ * deparse remote UPDATE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+void
+deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+{
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ deparse_expr_cxt context;
+ int nestlevel;
+ bool first;
+ ListCell *lc;
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = params_list;
+
+ appendStringInfoString(buf, "UPDATE ");
+ deparseRelation(buf, rel);
+ appendStringInfoString(buf, " SET ");
+
+ /* Make sure any constants in the exprs are printed portably */
+ nestlevel = set_transmission_modes();
+
+ first = true;
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(targetlist, attnum);
+
+ if (!first)
+ appendStringInfoString(buf, ", ");
+ first = false;
+
+ deparseColumnRef(buf, rtindex, attnum, root);
+ appendStringInfoString(buf, " = ");
+ deparseExpr((Expr *) tle->expr, &context);
+ }
+
+ reset_transmission_modes(nestlevel);
+
+ if (remote_conds)
+ appendWhereClause(remote_conds, &context);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+}
+
+/*
* deparse remote DELETE statement
*
* The statement text is appended to buf, and we also create an integer List
@@ -1091,6 +1151,40 @@ deparseDeleteSql(StringInfo buf, PlannerInfo *root,
}
/*
+ * deparse remote DELETE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+void
+deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+{
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ deparse_expr_cxt context;
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = params_list;
+
+ appendStringInfoString(buf, "DELETE FROM ");
+ deparseRelation(buf, rel);
+
+ if (remote_conds)
+ appendWhereClause(remote_conds, &context);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+}
+
+/*
* Add a RETURNING clause, if needed, to an INSERT/UPDATE/DELETE.
*/
static void
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index f621024..ed9ac3e 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -1314,7 +1314,26 @@ INSERT INTO ft2 (c1,c2,c3)
(3 rows)
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+EXPLAIN (verbose, costs off)
+UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 300), c3 = (c3 || '_update3'::text) WHERE ((("C 1" % 10) = 3))
+(3 rows)
+
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+EXPLAIN (verbose, costs off)
+UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Output: c1, c2, c3, c4, c5, c6, c7, c8
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 400), c3 = (c3 || '_update7'::text) WHERE ((("C 1" % 10) = 7)) RETURNING "C 1", c2, c3, c4, c5, c6, c7, c8
+(4 rows)
+
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
c1 | c2 | c3 | c4 | c5 | c6 | c7 | c8
------+-----+--------------------+------------------------------+--------------------------+----+------------+-----
@@ -1424,7 +1443,7 @@ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
- FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
+ FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
@@ -1445,16 +1464,14 @@ UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
- DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
- QUERY PLAN
-----------------------------------------------------------------------------------------
+ DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
+ QUERY PLAN
+--------------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
- Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1 RETURNING "C 1", c4
- -> Foreign Scan on public.ft2
- Output: ctid
- Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) FOR UPDATE
-(6 rows)
+ -> Foreign Delete on public.ft2
+ Remote SQL: DELETE FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) RETURNING "C 1", c4
+(4 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
@@ -1565,7 +1582,7 @@ DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
(103 rows)
EXPLAIN (verbose, costs off)
-DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
+DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
@@ -2426,16 +2443,14 @@ INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
(1 row)
EXPLAIN (verbose, costs off)
-UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
- QUERY PLAN
--------------------------------------------------------------------------------------------------------------------
+UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
+ QUERY PLAN
+------------------------------------------------------------------------------------
Update on public.ft2
Output: (tableoid)::regclass
- Remote SQL: UPDATE "S 1"."T 1" SET c3 = $2 WHERE ctid = $1
- -> Foreign Scan on public.ft2
- Output: c1, c2, NULL::integer, 'bar'::text, c4, c5, c6, c7, c8, ctid
- Remote SQL: SELECT "C 1", c2, c4, c5, c6, c7, c8, ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
-(6 rows)
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c3 = 'bar'::text WHERE (("C 1" = 9999))
+(4 rows)
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
@@ -2444,16 +2459,14 @@ UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
(1 row)
EXPLAIN (verbose, costs off)
-DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
- QUERY PLAN
-------------------------------------------------------------------------------------
+DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
+ QUERY PLAN
+--------------------------------------------------------------------
Delete on public.ft2
Output: (tableoid)::regclass
- Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1
- -> Foreign Scan on public.ft2
- Output: ctid
- Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
-(6 rows)
+ -> Foreign Delete on public.ft2
+ Remote SQL: DELETE FROM "S 1"."T 1" WHERE (("C 1" = 9999))
+(4 rows)
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
@@ -2607,7 +2620,7 @@ CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
-CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
+CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
@@ -2766,7 +2779,7 @@ savepoint s3;
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
-CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
+CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (-2) WHERE ((c2 = 42)) AND (("C 1" = 10))
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
@@ -2906,7 +2919,7 @@ CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
-CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
+CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2positive;
-- But inconsistent check constraints provide inconsistent results
ALTER FOREIGN TABLE ft1 ADD CONSTRAINT ft1_c2negative CHECK (c2 < 0);
@@ -3299,6 +3312,199 @@ NOTICE: NEW: (13,"test triggered !")
(0,27)
(1 row)
+-- cleanup
+DROP TRIGGER trig_row_before ON rem1;
+DROP TRIGGER trig_row_after ON rem1;
+DROP TRIGGER trig_local_before ON loc1;
+-- Test DML pushdown functionality
+-- Test with statement-level triggers
+CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+(3 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+(3 rows)
+
+DROP TRIGGER trig_stmt_before ON rem1;
+CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+(3 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+(3 rows)
+
+DROP TRIGGER trig_stmt_after ON rem1;
+-- Test with row-level ON INSERT triggers
+CREATE TRIGGER trig_row_before_insert
+BEFORE INSERT ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+(3 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+(3 rows)
+
+DROP TRIGGER trig_row_before_insert ON rem1;
+CREATE TRIGGER trig_row_after_insert
+AFTER INSERT ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+(3 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+(3 rows)
+
+DROP TRIGGER trig_row_after_insert ON rem1;
+-- Test with row-level ON UPDATE triggers
+CREATE TRIGGER trig_row_before_update
+BEFORE UPDATE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+---------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+(5 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+(3 rows)
+
+DROP TRIGGER trig_row_before_update ON rem1;
+CREATE TRIGGER trig_row_after_update
+AFTER UPDATE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+-------------------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+(5 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+(3 rows)
+
+DROP TRIGGER trig_row_after_update ON rem1;
+-- Test with row-level ON DELETE triggers
+CREATE TRIGGER trig_row_before_delete
+BEFORE DELETE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+(3 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+---------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+(5 rows)
+
+DROP TRIGGER trig_row_before_delete ON rem1;
+CREATE TRIGGER trig_row_after_delete
+AFTER DELETE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+(3 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+------------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+(5 rows)
+
+DROP TRIGGER trig_row_after_delete ON rem1;
-- ===================================================================
-- test inheritance features
-- ===================================================================
@@ -3768,6 +3974,56 @@ fetch from c;
update bar set f2 = null where current of c;
ERROR: WHERE CURRENT OF is not supported for this table type
rollback;
+explain (verbose, costs off)
+delete from foo where f1 < 5 returning *;
+ QUERY PLAN
+--------------------------------------------------------------------------------
+ Delete on public.foo
+ Output: foo.f1, foo.f2
+ Delete on public.foo
+ Foreign Delete on public.foo2
+ -> Index Scan using i_foo_f1 on public.foo
+ Output: foo.ctid
+ Index Cond: (foo.f1 < 5)
+ -> Foreign Delete on public.foo2
+ Remote SQL: DELETE FROM public.loct1 WHERE ((f1 < 5)) RETURNING f1, f2
+(9 rows)
+
+delete from foo where f1 < 5 returning *;
+ f1 | f2
+----+----
+ 1 | 1
+ 3 | 3
+ 0 | 0
+ 2 | 2
+ 4 | 4
+(5 rows)
+
+explain (verbose, costs off)
+update bar set f2 = f2 + 100 returning *;
+ QUERY PLAN
+------------------------------------------------------------------------------
+ Update on public.bar
+ Output: bar.f1, bar.f2
+ Update on public.bar
+ Foreign Update on public.bar2
+ -> Seq Scan on public.bar
+ Output: bar.f1, (bar.f2 + 100), bar.ctid
+ -> Foreign Update on public.bar2
+ Remote SQL: UPDATE public.loct2 SET f2 = (f2 + 100) RETURNING f1, f2
+(8 rows)
+
+update bar set f2 = f2 + 100 returning *;
+ f1 | f2
+----+-----
+ 1 | 311
+ 2 | 322
+ 6 | 266
+ 3 | 333
+ 4 | 344
+ 7 | 277
+(6 rows)
+
drop table foo cascade;
NOTICE: drop cascades to foreign table foo2
drop table bar cascade;
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index d5a2af9..a292dbb 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -61,6 +61,8 @@ enum FdwScanPrivateIndex
{
/* SQL statement to execute remotely (as a String node) */
FdwScanPrivateSelectSql,
+ /* List of restriction clauses that can be executed remotely */
+ FdwScanPrivateRemoteConds,
/* Integer list of attribute numbers retrieved by the SELECT */
FdwScanPrivateRetrievedAttrs,
/* Integer representing the desired fetch_size */
@@ -90,6 +92,28 @@ enum FdwModifyPrivateIndex
};
/*
+ * Similarly, this enum describes what's kept in the fdw_private list for
+ * a ForeignScan node that has pushed down an UPDATE/DELETE to the remote
+ * server. We store:
+ *
+ * 1) UPDATE/DELETE statement text to be sent to the remote server
+ * 2) Boolean flag showing if the remote query has a RETURNING clause
+ * 3) Integer list of attribute numbers retrieved by RETURNING, if any
+ * 4) Boolean flag showing if we set the command es_processed
+ */
+enum FdwDmlPushdownPrivateIndex
+{
+ /* SQL statement to execute remotely (as a String node) */
+ FdwDmlPushdownPrivateUpdateSql,
+ /* has-returning flag (as an integer Value node) */
+ FdwDmlPushdownPrivateHasReturning,
+ /* Integer list of attribute numbers retrieved by RETURNING */
+ FdwDmlPushdownPrivateRetrievedAttrs,
+ /* set-processed flag (as an integer Value node) */
+ FdwDmlPushdownPrivateSetProcessed
+};
+
+/*
* Execution state of a foreign scan using postgres_fdw.
*/
typedef struct PgFdwScanState
@@ -154,6 +178,37 @@ typedef struct PgFdwModifyState
} PgFdwModifyState;
/*
+ * Execution state of a foreign scan that has pushed down a foreign table
+ * modification to the remote server
+ */
+typedef struct PgFdwDmlPushdownState
+{
+ Relation rel; /* relcache entry for the foreign table */
+ AttInMetadata *attinmeta; /* attribute datatype conversion metadata */
+
+ /* extracted fdw_private data */
+ char *query; /* text of UPDATE/DELETE command */
+ bool has_returning; /* is there a RETURNING clause? */
+ List *retrieved_attrs; /* attr numbers retrieved by RETURNING */
+ bool set_processed; /* do we set the command es_processed? */
+
+ /* for remote query execution */
+ PGconn *conn; /* connection for the update */
+ int numParams; /* number of parameters passed to query */
+ FmgrInfo *param_flinfo; /* output conversion functions for them */
+ List *param_exprs; /* executable expressions for param values */
+ const char **param_values; /* textual values of query parameters */
+
+ /* for storing result tuples */
+ PGresult *result; /* result for query */
+ int num_tuples; /* # of result tuples */
+ int next_tuple; /* index of next one to return */
+
+ /* working memory context */
+ MemoryContext temp_cxt; /* context for per-tuple temporary data */
+} PgFdwDmlPushdownState;
+
+/*
* Workspace for analyzing a foreign table.
*/
typedef struct PgFdwAnalyzeState
@@ -245,6 +300,13 @@ static TupleTableSlot *postgresExecForeignDelete(EState *estate,
static void postgresEndForeignModify(EState *estate,
ResultRelInfo *resultRelInfo);
static int postgresIsForeignRelUpdatable(Relation rel);
+static bool postgresPlanDMLPushdown(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+static void postgresBeginDMLPushdown(ForeignScanState *node, int eflags);
+static TupleTableSlot *postgresIterateDMLPushdown(ForeignScanState *node);
+static void postgresEndDMLPushdown(ForeignScanState *node);
static void postgresExplainForeignScan(ForeignScanState *node,
ExplainState *es);
static void postgresExplainForeignModify(ModifyTableState *mtstate,
@@ -252,6 +314,8 @@ static void postgresExplainForeignModify(ModifyTableState *mtstate,
List *fdw_private,
int subplan_index,
ExplainState *es);
+static void postgresExplainDMLPushdown(ForeignScanState *node,
+ ExplainState *es);
static bool postgresAnalyzeForeignTable(Relation relation,
AcquireSampleRowsFunc *func,
BlockNumber *totalpages);
@@ -288,6 +352,8 @@ static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
TupleTableSlot *slot);
static void store_returning_result(PgFdwModifyState *fmstate,
TupleTableSlot *slot, PGresult *res);
+static void execute_dml_stmt(ForeignScanState *node);
+static TupleTableSlot *get_returning_data(ForeignScanState *node);
static int postgresAcquireSampleRowsFunc(Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
@@ -330,10 +396,15 @@ postgres_fdw_handler(PG_FUNCTION_ARGS)
routine->ExecForeignDelete = postgresExecForeignDelete;
routine->EndForeignModify = postgresEndForeignModify;
routine->IsForeignRelUpdatable = postgresIsForeignRelUpdatable;
+ routine->PlanDMLPushdown = postgresPlanDMLPushdown;
+ routine->BeginDMLPushdown = postgresBeginDMLPushdown;
+ routine->IterateDMLPushdown = postgresIterateDMLPushdown;
+ routine->EndDMLPushdown = postgresEndDMLPushdown;
/* Support functions for EXPLAIN */
routine->ExplainForeignScan = postgresExplainForeignScan;
routine->ExplainForeignModify = postgresExplainForeignModify;
+ routine->ExplainDMLPushdown = postgresExplainDMLPushdown;
/* Support functions for ANALYZE */
routine->AnalyzeForeignTable = postgresAnalyzeForeignTable;
@@ -1011,7 +1082,8 @@ postgresGetForeignPlan(PlannerInfo *root,
* Build the fdw_private list that will be available to the executor.
* Items in the list must match enum FdwScanPrivateIndex, above.
*/
- fdw_private = list_make3(makeString(sql.data),
+ fdw_private = list_make4(makeString(sql.data),
+ remote_conds,
retrieved_attrs,
makeInteger(fpinfo->fetch_size));
@@ -1308,13 +1380,6 @@ postgresAddForeignUpdateTargets(Query *parsetree,
/*
* postgresPlanForeignModify
* Plan an insert/update/delete operation on a foreign table
- *
- * Note: currently, the plan tree generated for UPDATE/DELETE will always
- * include a ForeignScan that retrieves ctids (using SELECT FOR UPDATE)
- * and then the ModifyTable node will have to execute individual remote
- * UPDATE/DELETE commands. If there are no local conditions or joins
- * needed, it'd be better to let the scan node do UPDATE/DELETE RETURNING
- * and then do nothing at ModifyTable. Room for future optimization ...
*/
static List *
postgresPlanForeignModify(PlannerInfo *root,
@@ -1825,6 +1890,337 @@ postgresIsForeignRelUpdatable(Relation rel)
}
/*
+ * postgresPlanDMLPushdown
+ * Consider pushing down a foreign table modification to the remote server
+ *
+ * Decide whether the table modification is safe to push down to the remote end,
+ * and if so, modify subplan so as to do that.
+ *
+ * Conditions checked here:
+ *
+ * 1. The table modification must be an UPDATE or DELETE.
+ *
+ * 2. It's unsafe to push down the command if there are any local joins needed.
+ *
+ * 3. It's unsafe to push down the command if there are any quals that can't be
+ * evaluated remotely.
+ *
+ * 4. We can't push down an UPDATE, if any expressions to assign to the target
+ * columns are unsafe to evaluate on the remote end.
+ */
+static bool
+postgresPlanDMLPushdown(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index)
+{
+ CmdType operation = plan->operation;
+ Plan *subplan = (Plan *) list_nth(plan->plans, subplan_index);
+ RangeTblEntry *rte = planner_rt_fetch(resultRelation, root);
+ Relation rel;
+ StringInfoData sql;
+ ForeignScan *fscan;
+ List *targetAttrs = NIL;
+ List *remote_conds;
+ List *params_list = NIL;
+ List *returningList = NIL;
+ List *retrieved_attrs = NIL;
+
+ /*
+ * Decide whether the table modification is safe to push down to the remote
+ * server.
+ */
+
+ /* Check point 1 */
+ if (operation == CMD_INSERT)
+ return false;
+
+ /* Check point 2 */
+ if (nodeTag(subplan) != T_ForeignScan)
+ return false;
+
+ /* Check point 3 */
+ if (subplan->qual != NIL)
+ return false;
+
+ /* Check point 4 */
+ if (operation == CMD_UPDATE)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[resultRelation];
+ int col;
+
+ /*
+ * We transmit only columns that were explicitly targets of the UPDATE,
+ * so as to avoid unnecessary data transmission.
+ */
+ col = -1;
+ while ((col = bms_next_member(rte->updatedCols, col)) >= 0)
+ {
+ /* bit numbers are offset by FirstLowInvalidHeapAttributeNumber */
+ AttrNumber attno = col + FirstLowInvalidHeapAttributeNumber;
+ TargetEntry *tle;
+
+ if (attno <= InvalidAttrNumber) /* shouldn't happen */
+ elog(ERROR, "system-column update is not supported");
+
+ tle = get_tle_by_resno(subplan->targetlist, attno);
+
+ if (!is_foreign_expr(root, baserel, (Expr *) tle->expr))
+ return false;
+
+ targetAttrs = lappend_int(targetAttrs, attno);
+ }
+ }
+
+ /*
+ * Ok, modify subplan so as to push down the command to the remote server.
+ */
+ fscan = (ForeignScan *) subplan;
+
+ initStringInfo(&sql);
+
+ /*
+ * Core code already has some lock on each rel being planned, so we can
+ * use NoLock here.
+ */
+ rel = heap_open(rte->relid, NoLock);
+
+ /*
+ * Extract the baserestrictinfo clauses that can be evaluated remotely.
+ */
+ remote_conds = (List *) list_nth(fscan->fdw_private,
+ FdwScanPrivateRemoteConds);
+
+ /*
+ * Extract the relevant RETURNING list if any.
+ */
+ if (plan->returningLists)
+ returningList = (List *) list_nth(plan->returningLists, subplan_index);
+
+ /*
+ * Construct the SQL command string.
+ */
+ switch (operation)
+ {
+ case CMD_UPDATE:
+ deparsePushedDownUpdateSql(&sql, root, resultRelation, rel,
+ ((Plan *) fscan)->targetlist,
+ targetAttrs,
+ remote_conds, ¶ms_list,
+ returningList, &retrieved_attrs);
+ break;
+ case CMD_DELETE:
+ deparsePushedDownDeleteSql(&sql, root, resultRelation, rel,
+ remote_conds, ¶ms_list,
+ returningList, &retrieved_attrs);
+ break;
+ default:
+ elog(ERROR, "unexpected operation: %d", (int) operation);
+ break;
+ }
+
+ /*
+ * Update the operation info.
+ */
+ fscan->operation = operation;
+
+ /*
+ * Update the fdw_exprs list that will be available to the executor.
+ */
+ fscan->fdw_exprs = params_list;
+
+ /*
+ * Update the fdw_private list that will be available to the executor.
+ * Items in the list must match enum FdwDmlPushdownPrivateIndex, above.
+ */
+ fscan->fdw_private = list_make4(makeString(sql.data),
+ makeInteger((retrieved_attrs != NIL)),
+ retrieved_attrs,
+ makeInteger(plan->canSetTag));
+
+ heap_close(rel, NoLock);
+ return true;
+}
+
+/*
+ * postgresBeginDMLPushdown
+ * Initiate pushing down a foreign table modification to the remote server
+ */
+static void
+postgresBeginDMLPushdown(ForeignScanState *node, int eflags)
+{
+ ForeignScan *fsplan = (ForeignScan *) node->ss.ps.plan;
+ EState *estate = node->ss.ps.state;
+ PgFdwDmlPushdownState *dpstate;
+ RangeTblEntry *rte;
+ Oid userid;
+ ForeignTable *table;
+ UserMapping *user;
+ int numParams;
+ int i;
+ ListCell *lc;
+
+ /*
+ * Do nothing in EXPLAIN (no ANALYZE) case. node->fdw_state stays NULL.
+ */
+ if (eflags & EXEC_FLAG_EXPLAIN_ONLY)
+ return;
+
+ /*
+ * We'll save private state in node->fdw_state.
+ */
+ dpstate = (PgFdwDmlPushdownState *) palloc0(sizeof(PgFdwDmlPushdownState));
+ node->fdw_state = (void *) dpstate;
+
+ /*
+ * Identify which user to do the remote access as. This should match what
+ * ExecCheckRTEPerms() does.
+ */
+ rte = rt_fetch(fsplan->scan.scanrelid, estate->es_range_table);
+ userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
+
+ /* Get info about foreign table. */
+ dpstate->rel = node->ss.ss_currentRelation;
+ table = GetForeignTable(RelationGetRelid(dpstate->rel));
+ user = GetUserMapping(userid, table->serverid);
+
+ /*
+ * Get connection to the foreign server. Connection manager will
+ * establish new connection if necessary.
+ */
+ dpstate->conn = GetConnection(user, false);
+
+ /* Initialize state variable */
+ dpstate->num_tuples = -1; /* -1 means not set yet */
+
+ /* Get private info created by planner functions. */
+ dpstate->query = strVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateUpdateSql));
+ dpstate->has_returning = intVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateHasReturning));
+ dpstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateRetrievedAttrs);
+ dpstate->set_processed = intVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateSetProcessed));
+
+ /* Create context for per-tuple temp workspace. */
+ dpstate->temp_cxt = AllocSetContextCreate(estate->es_query_cxt,
+ "postgres_fdw temporary data",
+ ALLOCSET_SMALL_MINSIZE,
+ ALLOCSET_SMALL_INITSIZE,
+ ALLOCSET_SMALL_MAXSIZE);
+
+ /* Prepare for input conversion of RETURNING results. */
+ if (dpstate->has_returning)
+ dpstate->attinmeta = TupleDescGetAttInMetadata(RelationGetDescr(dpstate->rel));
+
+ /* Prepare for output conversion of parameters used in remote query. */
+ numParams = list_length(fsplan->fdw_exprs);
+ dpstate->numParams = numParams;
+ dpstate->param_flinfo = (FmgrInfo *) palloc0(sizeof(FmgrInfo) * numParams);
+
+ i = 0;
+ foreach(lc, fsplan->fdw_exprs)
+ {
+ Node *param_expr = (Node *) lfirst(lc);
+ Oid typefnoid;
+ bool isvarlena;
+
+ getTypeOutputInfo(exprType(param_expr), &typefnoid, &isvarlena);
+ fmgr_info(typefnoid, &dpstate->param_flinfo[i]);
+ i++;
+ }
+
+ /*
+ * Prepare remote-parameter expressions for evaluation. (Note: in
+ * practice, we expect that all these expressions will be just Params, so
+ * we could possibly do something more efficient than using the full
+ * expression-eval machinery for this. But probably there would be little
+ * benefit, and it'd require postgres_fdw to know more than is desirable
+ * about Param evaluation.)
+ */
+ dpstate->param_exprs = (List *)
+ ExecInitExpr((Expr *) fsplan->fdw_exprs,
+ (PlanState *) node);
+
+ /*
+ * Allocate buffer for text form of query parameters, if any.
+ */
+ if (numParams > 0)
+ dpstate->param_values = (const char **) palloc0(numParams * sizeof(char *));
+ else
+ dpstate->param_values = NULL;
+}
+
+/*
+ * postgresIterateDMLPushdown
+ * Execute pushing down a foreign table modification to the remote server
+ */
+static TupleTableSlot *
+postgresIterateDMLPushdown(ForeignScanState *node)
+{
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ EState *estate = node->ss.ps.state;
+ ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
+
+ /*
+ * If this is the first call after Begin, execute the statement.
+ */
+ if (dpstate->num_tuples == -1)
+ execute_dml_stmt(node);
+
+ /*
+ * If the local query doesn't specify RETURNING, just clear tuple slot.
+ */
+ if (!resultRelInfo->ri_projectReturning)
+ {
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+ Instrumentation *instr = node->ss.ps.instrument;
+
+ Assert(!dpstate->has_returning);
+
+ /* Increment the command es_processed count if necessary. */
+ if (dpstate->set_processed)
+ estate->es_processed += dpstate->num_tuples;
+
+ /* Increment the tuple count for EXPLAIN ANALYZE if necessary. */
+ if (instr)
+ instr->tuplecount += dpstate->num_tuples;
+
+ return ExecClearTuple(slot);
+ }
+
+ /*
+ * Get the next RETURNING tuple.
+ */
+ return get_returning_data(node);
+}
+
+/*
+ * postgresEndDMLPushdown
+ * Finish pushing down a foreign table modification to the remote server
+ */
+static void
+postgresEndDMLPushdown(ForeignScanState *node)
+{
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+
+ /* if dpstate is NULL, we are in EXPLAIN; nothing to do */
+ if (dpstate == NULL)
+ return;
+
+ /* Release PGresult */
+ if (dpstate->result)
+ PQclear(dpstate->result);
+
+ /* Release remote connection */
+ ReleaseConnection(dpstate->conn);
+ dpstate->conn = NULL;
+
+ /* MemoryContext will be deleted automatically. */
+}
+
+/*
* postgresExplainForeignScan
* Produce extra output for EXPLAIN of a ForeignScan on a foreign table
*/
@@ -1862,6 +2258,25 @@ postgresExplainForeignModify(ModifyTableState *mtstate,
}
}
+/*
+ * postgresExplainDMLPushdown
+ * Produce extra output for EXPLAIN of a ForeignScan on a foreign table
+ * that has pushed down an UPDATE/DELETE to the remote server
+ */
+static void
+postgresExplainDMLPushdown(ForeignScanState *node, ExplainState *es)
+{
+ List *fdw_private;
+ char *sql;
+
+ if (es->verbose)
+ {
+ fdw_private = ((ForeignScan *) node->ss.ps.plan)->fdw_private;
+ sql = strVal(list_nth(fdw_private, FdwDmlPushdownPrivateUpdateSql));
+ ExplainPropertyText("Remote SQL", sql, es);
+ }
+}
+
/*
* estimate_path_cost_size
@@ -2474,6 +2889,139 @@ store_returning_result(PgFdwModifyState *fmstate,
}
/*
+ * Execute a pushed-down UPDATE/DELETE statement.
+ */
+static void
+execute_dml_stmt(ForeignScanState *node)
+{
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ ExprContext *econtext = node->ss.ps.ps_ExprContext;
+ int numParams = dpstate->numParams;
+ const char **values = dpstate->param_values;
+
+ /*
+ * Construct array of query parameter values in text format.
+ */
+ if (numParams > 0)
+ {
+ int nestlevel;
+ int i;
+ ListCell *lc;
+
+ nestlevel = set_transmission_modes();
+
+ i = 0;
+ foreach(lc, dpstate->param_exprs)
+ {
+ ExprState *expr_state = (ExprState *) lfirst(lc);
+ Datum expr_value;
+ bool isNull;
+
+ /* Evaluate the parameter expression */
+ expr_value = ExecEvalExpr(expr_state, econtext, &isNull, NULL);
+
+ /*
+ * Get string representation of each parameter value by invoking
+ * type-specific output function, unless the value is null.
+ */
+ if (isNull)
+ values[i] = NULL;
+ else
+ values[i] = OutputFunctionCall(&dpstate->param_flinfo[i],
+ expr_value);
+ i++;
+ }
+
+ reset_transmission_modes(nestlevel);
+ }
+
+ /*
+ * Notice that we pass NULL for paramTypes, thus forcing the remote server
+ * to infer types for all parameters. Since we explicitly cast every
+ * parameter (see deparse.c), the "inference" is trivial and will produce
+ * the desired result. This allows us to avoid assuming that the remote
+ * server has the same OIDs we do for the parameters' types.
+ *
+ * We don't use a PG_TRY block here, so be careful not to throw error
+ * without releasing the PGresult.
+ */
+ dpstate->result = PQexecParams(dpstate->conn, dpstate->query,
+ numParams, NULL, values, NULL, NULL, 0);
+ if (PQresultStatus(dpstate->result) !=
+ (dpstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
+ pgfdw_report_error(ERROR, dpstate->result, dpstate->conn, true,
+ dpstate->query);
+
+ /* Get the number of rows affected. */
+ if (dpstate->has_returning)
+ dpstate->num_tuples = PQntuples(dpstate->result);
+ else
+ dpstate->num_tuples = atoi(PQcmdTuples(dpstate->result));
+}
+
+/*
+ * Get the result of a RETURNING clause.
+ */
+static TupleTableSlot *
+get_returning_data(ForeignScanState *node)
+{
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ EState *estate = node->ss.ps.state;
+ ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+
+ Assert(resultRelInfo->ri_projectReturning);
+
+ /* If we didn't get any tuples, must be end of data. */
+ if (dpstate->next_tuple >= dpstate->num_tuples)
+ return ExecClearTuple(slot);
+
+ /* Increment the command es_processed count if necessary. */
+ if (dpstate->set_processed)
+ estate->es_processed += 1;
+
+ /*
+ * Store a RETURNING tuple. If has_returning is false, just emit a dummy
+ * tuple. (We have has_returning=false if the local query is of the form
+ * UPDATE/DELETE .. RETURNING 1 for example.)
+ */
+ if (!dpstate->has_returning)
+ ExecStoreAllNullTuple(slot);
+ else
+ {
+ /*
+ * On error, be sure to release the PGresult on the way out. Callers
+ * do not have PG_TRY blocks to ensure this happens.
+ */
+ PG_TRY();
+ {
+ HeapTuple newtup;
+
+ newtup = make_tuple_from_result_row(dpstate->result,
+ dpstate->next_tuple,
+ dpstate->rel,
+ dpstate->attinmeta,
+ dpstate->retrieved_attrs,
+ dpstate->temp_cxt);
+ ExecStoreTuple(newtup, slot, InvalidBuffer, false);
+ }
+ PG_CATCH();
+ {
+ if (dpstate->result)
+ PQclear(dpstate->result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ }
+ dpstate->next_tuple++;
+
+ /* Make slot available for evaluation of the local query RETURNING list. */
+ resultRelInfo->ri_projectReturning->pi_exprContext->ecxt_scantuple = slot;
+
+ return slot;
+}
+
+/*
* postgresAnalyzeForeignTable
* Test whether analyzing this foreign table is supported
*/
diff --git a/contrib/postgres_fdw/postgres_fdw.h b/contrib/postgres_fdw/postgres_fdw.h
index 2b63281..c0f5766 100644
--- a/contrib/postgres_fdw/postgres_fdw.h
+++ b/contrib/postgres_fdw/postgres_fdw.h
@@ -93,10 +93,24 @@ extern void deparseUpdateSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *targetAttrs, List *returningList,
List **retrieved_attrs);
+extern void deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseDeleteSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *returningList,
List **retrieved_attrs);
+extern void deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseAnalyzeSizeSql(StringInfo buf, Relation rel);
extern void deparseAnalyzeSql(StringInfo buf, Relation rel,
List **retrieved_attrs);
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 1978e16..d0493e1 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -399,28 +399,32 @@ INSERT INTO ft2 (c1,c2,c3) SELECT c1+1000,c2+100, c3 || c3 FROM ft2 LIMIT 20;
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+EXPLAIN (verbose, costs off)
+UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+EXPLAIN (verbose, costs off)
+UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
- FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
+ FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
- DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
+ DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
-DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
+DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
EXPLAIN (verbose, costs off)
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
-UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
+UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
-DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
+DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
-- Test that trigger on remote table works as expected
@@ -737,6 +741,90 @@ UPDATE rem1 SET f2 = 'testo';
-- Test returning a system attribute
INSERT INTO rem1(f2) VALUES ('test') RETURNING ctid;
+-- cleanup
+DROP TRIGGER trig_row_before ON rem1;
+DROP TRIGGER trig_row_after ON rem1;
+DROP TRIGGER trig_local_before ON loc1;
+
+
+-- Test DML pushdown functionality
+
+-- Test with statement-level triggers
+CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+DROP TRIGGER trig_stmt_before ON rem1;
+
+CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+DROP TRIGGER trig_stmt_after ON rem1;
+
+-- Test with row-level ON INSERT triggers
+CREATE TRIGGER trig_row_before_insert
+BEFORE INSERT ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+DROP TRIGGER trig_row_before_insert ON rem1;
+
+CREATE TRIGGER trig_row_after_insert
+AFTER INSERT ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+DROP TRIGGER trig_row_after_insert ON rem1;
+
+-- Test with row-level ON UPDATE triggers
+CREATE TRIGGER trig_row_before_update
+BEFORE UPDATE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can't be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+DROP TRIGGER trig_row_before_update ON rem1;
+
+CREATE TRIGGER trig_row_after_update
+AFTER UPDATE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can't be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+DROP TRIGGER trig_row_after_update ON rem1;
+
+-- Test with row-level ON DELETE triggers
+CREATE TRIGGER trig_row_before_delete
+BEFORE DELETE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can't be pushed down
+DROP TRIGGER trig_row_before_delete ON rem1;
+
+CREATE TRIGGER trig_row_after_delete
+AFTER DELETE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can't be pushed down
+DROP TRIGGER trig_row_after_delete ON rem1;
+
-- ===================================================================
-- test inheritance features
-- ===================================================================
@@ -868,6 +956,13 @@ fetch from c;
update bar set f2 = null where current of c;
rollback;
+explain (verbose, costs off)
+delete from foo where f1 < 5 returning *;
+delete from foo where f1 < 5 returning *;
+explain (verbose, costs off)
+update bar set f2 = f2 + 100 returning *;
+update bar set f2 = f2 + 100 returning *;
+
drop table foo cascade;
drop table bar cascade;
drop table loct1;
diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml
index a6945d3..88a3c16 100644
--- a/doc/src/sgml/fdwhandler.sgml
+++ b/doc/src/sgml/fdwhandler.sgml
@@ -672,13 +672,162 @@ IsForeignRelUpdatable (Relation rel);
<literal>NULL</>, foreign tables are assumed to be insertable, updatable,
or deletable if the FDW provides <function>ExecForeignInsert</>,
<function>ExecForeignUpdate</>, or <function>ExecForeignDelete</>
- respectively. This function is only needed if the FDW supports some
+ respectively.
+ (If the FDW optimizes a foreign table update on a foreign table using
+ <function>PlanDMLPushdown</>, it still needs to provide
+ <function>BeginDMLPushdown</>, <function>IterateDMLPushdown</> and
+ <function>EndDMLPushdown</> to execute the optimized update.)
+ This function is only needed if the FDW supports some
tables that are updatable and some that are not. (Even then, it's
permissible to throw an error in the execution routine instead of
checking in this function. However, this function is used to determine
updatability for display in the <literal>information_schema</> views.)
</para>
+ <para>
+ If an FDW supports an optimization that executes a foreign table update
+ directly on the remote server, it should provide the following callback
+ functions in addition to the table-updating functions described above:
+ </para>
+
+ <para>
+<programlisting>
+bool
+PlanDMLPushdown (PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+</programlisting>
+
+ Decide whether it is safe to execute a foreign table update directly
+ on the remote server. If so, return <literal>true</> after performing
+ planning actions needed for that. Otherwise, return <literal>false</>.
+ This optional function is called during query planning.
+ If this function succeeds, <function>BeginDMLPushdown</>,
+ <function>IterateDMLPushdown</> and <function>EndDMLPushdown</> will be
+ called at the execution stage, instead. Otherwise, the table update
+ will be executed using the table-updating functions described above.
+ The parameters are the same as for <function>PlanForeignModify</>.
+ </para>
+
+ <para>
+ To execute the table update directly on the remote server, this function
+ must rewrite the target subplan with a <structname>ForeignScan</> plan
+ node that executes the table update directly on the remote server. The
+ <structfield>operation</> field of the <structname>ForeignScan</> must
+ be set to the <literal>CmdType</> enumeration appropriately; that is,
+ <literal>CMD_UPDATE</> for <command>UPDATE</>,
+ <literal>CMD_INSERT</> for <command>INSERT</>, and
+ <literal>CMD_DELETE</> for <command>DELETE</>.
+ </para>
+
+ <para>
+ See <xref linkend="fdw-planning"> for additional information.
+ </para>
+
+ <para>
+ If the <function>PlanDMLPushdown</> pointer is set to
+ <literal>NULL</>, no attempts to execute the table update directly on
+ the remote server are taken.
+ </para>
+
+ <para>
+<programlisting>
+void
+BeginDMLPushdown (ForeignScanState *node,
+ int eflags);
+</programlisting>
+
+ Begin executing a foreign table update directly on the remote server.
+ This is called during executor startup. It should perform any
+ initialization needed prior to the actual table update (that should be
+ done upon the first call to <function>IterateDMLPushdown</>).
+ The <structname>ForeignScanState</> node has already been created, but
+ its <structfield>fdw_state</> field is still NULL. Information about
+ the table to update is accessible through the
+ <structname>ForeignScanState</> node (in particular, from the underlying
+ <structname>ForeignScan</> plan node, which contains any FDW-private
+ information provided by <function>PlanDMLPushdown</>).
+ <literal>eflags</> contains flag bits describing the executor's
+ operating mode for this plan node.
+ </para>
+
+ <para>
+ Note that when <literal>(eflags & EXEC_FLAG_EXPLAIN_ONLY)</> is
+ true, this function should not perform any externally-visible actions;
+ it should only do the minimum required to make the node state valid
+ for <function>ExplainDMLPushdown</> and <function>EndDMLPushdown</>.
+ </para>
+
+ <para>
+ If the <function>BeginDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to execute the table update directly on
+ the remote server will fail with an error message.
+ </para>
+
+ <para>
+<programlisting>
+TupleTableSlot *
+IterateDMLPushdown (ForeignScanState *node);
+</programlisting>
+
+ When the <command>INSERT</>, <command>UPDATE</> or <command>DELETE</>
+ query doesn't have a <literal>RETURNING</> clause, just return NULL
+ after the actual table update directly executed on the remote server.
+ When the query has the clause, fetch one result containing the data
+ needed for the <literal>RETURNING</> calculation, returning it in a
+ tuple table slot (the node's <structfield>ScanTupleSlot</> should be
+ used for this purpose). The data that was actually inserted, updated
+ or deleted must be stored in the
+ <literal>es_result_relation_info->ri_projectReturning->pi_exprContext->ecxt_scantuple</>
+ of the node's <structname>EState</>.
+ Return NULL if no more rows are available.
+ Note that this is called in a short-lived memory context that will be
+ reset between invocations. Create a memory context in
+ <function>BeginDMLPushdown</> if you need longer-lived storage, or use
+ the <structfield>es_query_cxt</> of the node's <structname>EState</>.
+ </para>
+
+ <para>
+ The rows returned must match the <structfield>fdw_scan_tlist</> target
+ list if one was supplied, otherwise they must match the row type of the
+ foreign table being updated. If you choose to optimize away fetching
+ columns that are not needed for the <literal>RETURNING</> calculation,
+ you should insert nulls in those column positions, or else generate a
+ <structfield>fdw_scan_tlist</> list with those columns omitted.
+ </para>
+
+ <para>
+ Whether the query has the clause or not, the query's reported row count
+ must be incremented by the FDW itself. When the query doesn't has the
+ clause, the FDW must also increment the row count for the
+ <structname>ForeignScanState</> node in the <command>EXPLAIN ANALYZE</>
+ case.
+ </para>
+
+ <para>
+ If the <function>IterateDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to execute the table update directly on
+ the remote server will fail with an error message.
+ </para>
+
+ <para>
+<programlisting>
+void
+EndDMLPushdown (ForeignScanState *node);
+</programlisting>
+
+ End the table update and release resources. It is normally not important
+ to release palloc'd memory, but for example open files and connections
+ to remote servers should be cleaned up.
+ </para>
+
+ <para>
+ If the <function>EndDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to execute the table update directly on
+ the remote server will fail with an error message.
+ </para>
+
</sect2>
<sect2 id="fdw-callbacks-row-locking">
@@ -866,6 +1015,29 @@ ExplainForeignModify (ModifyTableState *mtstate,
<command>EXPLAIN</>.
</para>
+ <para>
+<programlisting>
+void
+ExplainDMLPushdown (ForeignScanState *node,
+ ExplainState *es);
+</programlisting>
+
+ Print additional <command>EXPLAIN</> output for a foreign table update
+ that is executed directly on the remote server.
+ This function can call <function>ExplainPropertyText</> and
+ related functions to add fields to the <command>EXPLAIN</> output.
+ The flag fields in <literal>es</> can be used to determine what to
+ print, and the state of the <structname>ForeignScanState</> node
+ can be inspected to provide run-time statistics in the <command>EXPLAIN
+ ANALYZE</> case.
+ </para>
+
+ <para>
+ If the <function>ExplainDMLPushdown</> pointer is set to
+ <literal>NULL</>, no additional information is printed during
+ <command>EXPLAIN</>.
+ </para>
+
</sect2>
<sect2 id="fdw-callbacks-analyze">
@@ -1147,7 +1319,8 @@ GetForeignServerByName(const char *name, bool missing_ok);
<para>
The FDW callback functions <function>GetForeignRelSize</>,
<function>GetForeignPaths</>, <function>GetForeignPlan</>,
- <function>PlanForeignModify</>, and <function>GetForeignJoinPaths</>
+ <function>PlanForeignModify</>, <function>GetForeignJoinPaths</>, and
+ <function>PlanDMLPushdown</>
must fit into the workings of the <productname>PostgreSQL</> planner.
Here are some notes about what they must do.
</para>
@@ -1307,7 +1480,8 @@ GetForeignServerByName(const char *name, bool missing_ok);
<para>
When planning an <command>UPDATE</> or <command>DELETE</>,
- <function>PlanForeignModify</> can look up the <structname>RelOptInfo</>
+ <function>PlanForeignModify</> and <function>PlanDMLPushdown</>
+ can look up the <structname>RelOptInfo</>
struct for the foreign table and make use of the
<literal>baserel->fdw_private</> data previously created by the
scan-planning functions. However, in <command>INSERT</> the target
diff --git a/doc/src/sgml/postgres-fdw.sgml b/doc/src/sgml/postgres-fdw.sgml
index a90983c..ff86137 100644
--- a/doc/src/sgml/postgres-fdw.sgml
+++ b/doc/src/sgml/postgres-fdw.sgml
@@ -484,6 +484,15 @@
extension that's listed in the foreign server's <literal>extensions</>
option. Operators and functions in such clauses must
be <literal>IMMUTABLE</> as well.
+ For an <command>UPDATE</> or <command>DELETE</> query,
+ <filename>postgres_fdw</> attempts to optimize the query execution by
+ sending the whole query to the remote server if there are no query
+ <literal>WHERE</> clauses that cannot be sent to the remote server,
+ no local joins for the query, and no row-level local <literal>BEFORE</> or
+ <literal>AFTER</> triggers on the target table. In <command>UPDATE</>,
+ expressions to assign to target columns must use only built-in data types,
+ <literal>IMMUTABLE</> operators, or <literal>IMMUTABLE</> functions,
+ to reduce the risk of misexecution of the query.
</para>
<para>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 25d8ca0..05d4a35 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -888,7 +888,29 @@ ExplainNode(PlanState *planstate, List *ancestors,
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
- pname = sname = "Foreign Scan";
+ sname = "Foreign Scan";
+ switch (((ForeignScan *) plan)->operation)
+ {
+ case CMD_SELECT:
+ pname = "Foreign Scan";
+ operation = "Select";
+ break;
+ case CMD_INSERT:
+ pname = "Foreign Insert";
+ operation = "Insert";
+ break;
+ case CMD_UPDATE:
+ pname = "Foreign Update";
+ operation = "Update";
+ break;
+ case CMD_DELETE:
+ pname = "Foreign Delete";
+ operation = "Delete";
+ break;
+ default:
+ pname = "???";
+ break;
+ }
break;
case T_CustomScan:
sname = "Custom Scan";
@@ -1636,6 +1658,12 @@ show_plan_tlist(PlanState *planstate, List *ancestors, ExplainState *es)
return;
if (IsA(plan, RecursiveUnion))
return;
+ /* Likewise for ForeignScan that has pushed down INSERT/UPDATE/DELETE */
+ if (IsA(plan, ForeignScan) &&
+ (((ForeignScan *) plan)->operation == CMD_INSERT ||
+ ((ForeignScan *) plan)->operation == CMD_UPDATE ||
+ ((ForeignScan *) plan)->operation == CMD_DELETE))
+ return;
/* Set up deparsing context */
context = set_deparse_context_planstate(es->deparse_cxt,
@@ -2224,8 +2252,16 @@ show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es)
FdwRoutine *fdwroutine = fsstate->fdwroutine;
/* Let the FDW emit whatever fields it wants */
- if (fdwroutine->ExplainForeignScan != NULL)
- fdwroutine->ExplainForeignScan(fsstate, es);
+ if (((ForeignScan *) fsstate->ss.ps.plan)->operation != CMD_SELECT)
+ {
+ if (fdwroutine->ExplainDMLPushdown != NULL)
+ fdwroutine->ExplainDMLPushdown(fsstate, es);
+ }
+ else
+ {
+ if (fdwroutine->ExplainForeignScan != NULL)
+ fdwroutine->ExplainForeignScan(fsstate, es);
+ }
}
/*
@@ -2611,8 +2647,10 @@ show_modifytable_info(ModifyTableState *mtstate, List *ancestors,
}
}
- /* Give FDW a chance */
- if (fdwroutine && fdwroutine->ExplainForeignModify != NULL)
+ /* Give FDW a chance if needed */
+ if (!resultRelInfo->ri_FdwPushdown &&
+ fdwroutine != NULL &&
+ fdwroutine->ExplainForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, j);
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 76f7297..68a3a13 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1011,8 +1011,9 @@ InitPlan(QueryDesc *queryDesc, int eflags)
* CheckValidRowMarkRel.
*/
void
-CheckValidResultRel(Relation resultRel, CmdType operation)
+CheckValidResultRel(ResultRelInfo *resultRelInfo, CmdType operation)
{
+ Relation resultRel = resultRelInfo->ri_RelationDesc;
TriggerDesc *trigDesc = resultRel->trigdesc;
FdwRoutine *fdwroutine;
@@ -1086,7 +1087,8 @@ CheckValidResultRel(Relation resultRel, CmdType operation)
switch (operation)
{
case CMD_INSERT:
- if (fdwroutine->ExecForeignInsert == NULL)
+ if (fdwroutine->ExecForeignInsert == NULL &&
+ !resultRelInfo->ri_FdwPushdown)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot insert into foreign table \"%s\"",
@@ -1099,7 +1101,8 @@ CheckValidResultRel(Relation resultRel, CmdType operation)
RelationGetRelationName(resultRel))));
break;
case CMD_UPDATE:
- if (fdwroutine->ExecForeignUpdate == NULL)
+ if (fdwroutine->ExecForeignUpdate == NULL &&
+ !resultRelInfo->ri_FdwPushdown)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot update foreign table \"%s\"",
@@ -1112,7 +1115,8 @@ CheckValidResultRel(Relation resultRel, CmdType operation)
RelationGetRelationName(resultRel))));
break;
case CMD_DELETE:
- if (fdwroutine->ExecForeignDelete == NULL)
+ if (fdwroutine->ExecForeignDelete == NULL &&
+ !resultRelInfo->ri_FdwPushdown)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot delete from foreign table \"%s\"",
@@ -1245,6 +1249,7 @@ InitResultRelInfo(ResultRelInfo *resultRelInfo,
else
resultRelInfo->ri_FdwRoutine = NULL;
resultRelInfo->ri_FdwState = NULL;
+ resultRelInfo->ri_FdwPushdown = false;
resultRelInfo->ri_ConstraintExprs = NULL;
resultRelInfo->ri_junkFilter = NULL;
resultRelInfo->ri_projectReturning = NULL;
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index 388c922..2cfe256 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -48,7 +48,10 @@ ForeignNext(ForeignScanState *node)
/* Call the Iterate function in short-lived context */
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
- slot = node->fdwroutine->IterateForeignScan(node);
+ if (plan->operation != CMD_SELECT)
+ slot = node->fdwroutine->IterateDMLPushdown(node);
+ else
+ slot = node->fdwroutine->IterateForeignScan(node);
MemoryContextSwitchTo(oldcontext);
/*
@@ -226,7 +229,10 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
/*
* Tell the FDW to initialize the scan.
*/
- fdwroutine->BeginForeignScan(scanstate, eflags);
+ if (node->operation != CMD_SELECT)
+ fdwroutine->BeginDMLPushdown(scanstate, eflags);
+ else
+ fdwroutine->BeginForeignScan(scanstate, eflags);
return scanstate;
}
@@ -240,8 +246,13 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
void
ExecEndForeignScan(ForeignScanState *node)
{
+ ForeignScan *plan = (ForeignScan *) node->ss.ps.plan;
+
/* Let the FDW shut down */
- node->fdwroutine->EndForeignScan(node);
+ if (plan->operation != CMD_SELECT)
+ node->fdwroutine->EndDMLPushdown(node);
+ else
+ node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 27051e8..d87cd8a 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -138,13 +138,17 @@ ExecCheckPlanOutput(Relation resultRel, List *targetList)
* tupleSlot: slot holding tuple actually inserted/updated/deleted
* planSlot: slot holding tuple returned by top subplan node
*
+ * Note: If tupleSlot is NULL, the FDW should have already provided econtext's
+ * scan tuple.
+ *
* Returns a slot holding the result tuple
*/
static TupleTableSlot *
-ExecProcessReturning(ProjectionInfo *projectReturning,
+ExecProcessReturning(ResultRelInfo *resultRelInfo,
TupleTableSlot *tupleSlot,
TupleTableSlot *planSlot)
{
+ ProjectionInfo *projectReturning = resultRelInfo->ri_projectReturning;
ExprContext *econtext = projectReturning->pi_exprContext;
/*
@@ -154,7 +158,20 @@ ExecProcessReturning(ProjectionInfo *projectReturning,
ResetExprContext(econtext);
/* Make tuple and any needed join variables available to ExecProject */
- econtext->ecxt_scantuple = tupleSlot;
+ if (tupleSlot)
+ econtext->ecxt_scantuple = tupleSlot;
+ else
+ {
+ HeapTuple tuple;
+
+ /*
+ * RETURNING expressions might reference the tableoid column, so
+ * initialize t_tableOid before evaluating them.
+ */
+ Assert(!TupIsNull(econtext->ecxt_scantuple));
+ tuple = ExecMaterializeSlot(econtext->ecxt_scantuple);
+ tuple->t_tableOid = RelationGetRelid(resultRelInfo->ri_RelationDesc);
+ }
econtext->ecxt_outertuple = planSlot;
/* Compute the RETURNING expressions */
@@ -496,8 +513,7 @@ ExecInsert(ModifyTableState *mtstate,
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
- return ExecProcessReturning(resultRelInfo->ri_projectReturning,
- slot, planSlot);
+ return ExecProcessReturning(resultRelInfo, slot, planSlot);
return NULL;
}
@@ -738,8 +754,7 @@ ldelete:;
ExecStoreTuple(&deltuple, slot, InvalidBuffer, false);
}
- rslot = ExecProcessReturning(resultRelInfo->ri_projectReturning,
- slot, planSlot);
+ rslot = ExecProcessReturning(resultRelInfo, slot, planSlot);
/*
* Before releasing the target tuple again, make sure rslot has a
@@ -1024,8 +1039,7 @@ lreplace:;
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
- return ExecProcessReturning(resultRelInfo->ri_projectReturning,
- slot, planSlot);
+ return ExecProcessReturning(resultRelInfo, slot, planSlot);
return NULL;
}
@@ -1380,6 +1394,21 @@ ExecModifyTable(ModifyTableState *node)
break;
}
+ /*
+ * If ri_FdwPushdown is true, all we need to do here is compute the
+ * RETURNING expressions.
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ Assert(resultRelInfo->ri_projectReturning);
+
+ /* No need to provide scan tuple to ExecProcessReturning. */
+ slot = ExecProcessReturning(resultRelInfo, NULL, planSlot);
+
+ estate->es_result_relation_info = saved_resultRelInfo;
+ return slot;
+ }
+
EvalPlanQualSetSlot(&node->mt_epqstate, planSlot);
slot = planSlot;
@@ -1559,10 +1588,13 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
{
subplan = (Plan *) lfirst(l);
+ /* Initialize the FdwPushdown flag */
+ resultRelInfo->ri_FdwPushdown = list_nth_int(node->fdwPushdowns, i);
+
/*
* Verify result relation is a valid target for the current operation
*/
- CheckValidResultRel(resultRelInfo->ri_RelationDesc, operation);
+ CheckValidResultRel(resultRelInfo, operation);
/*
* If there are indices on the result relation, open them and save
@@ -1583,7 +1615,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
/* Also let FDWs init themselves for foreign-table result rels */
- if (resultRelInfo->ri_FdwRoutine != NULL &&
+ if (!resultRelInfo->ri_FdwPushdown &&
+ resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, i);
@@ -1754,13 +1787,25 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
erm = ExecFindRowMark(estate, rc->rti, false);
/* build ExecAuxRowMark for each subplan */
+ resultRelInfo = mtstate->resultRelInfo;
for (i = 0; i < nplans; i++)
{
ExecAuxRowMark *aerm;
+ /*
+ * ignore subplan if the FDW pushes down the command to the remote
+ * server
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ resultRelInfo++;
+ continue;
+ }
+
subplan = mtstate->mt_plans[i]->plan;
aerm = ExecBuildAuxRowMark(erm, subplan->targetlist);
mtstate->mt_arowmarks[i] = lappend(mtstate->mt_arowmarks[i], aerm);
+ resultRelInfo++;
}
}
@@ -1821,6 +1866,16 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
ExecCheckPlanOutput(resultRelInfo->ri_RelationDesc,
subplan->targetlist);
+ /*
+ * ignore subplan if the FDW pushes down the command to the
+ * remote server
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ resultRelInfo++;
+ continue;
+ }
+
j = ExecInitJunkFilter(subplan->targetlist,
resultRelInfo->ri_RelationDesc->rd_att->tdhasoid,
ExecInitExtraTupleSlot(estate));
@@ -1910,7 +1965,8 @@ ExecEndModifyTable(ModifyTableState *node)
{
ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
- if (resultRelInfo->ri_FdwRoutine != NULL &&
+ if (!resultRelInfo->ri_FdwPushdown &&
+ resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->EndForeignModify != NULL)
resultRelInfo->ri_FdwRoutine->EndForeignModify(node->ps.state,
resultRelInfo);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index a8b79fa..98d7226 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -187,6 +187,7 @@ _copyModifyTable(const ModifyTable *from)
COPY_NODE_FIELD(withCheckOptionLists);
COPY_NODE_FIELD(returningLists);
COPY_NODE_FIELD(fdwPrivLists);
+ COPY_NODE_FIELD(fdwPushdowns);
COPY_NODE_FIELD(rowMarks);
COPY_SCALAR_FIELD(epqParam);
COPY_SCALAR_FIELD(onConflictAction);
@@ -646,6 +647,7 @@ _copyForeignScan(const ForeignScan *from)
/*
* copy remainder of node
*/
+ COPY_SCALAR_FIELD(operation);
COPY_SCALAR_FIELD(fs_server);
COPY_NODE_FIELD(fdw_exprs);
COPY_NODE_FIELD(fdw_private);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index d59b954..3a73f02 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -341,6 +341,7 @@ _outModifyTable(StringInfo str, const ModifyTable *node)
WRITE_NODE_FIELD(withCheckOptionLists);
WRITE_NODE_FIELD(returningLists);
WRITE_NODE_FIELD(fdwPrivLists);
+ WRITE_NODE_FIELD(fdwPushdowns);
WRITE_NODE_FIELD(rowMarks);
WRITE_INT_FIELD(epqParam);
WRITE_ENUM_FIELD(onConflictAction, OnConflictAction);
@@ -592,6 +593,7 @@ _outForeignScan(StringInfo str, const ForeignScan *node)
_outScanInfo(str, (const Scan *) node);
+ WRITE_ENUM_FIELD(operation, CmdType);
WRITE_OID_FIELD(fs_server);
WRITE_NODE_FIELD(fdw_exprs);
WRITE_NODE_FIELD(fdw_private);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 6c46151..d413241 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -1472,6 +1472,7 @@ _readModifyTable(void)
READ_NODE_FIELD(withCheckOptionLists);
READ_NODE_FIELD(returningLists);
READ_NODE_FIELD(fdwPrivLists);
+ READ_NODE_FIELD(fdwPushdowns);
READ_NODE_FIELD(rowMarks);
READ_INT_FIELD(epqParam);
READ_ENUM_FIELD(onConflictAction, OnConflictAction);
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 54ff7f6..0dd05ca 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -3788,6 +3788,7 @@ make_foreignscan(List *qptlist,
plan->lefttree = outer_plan;
plan->righttree = NULL;
node->scan.scanrelid = scanrelid;
+ node->operation = CMD_SELECT;
/* fs_server will be filled in by create_foreignscan_plan */
node->fs_server = InvalidOid;
node->fdw_exprs = fdw_exprs;
@@ -5064,6 +5065,7 @@ make_modifytable(PlannerInfo *root,
Plan *plan = &node->plan;
double total_size;
List *fdw_private_list;
+ List *fdwpushdown_list;
ListCell *subnode;
ListCell *lc;
int i;
@@ -5144,12 +5146,14 @@ make_modifytable(PlannerInfo *root,
* construct private plan data, and accumulate it all into a list.
*/
fdw_private_list = NIL;
+ fdwpushdown_list = NIL;
i = 0;
foreach(lc, resultRelations)
{
Index rti = lfirst_int(lc);
FdwRoutine *fdwroutine;
List *fdw_private;
+ bool fdwpushdown;
/*
* If possible, we want to get the FdwRoutine from our RelOptInfo for
@@ -5176,7 +5180,23 @@ make_modifytable(PlannerInfo *root,
fdwroutine = NULL;
}
+ /*
+ * If the target relation has any row-level triggers, we can't push
+ * down the command to the remote server.
+ */
if (fdwroutine != NULL &&
+ fdwroutine->PlanDMLPushdown != NULL &&
+ fdwroutine->BeginDMLPushdown != NULL &&
+ fdwroutine->IterateDMLPushdown != NULL &&
+ fdwroutine->EndDMLPushdown != NULL &&
+ !has_row_triggers(root, rti, operation))
+ fdwpushdown = fdwroutine->PlanDMLPushdown(root, node, rti, i);
+ else
+ fdwpushdown = false;
+ fdwpushdown_list = lappend_int(fdwpushdown_list, fdwpushdown);
+
+ if (!fdwpushdown &&
+ fdwroutine != NULL &&
fdwroutine->PlanForeignModify != NULL)
fdw_private = fdwroutine->PlanForeignModify(root, node, rti, i);
else
@@ -5185,6 +5205,7 @@ make_modifytable(PlannerInfo *root,
i++;
}
node->fdwPrivLists = fdw_private_list;
+ node->fdwPushdowns = fdwpushdown_list;
return node;
}
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 0ea9fcf..8d19b37 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -1520,3 +1520,50 @@ has_unique_index(RelOptInfo *rel, AttrNumber attno)
}
return false;
}
+
+
+/*
+ * has_row_triggers
+ *
+ * Detect whether the specified relation has any row-level triggers for event.
+ */
+bool
+has_row_triggers(PlannerInfo *root, Index rti, CmdType event)
+{
+ RangeTblEntry *rte = planner_rt_fetch(rti, root);
+ Relation relation;
+ TriggerDesc *trigDesc;
+ bool result = false;
+
+ /* Assume we already have adequate lock */
+ relation = heap_open(rte->relid, NoLock);
+
+ trigDesc = relation->trigdesc;
+ switch (event)
+ {
+ case CMD_INSERT:
+ if (trigDesc &&
+ (trigDesc->trig_insert_after_row ||
+ trigDesc->trig_insert_before_row))
+ result = true;
+ break;
+ case CMD_UPDATE:
+ if (trigDesc &&
+ (trigDesc->trig_update_after_row ||
+ trigDesc->trig_update_before_row))
+ result = true;
+ break;
+ case CMD_DELETE:
+ if (trigDesc &&
+ (trigDesc->trig_delete_after_row ||
+ trigDesc->trig_delete_before_row))
+ result = true;
+ break;
+ default:
+ elog(ERROR, "unrecognized CmdType: %d", (int) event);
+ break;
+ }
+
+ heap_close(relation, NoLock);
+ return result;
+}
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 1a44085..fd12df4 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -184,7 +184,7 @@ extern void ExecutorEnd(QueryDesc *queryDesc);
extern void standard_ExecutorEnd(QueryDesc *queryDesc);
extern void ExecutorRewind(QueryDesc *queryDesc);
extern bool ExecCheckRTPerms(List *rangeTable, bool ereport_on_violation);
-extern void CheckValidResultRel(Relation resultRel, CmdType operation);
+extern void CheckValidResultRel(ResultRelInfo *resultRelInfo, CmdType operation);
extern void InitResultRelInfo(ResultRelInfo *resultRelInfo,
Relation resultRelationDesc,
Index resultRelationIndex,
diff --git a/src/include/foreign/fdwapi.h b/src/include/foreign/fdwapi.h
index 9fafab0..e150bea 100644
--- a/src/include/foreign/fdwapi.h
+++ b/src/include/foreign/fdwapi.h
@@ -94,6 +94,18 @@ typedef void (*EndForeignModify_function) (EState *estate,
typedef int (*IsForeignRelUpdatable_function) (Relation rel);
+typedef bool (*PlanDMLPushdown_function) (PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+
+typedef void (*BeginDMLPushdown_function) (ForeignScanState *node,
+ int eflags);
+
+typedef TupleTableSlot *(*IterateDMLPushdown_function) (ForeignScanState *node);
+
+typedef void (*EndDMLPushdown_function) (ForeignScanState *node);
+
typedef RowMarkType (*GetForeignRowMarkType_function) (RangeTblEntry *rte,
LockClauseStrength strength);
@@ -111,6 +123,9 @@ typedef void (*ExplainForeignModify_function) (ModifyTableState *mtstate,
int subplan_index,
struct ExplainState *es);
+typedef void (*ExplainDMLPushdown_function) (ForeignScanState *node,
+ struct ExplainState *es);
+
typedef int (*AcquireSampleRowsFunc) (Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
@@ -171,6 +186,10 @@ typedef struct FdwRoutine
ExecForeignDelete_function ExecForeignDelete;
EndForeignModify_function EndForeignModify;
IsForeignRelUpdatable_function IsForeignRelUpdatable;
+ PlanDMLPushdown_function PlanDMLPushdown;
+ BeginDMLPushdown_function BeginDMLPushdown;
+ IterateDMLPushdown_function IterateDMLPushdown;
+ EndDMLPushdown_function EndDMLPushdown;
/* Functions for SELECT FOR UPDATE/SHARE row locking */
GetForeignRowMarkType_function GetForeignRowMarkType;
@@ -180,6 +199,7 @@ typedef struct FdwRoutine
/* Support functions for EXPLAIN */
ExplainForeignScan_function ExplainForeignScan;
ExplainForeignModify_function ExplainForeignModify;
+ ExplainDMLPushdown_function ExplainDMLPushdown;
/* Support functions for ANALYZE */
AnalyzeForeignTable_function AnalyzeForeignTable;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 064a050..fe160db 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -311,6 +311,7 @@ typedef struct JunkFilter
* TrigInstrument optional runtime measurements for triggers
* FdwRoutine FDW callback functions, if foreign table
* FdwState available to save private state of FDW
+ * FdwPushdown true when the command is pushed down
* WithCheckOptions list of WithCheckOption's to be checked
* WithCheckOptionExprs list of WithCheckOption expr states
* ConstraintExprs array of constraint-checking expr states
@@ -334,6 +335,7 @@ typedef struct ResultRelInfo
Instrumentation *ri_TrigInstrument;
struct FdwRoutine *ri_FdwRoutine;
void *ri_FdwState;
+ bool ri_FdwPushdown;
List *ri_WithCheckOptions;
List *ri_WithCheckOptionExprs;
List **ri_ConstraintExprs;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 55d6bbe..4858c79 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -189,6 +189,7 @@ typedef struct ModifyTable
List *withCheckOptionLists; /* per-target-table WCO lists */
List *returningLists; /* per-target-table RETURNING tlists */
List *fdwPrivLists; /* per-target-table FDW private data lists */
+ List *fdwPushdowns; /* per-target-table FDW pushdown flags */
List *rowMarks; /* PlanRowMarks (non-locking only) */
int epqParam; /* ID of Param for EvalPlanQual re-eval */
OnConflictAction onConflictAction; /* ON CONFLICT action */
@@ -531,6 +532,7 @@ typedef struct WorkTableScan
typedef struct ForeignScan
{
Scan scan;
+ CmdType operation; /* SELECT/INSERT/UPDATE/DELETE */
Oid fs_server; /* OID of foreign server */
List *fdw_exprs; /* expressions that FDW may evaluate */
List *fdw_private; /* private data for FDW */
diff --git a/src/include/optimizer/plancat.h b/src/include/optimizer/plancat.h
index 52335fa..125274e 100644
--- a/src/include/optimizer/plancat.h
+++ b/src/include/optimizer/plancat.h
@@ -55,4 +55,6 @@ extern Selectivity join_selectivity(PlannerInfo *root,
JoinType jointype,
SpecialJoinInfo *sjinfo);
+extern bool has_row_triggers(PlannerInfo *root, Index rti, CmdType event);
+
#endif /* PLANCAT_H */
Fujita-san, I am attaching update version of the patch, which added
the documentation update.
Once we finalize this, I feel good with the patch and think that we
could mark this as ready for committer.
On Fri, Feb 5, 2016 at 5:33 PM, Rushabh Lathia <rushabh.lathia@gmail.com>
wrote:
On Fri, Feb 5, 2016 at 4:46 PM, Rushabh Lathia <rushabh.lathia@gmail.com>
wrote:On Wed, Feb 3, 2016 at 3:31 PM, Etsuro Fujita <
fujita.etsuro@lab.ntt.co.jp> wrote:On 2016/01/28 15:20, Rushabh Lathia wrote:
On Thu, Jan 28, 2016 at 11:33 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp <mailto:fujita.etsuro@lab.ntt.co.jp>>
wrote:On 2016/01/27 21:23, Rushabh Lathia wrote:
If I understood correctly, above documentation means, that if
FDW have
DMLPushdown APIs that is enough. But in reality thats not the
case, we
need ExecForeignInsert, ExecForeignUpdate, or ExecForeignDelete
in case
DML is not pushable.And here fact is DMLPushdown APIs are optional for FDW, so that
if FDW
don't have DMLPushdown APIs they can still very well perform
the DML
operations using ExecForeignInsert, ExecForeignUpdate, or
ExecForeignDelete.So documentation should be like:
If the IsForeignRelUpdatable pointer is set to NULL, foreign
tables are
assumed to be insertable, updatable, or deletable if the FDW
provides
ExecForeignInsert, ExecForeignUpdate, or ExecForeignDelete
respectively,If FDW provides DMLPushdown APIs and the DML are pushable to the
foreign
server, then FDW still needs ExecForeignInsert,
ExecForeignUpdate, or
ExecForeignDelete for the non-pushable DML operation.What's your opinion ?
I agree that we should add this to the documentation, too.
I added docs to the IsForeignRelUpdatable documentation. Also, a brief
introductory remark has been added at the beginning of the DML pushdown
APIs' documentation.BTW, if I understand correctly, I think we should also modify
relation_is_updatabale() accordingly. Am I right?
Yep, we need to modify relation_is_updatable().
I thought I'd modify that function in the same way as
CheckValidResultRel(), but I noticed that we cannot do that, because we
don't have any information on whether each update is pushed down to the
remote server by PlanDMLPushdown, during relation_is_updatabale().Sorry I didn't get you here. Can't resultRelInfo->ri_FdwPushdown gives
information update whether update is pushed down safe or not ? What my
concern here is, lets say resultRelInfo->ri_FdwPushdown marked as true
(PlanDMLPushdown return true), but later into CheckValidResultRel() it
found out that missing BeginDMLPushdown, IterateDMLPushdown and
EndDMLPushdown APIs and it will end up with error.What I think CheckValidResultRel() should do is, if
resultRelInfo->ri_FdwPushdown is true then check for the DMLPushdown API
and if it doesn't find those API then check for traditional APIs
(ExecForeignInsert, ExecForeignUpdate or ExecForeignDelete). And when it
doesn't find both then it should return an error.I changed CheckValidResultRel(), where
1) Don't throw an error if resultRelInfo->ri_FdwPushdown is true and
DMLPushdown APIs are missing as query can still perform operation with
traditional ExecForeign APIs. So in this situation just marked
resultRelInfo->ri_FdwPushdown to false.(Wondering can we add the checks for DMLPushdown APIs into
PlanDMLPushdown as additional check? Means PlanDMLPushdown should return
true only if FDW provides the BeginDMLPushdown & IterateDMLPushdown &
EndDMLPushdown APIs ? What you say ?)On another thought, we should not give responsibility to check for the
APIs to the FDW. So may be we should call PlanDMLPushdown only if
BeginDMLPushdown & IterateDMLPushdown & EndDMLPushdown APIs are present
into FDW. That means prepare DMLPushdown plan only when all the required
APIs are available with FDW. This will also reduce the changes into
CheckValidResultRel().Thanks Ashutosh Bapat for healthy discussion.
PFA patch.
2) Don't throw an error if resultRelInfo->ri_FdwPushdown is true and
DMLPushdown APIs is present but ExecForeign APIs are missing.
3) Throw an error if resultRelInfo->ri_FdwPushdown is false and
ExecForeign APIs are missing.Attaching is the WIP patch here, do share your thought.
(need to apply on top of V6 patch)So, I left that function as-is. relation_is_updatabale() is just used
for display in the information_schema views, so ISTM that that function is
fine as-is. (As for CheckValidResultRel(), I revised it so as to check the
presence of DML pushdown APIs after checking the existing APIs if the given
command will be pushed down. The reason is because we assume the presence
of the existing APIs, anyway.)I revised other docs and some comments, mostly for consistency.
Attached is an updated version of the patch, which has been created on
top of the updated version of the bugfix patch posted by Robert in [1]
(attached).Best regards,
Etsuro Fujita[1]
/messages/by-id/CA+TgmoZ40j2uC5aC1NXu03oj4CrVOLkS15XX+PTFP-1U-8zR1Q@mail.gmail.com--
Rushabh Lathia--
Rushabh Lathia
--
Rushabh Lathia
Attachments:
fdw-dml-pushdown-v7_update_doc.patchtext/x-diff; charset=US-ASCII; name=fdw-dml-pushdown-v7_update_doc.patchDownload
diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c
index d778e61..157de59 100644
--- a/contrib/postgres_fdw/deparse.c
+++ b/contrib/postgres_fdw/deparse.c
@@ -1069,6 +1069,66 @@ deparseUpdateSql(StringInfo buf, PlannerInfo *root,
}
/*
+ * deparse remote UPDATE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+void
+deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+{
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ deparse_expr_cxt context;
+ int nestlevel;
+ bool first;
+ ListCell *lc;
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = params_list;
+
+ appendStringInfoString(buf, "UPDATE ");
+ deparseRelation(buf, rel);
+ appendStringInfoString(buf, " SET ");
+
+ /* Make sure any constants in the exprs are printed portably */
+ nestlevel = set_transmission_modes();
+
+ first = true;
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(targetlist, attnum);
+
+ if (!first)
+ appendStringInfoString(buf, ", ");
+ first = false;
+
+ deparseColumnRef(buf, rtindex, attnum, root);
+ appendStringInfoString(buf, " = ");
+ deparseExpr((Expr *) tle->expr, &context);
+ }
+
+ reset_transmission_modes(nestlevel);
+
+ if (remote_conds)
+ appendWhereClause(remote_conds, &context);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+}
+
+/*
* deparse remote DELETE statement
*
* The statement text is appended to buf, and we also create an integer List
@@ -1091,6 +1151,40 @@ deparseDeleteSql(StringInfo buf, PlannerInfo *root,
}
/*
+ * deparse remote DELETE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+void
+deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+{
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ deparse_expr_cxt context;
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = params_list;
+
+ appendStringInfoString(buf, "DELETE FROM ");
+ deparseRelation(buf, rel);
+
+ if (remote_conds)
+ appendWhereClause(remote_conds, &context);
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+}
+
+/*
* Add a RETURNING clause, if needed, to an INSERT/UPDATE/DELETE.
*/
static void
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index f621024..ed9ac3e 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -1314,7 +1314,26 @@ INSERT INTO ft2 (c1,c2,c3)
(3 rows)
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+EXPLAIN (verbose, costs off)
+UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 300), c3 = (c3 || '_update3'::text) WHERE ((("C 1" % 10) = 3))
+(3 rows)
+
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+EXPLAIN (verbose, costs off)
+UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Output: c1, c2, c3, c4, c5, c6, c7, c8
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 400), c3 = (c3 || '_update7'::text) WHERE ((("C 1" % 10) = 7)) RETURNING "C 1", c2, c3, c4, c5, c6, c7, c8
+(4 rows)
+
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
c1 | c2 | c3 | c4 | c5 | c6 | c7 | c8
------+-----+--------------------+------------------------------+--------------------------+----+------------+-----
@@ -1424,7 +1443,7 @@ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
- FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
+ FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
@@ -1445,16 +1464,14 @@ UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
- DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
- QUERY PLAN
-----------------------------------------------------------------------------------------
+ DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
+ QUERY PLAN
+--------------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
- Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1 RETURNING "C 1", c4
- -> Foreign Scan on public.ft2
- Output: ctid
- Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) FOR UPDATE
-(6 rows)
+ -> Foreign Delete on public.ft2
+ Remote SQL: DELETE FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) RETURNING "C 1", c4
+(4 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
@@ -1565,7 +1582,7 @@ DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
(103 rows)
EXPLAIN (verbose, costs off)
-DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
+DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
@@ -2426,16 +2443,14 @@ INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
(1 row)
EXPLAIN (verbose, costs off)
-UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
- QUERY PLAN
--------------------------------------------------------------------------------------------------------------------
+UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
+ QUERY PLAN
+------------------------------------------------------------------------------------
Update on public.ft2
Output: (tableoid)::regclass
- Remote SQL: UPDATE "S 1"."T 1" SET c3 = $2 WHERE ctid = $1
- -> Foreign Scan on public.ft2
- Output: c1, c2, NULL::integer, 'bar'::text, c4, c5, c6, c7, c8, ctid
- Remote SQL: SELECT "C 1", c2, c4, c5, c6, c7, c8, ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
-(6 rows)
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c3 = 'bar'::text WHERE (("C 1" = 9999))
+(4 rows)
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
@@ -2444,16 +2459,14 @@ UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
(1 row)
EXPLAIN (verbose, costs off)
-DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
- QUERY PLAN
-------------------------------------------------------------------------------------
+DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
+ QUERY PLAN
+--------------------------------------------------------------------
Delete on public.ft2
Output: (tableoid)::regclass
- Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1
- -> Foreign Scan on public.ft2
- Output: ctid
- Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
-(6 rows)
+ -> Foreign Delete on public.ft2
+ Remote SQL: DELETE FROM "S 1"."T 1" WHERE (("C 1" = 9999))
+(4 rows)
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
@@ -2607,7 +2620,7 @@ CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
-CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
+CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
@@ -2766,7 +2779,7 @@ savepoint s3;
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
-CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
+CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (-2) WHERE ((c2 = 42)) AND (("C 1" = 10))
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
@@ -2906,7 +2919,7 @@ CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
-CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
+CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2positive;
-- But inconsistent check constraints provide inconsistent results
ALTER FOREIGN TABLE ft1 ADD CONSTRAINT ft1_c2negative CHECK (c2 < 0);
@@ -3299,6 +3312,199 @@ NOTICE: NEW: (13,"test triggered !")
(0,27)
(1 row)
+-- cleanup
+DROP TRIGGER trig_row_before ON rem1;
+DROP TRIGGER trig_row_after ON rem1;
+DROP TRIGGER trig_local_before ON loc1;
+-- Test DML pushdown functionality
+-- Test with statement-level triggers
+CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+(3 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+(3 rows)
+
+DROP TRIGGER trig_stmt_before ON rem1;
+CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+(3 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+(3 rows)
+
+DROP TRIGGER trig_stmt_after ON rem1;
+-- Test with row-level ON INSERT triggers
+CREATE TRIGGER trig_row_before_insert
+BEFORE INSERT ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+(3 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+(3 rows)
+
+DROP TRIGGER trig_row_before_insert ON rem1;
+CREATE TRIGGER trig_row_after_insert
+AFTER INSERT ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+(3 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+(3 rows)
+
+DROP TRIGGER trig_row_after_insert ON rem1;
+-- Test with row-level ON UPDATE triggers
+CREATE TRIGGER trig_row_before_update
+BEFORE UPDATE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+---------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+(5 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+(3 rows)
+
+DROP TRIGGER trig_row_before_update ON rem1;
+CREATE TRIGGER trig_row_after_update
+AFTER UPDATE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+-------------------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+(5 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+(3 rows)
+
+DROP TRIGGER trig_row_after_update ON rem1;
+-- Test with row-level ON DELETE triggers
+CREATE TRIGGER trig_row_before_delete
+BEFORE DELETE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+(3 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+---------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+(5 rows)
+
+DROP TRIGGER trig_row_before_delete ON rem1;
+CREATE TRIGGER trig_row_after_delete
+AFTER DELETE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+(3 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+------------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+(5 rows)
+
+DROP TRIGGER trig_row_after_delete ON rem1;
-- ===================================================================
-- test inheritance features
-- ===================================================================
@@ -3768,6 +3974,56 @@ fetch from c;
update bar set f2 = null where current of c;
ERROR: WHERE CURRENT OF is not supported for this table type
rollback;
+explain (verbose, costs off)
+delete from foo where f1 < 5 returning *;
+ QUERY PLAN
+--------------------------------------------------------------------------------
+ Delete on public.foo
+ Output: foo.f1, foo.f2
+ Delete on public.foo
+ Foreign Delete on public.foo2
+ -> Index Scan using i_foo_f1 on public.foo
+ Output: foo.ctid
+ Index Cond: (foo.f1 < 5)
+ -> Foreign Delete on public.foo2
+ Remote SQL: DELETE FROM public.loct1 WHERE ((f1 < 5)) RETURNING f1, f2
+(9 rows)
+
+delete from foo where f1 < 5 returning *;
+ f1 | f2
+----+----
+ 1 | 1
+ 3 | 3
+ 0 | 0
+ 2 | 2
+ 4 | 4
+(5 rows)
+
+explain (verbose, costs off)
+update bar set f2 = f2 + 100 returning *;
+ QUERY PLAN
+------------------------------------------------------------------------------
+ Update on public.bar
+ Output: bar.f1, bar.f2
+ Update on public.bar
+ Foreign Update on public.bar2
+ -> Seq Scan on public.bar
+ Output: bar.f1, (bar.f2 + 100), bar.ctid
+ -> Foreign Update on public.bar2
+ Remote SQL: UPDATE public.loct2 SET f2 = (f2 + 100) RETURNING f1, f2
+(8 rows)
+
+update bar set f2 = f2 + 100 returning *;
+ f1 | f2
+----+-----
+ 1 | 311
+ 2 | 322
+ 6 | 266
+ 3 | 333
+ 4 | 344
+ 7 | 277
+(6 rows)
+
drop table foo cascade;
NOTICE: drop cascades to foreign table foo2
drop table bar cascade;
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index d5a2af9..a292dbb 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -61,6 +61,8 @@ enum FdwScanPrivateIndex
{
/* SQL statement to execute remotely (as a String node) */
FdwScanPrivateSelectSql,
+ /* List of restriction clauses that can be executed remotely */
+ FdwScanPrivateRemoteConds,
/* Integer list of attribute numbers retrieved by the SELECT */
FdwScanPrivateRetrievedAttrs,
/* Integer representing the desired fetch_size */
@@ -90,6 +92,28 @@ enum FdwModifyPrivateIndex
};
/*
+ * Similarly, this enum describes what's kept in the fdw_private list for
+ * a ForeignScan node that has pushed down an UPDATE/DELETE to the remote
+ * server. We store:
+ *
+ * 1) UPDATE/DELETE statement text to be sent to the remote server
+ * 2) Boolean flag showing if the remote query has a RETURNING clause
+ * 3) Integer list of attribute numbers retrieved by RETURNING, if any
+ * 4) Boolean flag showing if we set the command es_processed
+ */
+enum FdwDmlPushdownPrivateIndex
+{
+ /* SQL statement to execute remotely (as a String node) */
+ FdwDmlPushdownPrivateUpdateSql,
+ /* has-returning flag (as an integer Value node) */
+ FdwDmlPushdownPrivateHasReturning,
+ /* Integer list of attribute numbers retrieved by RETURNING */
+ FdwDmlPushdownPrivateRetrievedAttrs,
+ /* set-processed flag (as an integer Value node) */
+ FdwDmlPushdownPrivateSetProcessed
+};
+
+/*
* Execution state of a foreign scan using postgres_fdw.
*/
typedef struct PgFdwScanState
@@ -154,6 +178,37 @@ typedef struct PgFdwModifyState
} PgFdwModifyState;
/*
+ * Execution state of a foreign scan that has pushed down a foreign table
+ * modification to the remote server
+ */
+typedef struct PgFdwDmlPushdownState
+{
+ Relation rel; /* relcache entry for the foreign table */
+ AttInMetadata *attinmeta; /* attribute datatype conversion metadata */
+
+ /* extracted fdw_private data */
+ char *query; /* text of UPDATE/DELETE command */
+ bool has_returning; /* is there a RETURNING clause? */
+ List *retrieved_attrs; /* attr numbers retrieved by RETURNING */
+ bool set_processed; /* do we set the command es_processed? */
+
+ /* for remote query execution */
+ PGconn *conn; /* connection for the update */
+ int numParams; /* number of parameters passed to query */
+ FmgrInfo *param_flinfo; /* output conversion functions for them */
+ List *param_exprs; /* executable expressions for param values */
+ const char **param_values; /* textual values of query parameters */
+
+ /* for storing result tuples */
+ PGresult *result; /* result for query */
+ int num_tuples; /* # of result tuples */
+ int next_tuple; /* index of next one to return */
+
+ /* working memory context */
+ MemoryContext temp_cxt; /* context for per-tuple temporary data */
+} PgFdwDmlPushdownState;
+
+/*
* Workspace for analyzing a foreign table.
*/
typedef struct PgFdwAnalyzeState
@@ -245,6 +300,13 @@ static TupleTableSlot *postgresExecForeignDelete(EState *estate,
static void postgresEndForeignModify(EState *estate,
ResultRelInfo *resultRelInfo);
static int postgresIsForeignRelUpdatable(Relation rel);
+static bool postgresPlanDMLPushdown(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+static void postgresBeginDMLPushdown(ForeignScanState *node, int eflags);
+static TupleTableSlot *postgresIterateDMLPushdown(ForeignScanState *node);
+static void postgresEndDMLPushdown(ForeignScanState *node);
static void postgresExplainForeignScan(ForeignScanState *node,
ExplainState *es);
static void postgresExplainForeignModify(ModifyTableState *mtstate,
@@ -252,6 +314,8 @@ static void postgresExplainForeignModify(ModifyTableState *mtstate,
List *fdw_private,
int subplan_index,
ExplainState *es);
+static void postgresExplainDMLPushdown(ForeignScanState *node,
+ ExplainState *es);
static bool postgresAnalyzeForeignTable(Relation relation,
AcquireSampleRowsFunc *func,
BlockNumber *totalpages);
@@ -288,6 +352,8 @@ static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
TupleTableSlot *slot);
static void store_returning_result(PgFdwModifyState *fmstate,
TupleTableSlot *slot, PGresult *res);
+static void execute_dml_stmt(ForeignScanState *node);
+static TupleTableSlot *get_returning_data(ForeignScanState *node);
static int postgresAcquireSampleRowsFunc(Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
@@ -330,10 +396,15 @@ postgres_fdw_handler(PG_FUNCTION_ARGS)
routine->ExecForeignDelete = postgresExecForeignDelete;
routine->EndForeignModify = postgresEndForeignModify;
routine->IsForeignRelUpdatable = postgresIsForeignRelUpdatable;
+ routine->PlanDMLPushdown = postgresPlanDMLPushdown;
+ routine->BeginDMLPushdown = postgresBeginDMLPushdown;
+ routine->IterateDMLPushdown = postgresIterateDMLPushdown;
+ routine->EndDMLPushdown = postgresEndDMLPushdown;
/* Support functions for EXPLAIN */
routine->ExplainForeignScan = postgresExplainForeignScan;
routine->ExplainForeignModify = postgresExplainForeignModify;
+ routine->ExplainDMLPushdown = postgresExplainDMLPushdown;
/* Support functions for ANALYZE */
routine->AnalyzeForeignTable = postgresAnalyzeForeignTable;
@@ -1011,7 +1082,8 @@ postgresGetForeignPlan(PlannerInfo *root,
* Build the fdw_private list that will be available to the executor.
* Items in the list must match enum FdwScanPrivateIndex, above.
*/
- fdw_private = list_make3(makeString(sql.data),
+ fdw_private = list_make4(makeString(sql.data),
+ remote_conds,
retrieved_attrs,
makeInteger(fpinfo->fetch_size));
@@ -1308,13 +1380,6 @@ postgresAddForeignUpdateTargets(Query *parsetree,
/*
* postgresPlanForeignModify
* Plan an insert/update/delete operation on a foreign table
- *
- * Note: currently, the plan tree generated for UPDATE/DELETE will always
- * include a ForeignScan that retrieves ctids (using SELECT FOR UPDATE)
- * and then the ModifyTable node will have to execute individual remote
- * UPDATE/DELETE commands. If there are no local conditions or joins
- * needed, it'd be better to let the scan node do UPDATE/DELETE RETURNING
- * and then do nothing at ModifyTable. Room for future optimization ...
*/
static List *
postgresPlanForeignModify(PlannerInfo *root,
@@ -1825,6 +1890,337 @@ postgresIsForeignRelUpdatable(Relation rel)
}
/*
+ * postgresPlanDMLPushdown
+ * Consider pushing down a foreign table modification to the remote server
+ *
+ * Decide whether the table modification is safe to push down to the remote end,
+ * and if so, modify subplan so as to do that.
+ *
+ * Conditions checked here:
+ *
+ * 1. The table modification must be an UPDATE or DELETE.
+ *
+ * 2. It's unsafe to push down the command if there are any local joins needed.
+ *
+ * 3. It's unsafe to push down the command if there are any quals that can't be
+ * evaluated remotely.
+ *
+ * 4. We can't push down an UPDATE, if any expressions to assign to the target
+ * columns are unsafe to evaluate on the remote end.
+ */
+static bool
+postgresPlanDMLPushdown(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index)
+{
+ CmdType operation = plan->operation;
+ Plan *subplan = (Plan *) list_nth(plan->plans, subplan_index);
+ RangeTblEntry *rte = planner_rt_fetch(resultRelation, root);
+ Relation rel;
+ StringInfoData sql;
+ ForeignScan *fscan;
+ List *targetAttrs = NIL;
+ List *remote_conds;
+ List *params_list = NIL;
+ List *returningList = NIL;
+ List *retrieved_attrs = NIL;
+
+ /*
+ * Decide whether the table modification is safe to push down to the remote
+ * server.
+ */
+
+ /* Check point 1 */
+ if (operation == CMD_INSERT)
+ return false;
+
+ /* Check point 2 */
+ if (nodeTag(subplan) != T_ForeignScan)
+ return false;
+
+ /* Check point 3 */
+ if (subplan->qual != NIL)
+ return false;
+
+ /* Check point 4 */
+ if (operation == CMD_UPDATE)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[resultRelation];
+ int col;
+
+ /*
+ * We transmit only columns that were explicitly targets of the UPDATE,
+ * so as to avoid unnecessary data transmission.
+ */
+ col = -1;
+ while ((col = bms_next_member(rte->updatedCols, col)) >= 0)
+ {
+ /* bit numbers are offset by FirstLowInvalidHeapAttributeNumber */
+ AttrNumber attno = col + FirstLowInvalidHeapAttributeNumber;
+ TargetEntry *tle;
+
+ if (attno <= InvalidAttrNumber) /* shouldn't happen */
+ elog(ERROR, "system-column update is not supported");
+
+ tle = get_tle_by_resno(subplan->targetlist, attno);
+
+ if (!is_foreign_expr(root, baserel, (Expr *) tle->expr))
+ return false;
+
+ targetAttrs = lappend_int(targetAttrs, attno);
+ }
+ }
+
+ /*
+ * Ok, modify subplan so as to push down the command to the remote server.
+ */
+ fscan = (ForeignScan *) subplan;
+
+ initStringInfo(&sql);
+
+ /*
+ * Core code already has some lock on each rel being planned, so we can
+ * use NoLock here.
+ */
+ rel = heap_open(rte->relid, NoLock);
+
+ /*
+ * Extract the baserestrictinfo clauses that can be evaluated remotely.
+ */
+ remote_conds = (List *) list_nth(fscan->fdw_private,
+ FdwScanPrivateRemoteConds);
+
+ /*
+ * Extract the relevant RETURNING list if any.
+ */
+ if (plan->returningLists)
+ returningList = (List *) list_nth(plan->returningLists, subplan_index);
+
+ /*
+ * Construct the SQL command string.
+ */
+ switch (operation)
+ {
+ case CMD_UPDATE:
+ deparsePushedDownUpdateSql(&sql, root, resultRelation, rel,
+ ((Plan *) fscan)->targetlist,
+ targetAttrs,
+ remote_conds, ¶ms_list,
+ returningList, &retrieved_attrs);
+ break;
+ case CMD_DELETE:
+ deparsePushedDownDeleteSql(&sql, root, resultRelation, rel,
+ remote_conds, ¶ms_list,
+ returningList, &retrieved_attrs);
+ break;
+ default:
+ elog(ERROR, "unexpected operation: %d", (int) operation);
+ break;
+ }
+
+ /*
+ * Update the operation info.
+ */
+ fscan->operation = operation;
+
+ /*
+ * Update the fdw_exprs list that will be available to the executor.
+ */
+ fscan->fdw_exprs = params_list;
+
+ /*
+ * Update the fdw_private list that will be available to the executor.
+ * Items in the list must match enum FdwDmlPushdownPrivateIndex, above.
+ */
+ fscan->fdw_private = list_make4(makeString(sql.data),
+ makeInteger((retrieved_attrs != NIL)),
+ retrieved_attrs,
+ makeInteger(plan->canSetTag));
+
+ heap_close(rel, NoLock);
+ return true;
+}
+
+/*
+ * postgresBeginDMLPushdown
+ * Initiate pushing down a foreign table modification to the remote server
+ */
+static void
+postgresBeginDMLPushdown(ForeignScanState *node, int eflags)
+{
+ ForeignScan *fsplan = (ForeignScan *) node->ss.ps.plan;
+ EState *estate = node->ss.ps.state;
+ PgFdwDmlPushdownState *dpstate;
+ RangeTblEntry *rte;
+ Oid userid;
+ ForeignTable *table;
+ UserMapping *user;
+ int numParams;
+ int i;
+ ListCell *lc;
+
+ /*
+ * Do nothing in EXPLAIN (no ANALYZE) case. node->fdw_state stays NULL.
+ */
+ if (eflags & EXEC_FLAG_EXPLAIN_ONLY)
+ return;
+
+ /*
+ * We'll save private state in node->fdw_state.
+ */
+ dpstate = (PgFdwDmlPushdownState *) palloc0(sizeof(PgFdwDmlPushdownState));
+ node->fdw_state = (void *) dpstate;
+
+ /*
+ * Identify which user to do the remote access as. This should match what
+ * ExecCheckRTEPerms() does.
+ */
+ rte = rt_fetch(fsplan->scan.scanrelid, estate->es_range_table);
+ userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
+
+ /* Get info about foreign table. */
+ dpstate->rel = node->ss.ss_currentRelation;
+ table = GetForeignTable(RelationGetRelid(dpstate->rel));
+ user = GetUserMapping(userid, table->serverid);
+
+ /*
+ * Get connection to the foreign server. Connection manager will
+ * establish new connection if necessary.
+ */
+ dpstate->conn = GetConnection(user, false);
+
+ /* Initialize state variable */
+ dpstate->num_tuples = -1; /* -1 means not set yet */
+
+ /* Get private info created by planner functions. */
+ dpstate->query = strVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateUpdateSql));
+ dpstate->has_returning = intVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateHasReturning));
+ dpstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateRetrievedAttrs);
+ dpstate->set_processed = intVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateSetProcessed));
+
+ /* Create context for per-tuple temp workspace. */
+ dpstate->temp_cxt = AllocSetContextCreate(estate->es_query_cxt,
+ "postgres_fdw temporary data",
+ ALLOCSET_SMALL_MINSIZE,
+ ALLOCSET_SMALL_INITSIZE,
+ ALLOCSET_SMALL_MAXSIZE);
+
+ /* Prepare for input conversion of RETURNING results. */
+ if (dpstate->has_returning)
+ dpstate->attinmeta = TupleDescGetAttInMetadata(RelationGetDescr(dpstate->rel));
+
+ /* Prepare for output conversion of parameters used in remote query. */
+ numParams = list_length(fsplan->fdw_exprs);
+ dpstate->numParams = numParams;
+ dpstate->param_flinfo = (FmgrInfo *) palloc0(sizeof(FmgrInfo) * numParams);
+
+ i = 0;
+ foreach(lc, fsplan->fdw_exprs)
+ {
+ Node *param_expr = (Node *) lfirst(lc);
+ Oid typefnoid;
+ bool isvarlena;
+
+ getTypeOutputInfo(exprType(param_expr), &typefnoid, &isvarlena);
+ fmgr_info(typefnoid, &dpstate->param_flinfo[i]);
+ i++;
+ }
+
+ /*
+ * Prepare remote-parameter expressions for evaluation. (Note: in
+ * practice, we expect that all these expressions will be just Params, so
+ * we could possibly do something more efficient than using the full
+ * expression-eval machinery for this. But probably there would be little
+ * benefit, and it'd require postgres_fdw to know more than is desirable
+ * about Param evaluation.)
+ */
+ dpstate->param_exprs = (List *)
+ ExecInitExpr((Expr *) fsplan->fdw_exprs,
+ (PlanState *) node);
+
+ /*
+ * Allocate buffer for text form of query parameters, if any.
+ */
+ if (numParams > 0)
+ dpstate->param_values = (const char **) palloc0(numParams * sizeof(char *));
+ else
+ dpstate->param_values = NULL;
+}
+
+/*
+ * postgresIterateDMLPushdown
+ * Execute pushing down a foreign table modification to the remote server
+ */
+static TupleTableSlot *
+postgresIterateDMLPushdown(ForeignScanState *node)
+{
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ EState *estate = node->ss.ps.state;
+ ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
+
+ /*
+ * If this is the first call after Begin, execute the statement.
+ */
+ if (dpstate->num_tuples == -1)
+ execute_dml_stmt(node);
+
+ /*
+ * If the local query doesn't specify RETURNING, just clear tuple slot.
+ */
+ if (!resultRelInfo->ri_projectReturning)
+ {
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+ Instrumentation *instr = node->ss.ps.instrument;
+
+ Assert(!dpstate->has_returning);
+
+ /* Increment the command es_processed count if necessary. */
+ if (dpstate->set_processed)
+ estate->es_processed += dpstate->num_tuples;
+
+ /* Increment the tuple count for EXPLAIN ANALYZE if necessary. */
+ if (instr)
+ instr->tuplecount += dpstate->num_tuples;
+
+ return ExecClearTuple(slot);
+ }
+
+ /*
+ * Get the next RETURNING tuple.
+ */
+ return get_returning_data(node);
+}
+
+/*
+ * postgresEndDMLPushdown
+ * Finish pushing down a foreign table modification to the remote server
+ */
+static void
+postgresEndDMLPushdown(ForeignScanState *node)
+{
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+
+ /* if dpstate is NULL, we are in EXPLAIN; nothing to do */
+ if (dpstate == NULL)
+ return;
+
+ /* Release PGresult */
+ if (dpstate->result)
+ PQclear(dpstate->result);
+
+ /* Release remote connection */
+ ReleaseConnection(dpstate->conn);
+ dpstate->conn = NULL;
+
+ /* MemoryContext will be deleted automatically. */
+}
+
+/*
* postgresExplainForeignScan
* Produce extra output for EXPLAIN of a ForeignScan on a foreign table
*/
@@ -1862,6 +2258,25 @@ postgresExplainForeignModify(ModifyTableState *mtstate,
}
}
+/*
+ * postgresExplainDMLPushdown
+ * Produce extra output for EXPLAIN of a ForeignScan on a foreign table
+ * that has pushed down an UPDATE/DELETE to the remote server
+ */
+static void
+postgresExplainDMLPushdown(ForeignScanState *node, ExplainState *es)
+{
+ List *fdw_private;
+ char *sql;
+
+ if (es->verbose)
+ {
+ fdw_private = ((ForeignScan *) node->ss.ps.plan)->fdw_private;
+ sql = strVal(list_nth(fdw_private, FdwDmlPushdownPrivateUpdateSql));
+ ExplainPropertyText("Remote SQL", sql, es);
+ }
+}
+
/*
* estimate_path_cost_size
@@ -2474,6 +2889,139 @@ store_returning_result(PgFdwModifyState *fmstate,
}
/*
+ * Execute a pushed-down UPDATE/DELETE statement.
+ */
+static void
+execute_dml_stmt(ForeignScanState *node)
+{
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ ExprContext *econtext = node->ss.ps.ps_ExprContext;
+ int numParams = dpstate->numParams;
+ const char **values = dpstate->param_values;
+
+ /*
+ * Construct array of query parameter values in text format.
+ */
+ if (numParams > 0)
+ {
+ int nestlevel;
+ int i;
+ ListCell *lc;
+
+ nestlevel = set_transmission_modes();
+
+ i = 0;
+ foreach(lc, dpstate->param_exprs)
+ {
+ ExprState *expr_state = (ExprState *) lfirst(lc);
+ Datum expr_value;
+ bool isNull;
+
+ /* Evaluate the parameter expression */
+ expr_value = ExecEvalExpr(expr_state, econtext, &isNull, NULL);
+
+ /*
+ * Get string representation of each parameter value by invoking
+ * type-specific output function, unless the value is null.
+ */
+ if (isNull)
+ values[i] = NULL;
+ else
+ values[i] = OutputFunctionCall(&dpstate->param_flinfo[i],
+ expr_value);
+ i++;
+ }
+
+ reset_transmission_modes(nestlevel);
+ }
+
+ /*
+ * Notice that we pass NULL for paramTypes, thus forcing the remote server
+ * to infer types for all parameters. Since we explicitly cast every
+ * parameter (see deparse.c), the "inference" is trivial and will produce
+ * the desired result. This allows us to avoid assuming that the remote
+ * server has the same OIDs we do for the parameters' types.
+ *
+ * We don't use a PG_TRY block here, so be careful not to throw error
+ * without releasing the PGresult.
+ */
+ dpstate->result = PQexecParams(dpstate->conn, dpstate->query,
+ numParams, NULL, values, NULL, NULL, 0);
+ if (PQresultStatus(dpstate->result) !=
+ (dpstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
+ pgfdw_report_error(ERROR, dpstate->result, dpstate->conn, true,
+ dpstate->query);
+
+ /* Get the number of rows affected. */
+ if (dpstate->has_returning)
+ dpstate->num_tuples = PQntuples(dpstate->result);
+ else
+ dpstate->num_tuples = atoi(PQcmdTuples(dpstate->result));
+}
+
+/*
+ * Get the result of a RETURNING clause.
+ */
+static TupleTableSlot *
+get_returning_data(ForeignScanState *node)
+{
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ EState *estate = node->ss.ps.state;
+ ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+
+ Assert(resultRelInfo->ri_projectReturning);
+
+ /* If we didn't get any tuples, must be end of data. */
+ if (dpstate->next_tuple >= dpstate->num_tuples)
+ return ExecClearTuple(slot);
+
+ /* Increment the command es_processed count if necessary. */
+ if (dpstate->set_processed)
+ estate->es_processed += 1;
+
+ /*
+ * Store a RETURNING tuple. If has_returning is false, just emit a dummy
+ * tuple. (We have has_returning=false if the local query is of the form
+ * UPDATE/DELETE .. RETURNING 1 for example.)
+ */
+ if (!dpstate->has_returning)
+ ExecStoreAllNullTuple(slot);
+ else
+ {
+ /*
+ * On error, be sure to release the PGresult on the way out. Callers
+ * do not have PG_TRY blocks to ensure this happens.
+ */
+ PG_TRY();
+ {
+ HeapTuple newtup;
+
+ newtup = make_tuple_from_result_row(dpstate->result,
+ dpstate->next_tuple,
+ dpstate->rel,
+ dpstate->attinmeta,
+ dpstate->retrieved_attrs,
+ dpstate->temp_cxt);
+ ExecStoreTuple(newtup, slot, InvalidBuffer, false);
+ }
+ PG_CATCH();
+ {
+ if (dpstate->result)
+ PQclear(dpstate->result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ }
+ dpstate->next_tuple++;
+
+ /* Make slot available for evaluation of the local query RETURNING list. */
+ resultRelInfo->ri_projectReturning->pi_exprContext->ecxt_scantuple = slot;
+
+ return slot;
+}
+
+/*
* postgresAnalyzeForeignTable
* Test whether analyzing this foreign table is supported
*/
diff --git a/contrib/postgres_fdw/postgres_fdw.h b/contrib/postgres_fdw/postgres_fdw.h
index 2b63281..c0f5766 100644
--- a/contrib/postgres_fdw/postgres_fdw.h
+++ b/contrib/postgres_fdw/postgres_fdw.h
@@ -93,10 +93,24 @@ extern void deparseUpdateSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *targetAttrs, List *returningList,
List **retrieved_attrs);
+extern void deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseDeleteSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *returningList,
List **retrieved_attrs);
+extern void deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseAnalyzeSizeSql(StringInfo buf, Relation rel);
extern void deparseAnalyzeSql(StringInfo buf, Relation rel,
List **retrieved_attrs);
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 1978e16..d0493e1 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -399,28 +399,32 @@ INSERT INTO ft2 (c1,c2,c3) SELECT c1+1000,c2+100, c3 || c3 FROM ft2 LIMIT 20;
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+EXPLAIN (verbose, costs off)
+UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+EXPLAIN (verbose, costs off)
+UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
- FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
+ FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
- DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
+ DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
-DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
+DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
EXPLAIN (verbose, costs off)
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
-UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
+UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
-DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
+DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
-- Test that trigger on remote table works as expected
@@ -737,6 +741,90 @@ UPDATE rem1 SET f2 = 'testo';
-- Test returning a system attribute
INSERT INTO rem1(f2) VALUES ('test') RETURNING ctid;
+-- cleanup
+DROP TRIGGER trig_row_before ON rem1;
+DROP TRIGGER trig_row_after ON rem1;
+DROP TRIGGER trig_local_before ON loc1;
+
+
+-- Test DML pushdown functionality
+
+-- Test with statement-level triggers
+CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+DROP TRIGGER trig_stmt_before ON rem1;
+
+CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+DROP TRIGGER trig_stmt_after ON rem1;
+
+-- Test with row-level ON INSERT triggers
+CREATE TRIGGER trig_row_before_insert
+BEFORE INSERT ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+DROP TRIGGER trig_row_before_insert ON rem1;
+
+CREATE TRIGGER trig_row_after_insert
+AFTER INSERT ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+DROP TRIGGER trig_row_after_insert ON rem1;
+
+-- Test with row-level ON UPDATE triggers
+CREATE TRIGGER trig_row_before_update
+BEFORE UPDATE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can't be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+DROP TRIGGER trig_row_before_update ON rem1;
+
+CREATE TRIGGER trig_row_after_update
+AFTER UPDATE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can't be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+DROP TRIGGER trig_row_after_update ON rem1;
+
+-- Test with row-level ON DELETE triggers
+CREATE TRIGGER trig_row_before_delete
+BEFORE DELETE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can't be pushed down
+DROP TRIGGER trig_row_before_delete ON rem1;
+
+CREATE TRIGGER trig_row_after_delete
+AFTER DELETE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can't be pushed down
+DROP TRIGGER trig_row_after_delete ON rem1;
+
-- ===================================================================
-- test inheritance features
-- ===================================================================
@@ -868,6 +956,13 @@ fetch from c;
update bar set f2 = null where current of c;
rollback;
+explain (verbose, costs off)
+delete from foo where f1 < 5 returning *;
+delete from foo where f1 < 5 returning *;
+explain (verbose, costs off)
+update bar set f2 = f2 + 100 returning *;
+update bar set f2 = f2 + 100 returning *;
+
drop table foo cascade;
drop table bar cascade;
drop table loct1;
diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml
index a6945d3..3355f72 100644
--- a/doc/src/sgml/fdwhandler.sgml
+++ b/doc/src/sgml/fdwhandler.sgml
@@ -670,15 +670,164 @@ IsForeignRelUpdatable (Relation rel);
<para>
If the <function>IsForeignRelUpdatable</> pointer is set to
<literal>NULL</>, foreign tables are assumed to be insertable, updatable,
- or deletable if the FDW provides <function>ExecForeignInsert</>,
+ or deletable either the FDW provides <function>ExecForeignInsert</>,
<function>ExecForeignUpdate</>, or <function>ExecForeignDelete</>
- respectively. This function is only needed if the FDW supports some
+ respectively or if the FDW optimizes a foreign table update on a foreign
+ table using <function>PlanDMLPushdown</> (<function>PlanDMLPushdown</>
+ still needs to provide <function>BeginDMLPushdown</>,
+ <function>IterateDMLPushdown</> and <function>EndDMLPushdown</> to execute
+ the optimized update.).
+ This function is only needed if the FDW supports some
tables that are updatable and some that are not. (Even then, it's
permissible to throw an error in the execution routine instead of
checking in this function. However, this function is used to determine
updatability for display in the <literal>information_schema</> views.)
</para>
+ <para>
+ If an FDW supports an optimization that executes a foreign table update
+ directly on the remote server, it should provide the following callback
+ functions in addition to the table-updating functions described above:
+ </para>
+
+ <para>
+<programlisting>
+bool
+PlanDMLPushdown (PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+</programlisting>
+
+ Decide whether it is safe to execute a foreign table update directly
+ on the remote server. If so, return <literal>true</> after performing
+ planning actions needed for that. Otherwise, return <literal>false</>.
+ This optional function is called during query planning.
+ If this function succeeds, <function>BeginDMLPushdown</>,
+ <function>IterateDMLPushdown</> and <function>EndDMLPushdown</> will be
+ called at the execution stage, instead. Otherwise, the table update
+ will be executed using the table-updating functions described above.
+ The parameters are the same as for <function>PlanForeignModify</>.
+ </para>
+
+ <para>
+ To execute the table update directly on the remote server, this function
+ must rewrite the target subplan with a <structname>ForeignScan</> plan
+ node that executes the table update directly on the remote server. The
+ <structfield>operation</> field of the <structname>ForeignScan</> must
+ be set to the <literal>CmdType</> enumeration appropriately; that is,
+ <literal>CMD_UPDATE</> for <command>UPDATE</>,
+ <literal>CMD_INSERT</> for <command>INSERT</>, and
+ <literal>CMD_DELETE</> for <command>DELETE</>.
+ </para>
+
+ <para>
+ See <xref linkend="fdw-planning"> for additional information.
+ </para>
+
+ <para>
+ If the <function>PlanDMLPushdown</> pointer is set to
+ <literal>NULL</>, no attempts to execute the table update directly on
+ the remote server are taken.
+ </para>
+
+ <para>
+<programlisting>
+void
+BeginDMLPushdown (ForeignScanState *node,
+ int eflags);
+</programlisting>
+
+ Begin executing a foreign table update directly on the remote server.
+ This is called during executor startup. It should perform any
+ initialization needed prior to the actual table update (that should be
+ done upon the first call to <function>IterateDMLPushdown</>).
+ The <structname>ForeignScanState</> node has already been created, but
+ its <structfield>fdw_state</> field is still NULL. Information about
+ the table to update is accessible through the
+ <structname>ForeignScanState</> node (in particular, from the underlying
+ <structname>ForeignScan</> plan node, which contains any FDW-private
+ information provided by <function>PlanDMLPushdown</>).
+ <literal>eflags</> contains flag bits describing the executor's
+ operating mode for this plan node.
+ </para>
+
+ <para>
+ Note that when <literal>(eflags & EXEC_FLAG_EXPLAIN_ONLY)</> is
+ true, this function should not perform any externally-visible actions;
+ it should only do the minimum required to make the node state valid
+ for <function>ExplainDMLPushdown</> and <function>EndDMLPushdown</>.
+ </para>
+
+ <para>
+ If the <function>BeginDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to execute the table update directly on
+ the remote server will fail with an error message.
+ </para>
+
+ <para>
+<programlisting>
+TupleTableSlot *
+IterateDMLPushdown (ForeignScanState *node);
+</programlisting>
+
+ When the <command>INSERT</>, <command>UPDATE</> or <command>DELETE</>
+ query doesn't have a <literal>RETURNING</> clause, just return NULL
+ after the actual table update directly executed on the remote server.
+ When the query has the clause, fetch one result containing the data
+ needed for the <literal>RETURNING</> calculation, returning it in a
+ tuple table slot (the node's <structfield>ScanTupleSlot</> should be
+ used for this purpose). The data that was actually inserted, updated
+ or deleted must be stored in the
+ <literal>es_result_relation_info->ri_projectReturning->pi_exprContext->ecxt_scantuple</>
+ of the node's <structname>EState</>.
+ Return NULL if no more rows are available.
+ Note that this is called in a short-lived memory context that will be
+ reset between invocations. Create a memory context in
+ <function>BeginDMLPushdown</> if you need longer-lived storage, or use
+ the <structfield>es_query_cxt</> of the node's <structname>EState</>.
+ </para>
+
+ <para>
+ The rows returned must match the <structfield>fdw_scan_tlist</> target
+ list if one was supplied, otherwise they must match the row type of the
+ foreign table being updated. If you choose to optimize away fetching
+ columns that are not needed for the <literal>RETURNING</> calculation,
+ you should insert nulls in those column positions, or else generate a
+ <structfield>fdw_scan_tlist</> list with those columns omitted.
+ </para>
+
+ <para>
+ Whether the query has the clause or not, the query's reported row count
+ must be incremented by the FDW itself. When the query doesn't has the
+ clause, the FDW must also increment the row count for the
+ <structname>ForeignScanState</> node in the <command>EXPLAIN ANALYZE</>
+ case.
+ </para>
+
+ <para>
+ If the <function>IterateDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to execute the table update directly on
+ the remote server will fail with an error message.
+ </para>
+
+ <para>
+<programlisting>
+void
+EndDMLPushdown (ForeignScanState *node);
+</programlisting>
+
+ End the table update and release resources. It is normally not important
+ to release palloc'd memory, but for example open files and connections
+ to remote servers should be cleaned up.
+ </para>
+
+ <para>
+ If the <function>EndDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to execute the table update directly on
+ the remote server will fail with an error message.
+ </para>
+
</sect2>
<sect2 id="fdw-callbacks-row-locking">
@@ -866,6 +1015,29 @@ ExplainForeignModify (ModifyTableState *mtstate,
<command>EXPLAIN</>.
</para>
+ <para>
+<programlisting>
+void
+ExplainDMLPushdown (ForeignScanState *node,
+ ExplainState *es);
+</programlisting>
+
+ Print additional <command>EXPLAIN</> output for a foreign table update
+ that is executed directly on the remote server.
+ This function can call <function>ExplainPropertyText</> and
+ related functions to add fields to the <command>EXPLAIN</> output.
+ The flag fields in <literal>es</> can be used to determine what to
+ print, and the state of the <structname>ForeignScanState</> node
+ can be inspected to provide run-time statistics in the <command>EXPLAIN
+ ANALYZE</> case.
+ </para>
+
+ <para>
+ If the <function>ExplainDMLPushdown</> pointer is set to
+ <literal>NULL</>, no additional information is printed during
+ <command>EXPLAIN</>.
+ </para>
+
</sect2>
<sect2 id="fdw-callbacks-analyze">
@@ -1147,7 +1319,8 @@ GetForeignServerByName(const char *name, bool missing_ok);
<para>
The FDW callback functions <function>GetForeignRelSize</>,
<function>GetForeignPaths</>, <function>GetForeignPlan</>,
- <function>PlanForeignModify</>, and <function>GetForeignJoinPaths</>
+ <function>PlanForeignModify</>, <function>GetForeignJoinPaths</>, and
+ <function>PlanDMLPushdown</>
must fit into the workings of the <productname>PostgreSQL</> planner.
Here are some notes about what they must do.
</para>
@@ -1307,7 +1480,8 @@ GetForeignServerByName(const char *name, bool missing_ok);
<para>
When planning an <command>UPDATE</> or <command>DELETE</>,
- <function>PlanForeignModify</> can look up the <structname>RelOptInfo</>
+ <function>PlanForeignModify</> and <function>PlanDMLPushdown</>
+ can look up the <structname>RelOptInfo</>
struct for the foreign table and make use of the
<literal>baserel->fdw_private</> data previously created by the
scan-planning functions. However, in <command>INSERT</> the target
diff --git a/doc/src/sgml/postgres-fdw.sgml b/doc/src/sgml/postgres-fdw.sgml
index a90983c..ff86137 100644
--- a/doc/src/sgml/postgres-fdw.sgml
+++ b/doc/src/sgml/postgres-fdw.sgml
@@ -484,6 +484,15 @@
extension that's listed in the foreign server's <literal>extensions</>
option. Operators and functions in such clauses must
be <literal>IMMUTABLE</> as well.
+ For an <command>UPDATE</> or <command>DELETE</> query,
+ <filename>postgres_fdw</> attempts to optimize the query execution by
+ sending the whole query to the remote server if there are no query
+ <literal>WHERE</> clauses that cannot be sent to the remote server,
+ no local joins for the query, and no row-level local <literal>BEFORE</> or
+ <literal>AFTER</> triggers on the target table. In <command>UPDATE</>,
+ expressions to assign to target columns must use only built-in data types,
+ <literal>IMMUTABLE</> operators, or <literal>IMMUTABLE</> functions,
+ to reduce the risk of misexecution of the query.
</para>
<para>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index 25d8ca0..05d4a35 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -888,7 +888,29 @@ ExplainNode(PlanState *planstate, List *ancestors,
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
- pname = sname = "Foreign Scan";
+ sname = "Foreign Scan";
+ switch (((ForeignScan *) plan)->operation)
+ {
+ case CMD_SELECT:
+ pname = "Foreign Scan";
+ operation = "Select";
+ break;
+ case CMD_INSERT:
+ pname = "Foreign Insert";
+ operation = "Insert";
+ break;
+ case CMD_UPDATE:
+ pname = "Foreign Update";
+ operation = "Update";
+ break;
+ case CMD_DELETE:
+ pname = "Foreign Delete";
+ operation = "Delete";
+ break;
+ default:
+ pname = "???";
+ break;
+ }
break;
case T_CustomScan:
sname = "Custom Scan";
@@ -1636,6 +1658,12 @@ show_plan_tlist(PlanState *planstate, List *ancestors, ExplainState *es)
return;
if (IsA(plan, RecursiveUnion))
return;
+ /* Likewise for ForeignScan that has pushed down INSERT/UPDATE/DELETE */
+ if (IsA(plan, ForeignScan) &&
+ (((ForeignScan *) plan)->operation == CMD_INSERT ||
+ ((ForeignScan *) plan)->operation == CMD_UPDATE ||
+ ((ForeignScan *) plan)->operation == CMD_DELETE))
+ return;
/* Set up deparsing context */
context = set_deparse_context_planstate(es->deparse_cxt,
@@ -2224,8 +2252,16 @@ show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es)
FdwRoutine *fdwroutine = fsstate->fdwroutine;
/* Let the FDW emit whatever fields it wants */
- if (fdwroutine->ExplainForeignScan != NULL)
- fdwroutine->ExplainForeignScan(fsstate, es);
+ if (((ForeignScan *) fsstate->ss.ps.plan)->operation != CMD_SELECT)
+ {
+ if (fdwroutine->ExplainDMLPushdown != NULL)
+ fdwroutine->ExplainDMLPushdown(fsstate, es);
+ }
+ else
+ {
+ if (fdwroutine->ExplainForeignScan != NULL)
+ fdwroutine->ExplainForeignScan(fsstate, es);
+ }
}
/*
@@ -2611,8 +2647,10 @@ show_modifytable_info(ModifyTableState *mtstate, List *ancestors,
}
}
- /* Give FDW a chance */
- if (fdwroutine && fdwroutine->ExplainForeignModify != NULL)
+ /* Give FDW a chance if needed */
+ if (!resultRelInfo->ri_FdwPushdown &&
+ fdwroutine != NULL &&
+ fdwroutine->ExplainForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, j);
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 76f7297..68a3a13 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1011,8 +1011,9 @@ InitPlan(QueryDesc *queryDesc, int eflags)
* CheckValidRowMarkRel.
*/
void
-CheckValidResultRel(Relation resultRel, CmdType operation)
+CheckValidResultRel(ResultRelInfo *resultRelInfo, CmdType operation)
{
+ Relation resultRel = resultRelInfo->ri_RelationDesc;
TriggerDesc *trigDesc = resultRel->trigdesc;
FdwRoutine *fdwroutine;
@@ -1086,7 +1087,8 @@ CheckValidResultRel(Relation resultRel, CmdType operation)
switch (operation)
{
case CMD_INSERT:
- if (fdwroutine->ExecForeignInsert == NULL)
+ if (fdwroutine->ExecForeignInsert == NULL &&
+ !resultRelInfo->ri_FdwPushdown)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot insert into foreign table \"%s\"",
@@ -1099,7 +1101,8 @@ CheckValidResultRel(Relation resultRel, CmdType operation)
RelationGetRelationName(resultRel))));
break;
case CMD_UPDATE:
- if (fdwroutine->ExecForeignUpdate == NULL)
+ if (fdwroutine->ExecForeignUpdate == NULL &&
+ !resultRelInfo->ri_FdwPushdown)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot update foreign table \"%s\"",
@@ -1112,7 +1115,8 @@ CheckValidResultRel(Relation resultRel, CmdType operation)
RelationGetRelationName(resultRel))));
break;
case CMD_DELETE:
- if (fdwroutine->ExecForeignDelete == NULL)
+ if (fdwroutine->ExecForeignDelete == NULL &&
+ !resultRelInfo->ri_FdwPushdown)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot delete from foreign table \"%s\"",
@@ -1245,6 +1249,7 @@ InitResultRelInfo(ResultRelInfo *resultRelInfo,
else
resultRelInfo->ri_FdwRoutine = NULL;
resultRelInfo->ri_FdwState = NULL;
+ resultRelInfo->ri_FdwPushdown = false;
resultRelInfo->ri_ConstraintExprs = NULL;
resultRelInfo->ri_junkFilter = NULL;
resultRelInfo->ri_projectReturning = NULL;
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index 388c922..2cfe256 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -48,7 +48,10 @@ ForeignNext(ForeignScanState *node)
/* Call the Iterate function in short-lived context */
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
- slot = node->fdwroutine->IterateForeignScan(node);
+ if (plan->operation != CMD_SELECT)
+ slot = node->fdwroutine->IterateDMLPushdown(node);
+ else
+ slot = node->fdwroutine->IterateForeignScan(node);
MemoryContextSwitchTo(oldcontext);
/*
@@ -226,7 +229,10 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
/*
* Tell the FDW to initialize the scan.
*/
- fdwroutine->BeginForeignScan(scanstate, eflags);
+ if (node->operation != CMD_SELECT)
+ fdwroutine->BeginDMLPushdown(scanstate, eflags);
+ else
+ fdwroutine->BeginForeignScan(scanstate, eflags);
return scanstate;
}
@@ -240,8 +246,13 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
void
ExecEndForeignScan(ForeignScanState *node)
{
+ ForeignScan *plan = (ForeignScan *) node->ss.ps.plan;
+
/* Let the FDW shut down */
- node->fdwroutine->EndForeignScan(node);
+ if (plan->operation != CMD_SELECT)
+ node->fdwroutine->EndDMLPushdown(node);
+ else
+ node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 27051e8..d87cd8a 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -138,13 +138,17 @@ ExecCheckPlanOutput(Relation resultRel, List *targetList)
* tupleSlot: slot holding tuple actually inserted/updated/deleted
* planSlot: slot holding tuple returned by top subplan node
*
+ * Note: If tupleSlot is NULL, the FDW should have already provided econtext's
+ * scan tuple.
+ *
* Returns a slot holding the result tuple
*/
static TupleTableSlot *
-ExecProcessReturning(ProjectionInfo *projectReturning,
+ExecProcessReturning(ResultRelInfo *resultRelInfo,
TupleTableSlot *tupleSlot,
TupleTableSlot *planSlot)
{
+ ProjectionInfo *projectReturning = resultRelInfo->ri_projectReturning;
ExprContext *econtext = projectReturning->pi_exprContext;
/*
@@ -154,7 +158,20 @@ ExecProcessReturning(ProjectionInfo *projectReturning,
ResetExprContext(econtext);
/* Make tuple and any needed join variables available to ExecProject */
- econtext->ecxt_scantuple = tupleSlot;
+ if (tupleSlot)
+ econtext->ecxt_scantuple = tupleSlot;
+ else
+ {
+ HeapTuple tuple;
+
+ /*
+ * RETURNING expressions might reference the tableoid column, so
+ * initialize t_tableOid before evaluating them.
+ */
+ Assert(!TupIsNull(econtext->ecxt_scantuple));
+ tuple = ExecMaterializeSlot(econtext->ecxt_scantuple);
+ tuple->t_tableOid = RelationGetRelid(resultRelInfo->ri_RelationDesc);
+ }
econtext->ecxt_outertuple = planSlot;
/* Compute the RETURNING expressions */
@@ -496,8 +513,7 @@ ExecInsert(ModifyTableState *mtstate,
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
- return ExecProcessReturning(resultRelInfo->ri_projectReturning,
- slot, planSlot);
+ return ExecProcessReturning(resultRelInfo, slot, planSlot);
return NULL;
}
@@ -738,8 +754,7 @@ ldelete:;
ExecStoreTuple(&deltuple, slot, InvalidBuffer, false);
}
- rslot = ExecProcessReturning(resultRelInfo->ri_projectReturning,
- slot, planSlot);
+ rslot = ExecProcessReturning(resultRelInfo, slot, planSlot);
/*
* Before releasing the target tuple again, make sure rslot has a
@@ -1024,8 +1039,7 @@ lreplace:;
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
- return ExecProcessReturning(resultRelInfo->ri_projectReturning,
- slot, planSlot);
+ return ExecProcessReturning(resultRelInfo, slot, planSlot);
return NULL;
}
@@ -1380,6 +1394,21 @@ ExecModifyTable(ModifyTableState *node)
break;
}
+ /*
+ * If ri_FdwPushdown is true, all we need to do here is compute the
+ * RETURNING expressions.
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ Assert(resultRelInfo->ri_projectReturning);
+
+ /* No need to provide scan tuple to ExecProcessReturning. */
+ slot = ExecProcessReturning(resultRelInfo, NULL, planSlot);
+
+ estate->es_result_relation_info = saved_resultRelInfo;
+ return slot;
+ }
+
EvalPlanQualSetSlot(&node->mt_epqstate, planSlot);
slot = planSlot;
@@ -1559,10 +1588,13 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
{
subplan = (Plan *) lfirst(l);
+ /* Initialize the FdwPushdown flag */
+ resultRelInfo->ri_FdwPushdown = list_nth_int(node->fdwPushdowns, i);
+
/*
* Verify result relation is a valid target for the current operation
*/
- CheckValidResultRel(resultRelInfo->ri_RelationDesc, operation);
+ CheckValidResultRel(resultRelInfo, operation);
/*
* If there are indices on the result relation, open them and save
@@ -1583,7 +1615,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
/* Also let FDWs init themselves for foreign-table result rels */
- if (resultRelInfo->ri_FdwRoutine != NULL &&
+ if (!resultRelInfo->ri_FdwPushdown &&
+ resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, i);
@@ -1754,13 +1787,25 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
erm = ExecFindRowMark(estate, rc->rti, false);
/* build ExecAuxRowMark for each subplan */
+ resultRelInfo = mtstate->resultRelInfo;
for (i = 0; i < nplans; i++)
{
ExecAuxRowMark *aerm;
+ /*
+ * ignore subplan if the FDW pushes down the command to the remote
+ * server
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ resultRelInfo++;
+ continue;
+ }
+
subplan = mtstate->mt_plans[i]->plan;
aerm = ExecBuildAuxRowMark(erm, subplan->targetlist);
mtstate->mt_arowmarks[i] = lappend(mtstate->mt_arowmarks[i], aerm);
+ resultRelInfo++;
}
}
@@ -1821,6 +1866,16 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
ExecCheckPlanOutput(resultRelInfo->ri_RelationDesc,
subplan->targetlist);
+ /*
+ * ignore subplan if the FDW pushes down the command to the
+ * remote server
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ resultRelInfo++;
+ continue;
+ }
+
j = ExecInitJunkFilter(subplan->targetlist,
resultRelInfo->ri_RelationDesc->rd_att->tdhasoid,
ExecInitExtraTupleSlot(estate));
@@ -1910,7 +1965,8 @@ ExecEndModifyTable(ModifyTableState *node)
{
ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
- if (resultRelInfo->ri_FdwRoutine != NULL &&
+ if (!resultRelInfo->ri_FdwPushdown &&
+ resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->EndForeignModify != NULL)
resultRelInfo->ri_FdwRoutine->EndForeignModify(node->ps.state,
resultRelInfo);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index a8b79fa..98d7226 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -187,6 +187,7 @@ _copyModifyTable(const ModifyTable *from)
COPY_NODE_FIELD(withCheckOptionLists);
COPY_NODE_FIELD(returningLists);
COPY_NODE_FIELD(fdwPrivLists);
+ COPY_NODE_FIELD(fdwPushdowns);
COPY_NODE_FIELD(rowMarks);
COPY_SCALAR_FIELD(epqParam);
COPY_SCALAR_FIELD(onConflictAction);
@@ -646,6 +647,7 @@ _copyForeignScan(const ForeignScan *from)
/*
* copy remainder of node
*/
+ COPY_SCALAR_FIELD(operation);
COPY_SCALAR_FIELD(fs_server);
COPY_NODE_FIELD(fdw_exprs);
COPY_NODE_FIELD(fdw_private);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index d59b954..3a73f02 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -341,6 +341,7 @@ _outModifyTable(StringInfo str, const ModifyTable *node)
WRITE_NODE_FIELD(withCheckOptionLists);
WRITE_NODE_FIELD(returningLists);
WRITE_NODE_FIELD(fdwPrivLists);
+ WRITE_NODE_FIELD(fdwPushdowns);
WRITE_NODE_FIELD(rowMarks);
WRITE_INT_FIELD(epqParam);
WRITE_ENUM_FIELD(onConflictAction, OnConflictAction);
@@ -592,6 +593,7 @@ _outForeignScan(StringInfo str, const ForeignScan *node)
_outScanInfo(str, (const Scan *) node);
+ WRITE_ENUM_FIELD(operation, CmdType);
WRITE_OID_FIELD(fs_server);
WRITE_NODE_FIELD(fdw_exprs);
WRITE_NODE_FIELD(fdw_private);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 6c46151..d413241 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -1472,6 +1472,7 @@ _readModifyTable(void)
READ_NODE_FIELD(withCheckOptionLists);
READ_NODE_FIELD(returningLists);
READ_NODE_FIELD(fdwPrivLists);
+ READ_NODE_FIELD(fdwPushdowns);
READ_NODE_FIELD(rowMarks);
READ_INT_FIELD(epqParam);
READ_ENUM_FIELD(onConflictAction, OnConflictAction);
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 54ff7f6..0dd05ca 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -3788,6 +3788,7 @@ make_foreignscan(List *qptlist,
plan->lefttree = outer_plan;
plan->righttree = NULL;
node->scan.scanrelid = scanrelid;
+ node->operation = CMD_SELECT;
/* fs_server will be filled in by create_foreignscan_plan */
node->fs_server = InvalidOid;
node->fdw_exprs = fdw_exprs;
@@ -5064,6 +5065,7 @@ make_modifytable(PlannerInfo *root,
Plan *plan = &node->plan;
double total_size;
List *fdw_private_list;
+ List *fdwpushdown_list;
ListCell *subnode;
ListCell *lc;
int i;
@@ -5144,12 +5146,14 @@ make_modifytable(PlannerInfo *root,
* construct private plan data, and accumulate it all into a list.
*/
fdw_private_list = NIL;
+ fdwpushdown_list = NIL;
i = 0;
foreach(lc, resultRelations)
{
Index rti = lfirst_int(lc);
FdwRoutine *fdwroutine;
List *fdw_private;
+ bool fdwpushdown;
/*
* If possible, we want to get the FdwRoutine from our RelOptInfo for
@@ -5176,7 +5180,23 @@ make_modifytable(PlannerInfo *root,
fdwroutine = NULL;
}
+ /*
+ * If the target relation has any row-level triggers, we can't push
+ * down the command to the remote server.
+ */
if (fdwroutine != NULL &&
+ fdwroutine->PlanDMLPushdown != NULL &&
+ fdwroutine->BeginDMLPushdown != NULL &&
+ fdwroutine->IterateDMLPushdown != NULL &&
+ fdwroutine->EndDMLPushdown != NULL &&
+ !has_row_triggers(root, rti, operation))
+ fdwpushdown = fdwroutine->PlanDMLPushdown(root, node, rti, i);
+ else
+ fdwpushdown = false;
+ fdwpushdown_list = lappend_int(fdwpushdown_list, fdwpushdown);
+
+ if (!fdwpushdown &&
+ fdwroutine != NULL &&
fdwroutine->PlanForeignModify != NULL)
fdw_private = fdwroutine->PlanForeignModify(root, node, rti, i);
else
@@ -5185,6 +5205,7 @@ make_modifytable(PlannerInfo *root,
i++;
}
node->fdwPrivLists = fdw_private_list;
+ node->fdwPushdowns = fdwpushdown_list;
return node;
}
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 0ea9fcf..8d19b37 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -1520,3 +1520,50 @@ has_unique_index(RelOptInfo *rel, AttrNumber attno)
}
return false;
}
+
+
+/*
+ * has_row_triggers
+ *
+ * Detect whether the specified relation has any row-level triggers for event.
+ */
+bool
+has_row_triggers(PlannerInfo *root, Index rti, CmdType event)
+{
+ RangeTblEntry *rte = planner_rt_fetch(rti, root);
+ Relation relation;
+ TriggerDesc *trigDesc;
+ bool result = false;
+
+ /* Assume we already have adequate lock */
+ relation = heap_open(rte->relid, NoLock);
+
+ trigDesc = relation->trigdesc;
+ switch (event)
+ {
+ case CMD_INSERT:
+ if (trigDesc &&
+ (trigDesc->trig_insert_after_row ||
+ trigDesc->trig_insert_before_row))
+ result = true;
+ break;
+ case CMD_UPDATE:
+ if (trigDesc &&
+ (trigDesc->trig_update_after_row ||
+ trigDesc->trig_update_before_row))
+ result = true;
+ break;
+ case CMD_DELETE:
+ if (trigDesc &&
+ (trigDesc->trig_delete_after_row ||
+ trigDesc->trig_delete_before_row))
+ result = true;
+ break;
+ default:
+ elog(ERROR, "unrecognized CmdType: %d", (int) event);
+ break;
+ }
+
+ heap_close(relation, NoLock);
+ return result;
+}
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 1a44085..fd12df4 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -184,7 +184,7 @@ extern void ExecutorEnd(QueryDesc *queryDesc);
extern void standard_ExecutorEnd(QueryDesc *queryDesc);
extern void ExecutorRewind(QueryDesc *queryDesc);
extern bool ExecCheckRTPerms(List *rangeTable, bool ereport_on_violation);
-extern void CheckValidResultRel(Relation resultRel, CmdType operation);
+extern void CheckValidResultRel(ResultRelInfo *resultRelInfo, CmdType operation);
extern void InitResultRelInfo(ResultRelInfo *resultRelInfo,
Relation resultRelationDesc,
Index resultRelationIndex,
diff --git a/src/include/foreign/fdwapi.h b/src/include/foreign/fdwapi.h
index 9fafab0..e150bea 100644
--- a/src/include/foreign/fdwapi.h
+++ b/src/include/foreign/fdwapi.h
@@ -94,6 +94,18 @@ typedef void (*EndForeignModify_function) (EState *estate,
typedef int (*IsForeignRelUpdatable_function) (Relation rel);
+typedef bool (*PlanDMLPushdown_function) (PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+
+typedef void (*BeginDMLPushdown_function) (ForeignScanState *node,
+ int eflags);
+
+typedef TupleTableSlot *(*IterateDMLPushdown_function) (ForeignScanState *node);
+
+typedef void (*EndDMLPushdown_function) (ForeignScanState *node);
+
typedef RowMarkType (*GetForeignRowMarkType_function) (RangeTblEntry *rte,
LockClauseStrength strength);
@@ -111,6 +123,9 @@ typedef void (*ExplainForeignModify_function) (ModifyTableState *mtstate,
int subplan_index,
struct ExplainState *es);
+typedef void (*ExplainDMLPushdown_function) (ForeignScanState *node,
+ struct ExplainState *es);
+
typedef int (*AcquireSampleRowsFunc) (Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
@@ -171,6 +186,10 @@ typedef struct FdwRoutine
ExecForeignDelete_function ExecForeignDelete;
EndForeignModify_function EndForeignModify;
IsForeignRelUpdatable_function IsForeignRelUpdatable;
+ PlanDMLPushdown_function PlanDMLPushdown;
+ BeginDMLPushdown_function BeginDMLPushdown;
+ IterateDMLPushdown_function IterateDMLPushdown;
+ EndDMLPushdown_function EndDMLPushdown;
/* Functions for SELECT FOR UPDATE/SHARE row locking */
GetForeignRowMarkType_function GetForeignRowMarkType;
@@ -180,6 +199,7 @@ typedef struct FdwRoutine
/* Support functions for EXPLAIN */
ExplainForeignScan_function ExplainForeignScan;
ExplainForeignModify_function ExplainForeignModify;
+ ExplainDMLPushdown_function ExplainDMLPushdown;
/* Support functions for ANALYZE */
AnalyzeForeignTable_function AnalyzeForeignTable;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 064a050..fe160db 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -311,6 +311,7 @@ typedef struct JunkFilter
* TrigInstrument optional runtime measurements for triggers
* FdwRoutine FDW callback functions, if foreign table
* FdwState available to save private state of FDW
+ * FdwPushdown true when the command is pushed down
* WithCheckOptions list of WithCheckOption's to be checked
* WithCheckOptionExprs list of WithCheckOption expr states
* ConstraintExprs array of constraint-checking expr states
@@ -334,6 +335,7 @@ typedef struct ResultRelInfo
Instrumentation *ri_TrigInstrument;
struct FdwRoutine *ri_FdwRoutine;
void *ri_FdwState;
+ bool ri_FdwPushdown;
List *ri_WithCheckOptions;
List *ri_WithCheckOptionExprs;
List **ri_ConstraintExprs;
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 55d6bbe..4858c79 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -189,6 +189,7 @@ typedef struct ModifyTable
List *withCheckOptionLists; /* per-target-table WCO lists */
List *returningLists; /* per-target-table RETURNING tlists */
List *fdwPrivLists; /* per-target-table FDW private data lists */
+ List *fdwPushdowns; /* per-target-table FDW pushdown flags */
List *rowMarks; /* PlanRowMarks (non-locking only) */
int epqParam; /* ID of Param for EvalPlanQual re-eval */
OnConflictAction onConflictAction; /* ON CONFLICT action */
@@ -531,6 +532,7 @@ typedef struct WorkTableScan
typedef struct ForeignScan
{
Scan scan;
+ CmdType operation; /* SELECT/INSERT/UPDATE/DELETE */
Oid fs_server; /* OID of foreign server */
List *fdw_exprs; /* expressions that FDW may evaluate */
List *fdw_private; /* private data for FDW */
diff --git a/src/include/optimizer/plancat.h b/src/include/optimizer/plancat.h
index 52335fa..125274e 100644
--- a/src/include/optimizer/plancat.h
+++ b/src/include/optimizer/plancat.h
@@ -55,4 +55,6 @@ extern Selectivity join_selectivity(PlannerInfo *root,
JoinType jointype,
SpecialJoinInfo *sjinfo);
+extern bool has_row_triggers(PlannerInfo *root, Index rti, CmdType event);
+
#endif /* PLANCAT_H */
On 10 February 2016 at 08:00, Rushabh Lathia <rushabh.lathia@gmail.com> wrote:
Fujita-san, I am attaching update version of the patch, which added
the documentation update.Once we finalize this, I feel good with the patch and think that we
could mark this as ready for committer.
I find this wording a bit confusing:
"If the IsForeignRelUpdatable pointer is set to NULL, foreign tables
are assumed to be insertable, updatable, or deletable either the FDW
provides ExecForeignInsert,ExecForeignUpdate, or ExecForeignDelete
respectively or if the FDW optimizes a foreign table update on a
foreign table using PlanDMLPushdown (PlanDMLPushdown still needs to
provide BeginDMLPushdown, IterateDMLPushdown and EndDMLPushdown to
execute the optimized update.)."
This is a very long sentence, and the word "either" doesn't work here.
Also:
"When the query doesn't has the clause, the FDW must also increment
the row count for the ForeignScanState node in the EXPLAIN ANALYZE
case."
Should read "doesn't have"
The rest looks fine AFAICT.
Regards
Thom
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Feb 10, 2016 at 3:00 AM, Rushabh Lathia
<rushabh.lathia@gmail.com> wrote:
Fujita-san, I am attaching update version of the patch, which added
the documentation update.Once we finalize this, I feel good with the patch and think that we
could mark this as ready for committer.
It would be nice to hear from Tom and/or Stephen whether the changes
that have been made since they last commented on it. I feel like the
design is reasonably OK, but I don't want to push this through if
they're still not happy with it. One thing I'm not altogether keen on
is the use of "pushdown" or "dml pushdown" as a term of art; on the
other hand, I'm not sure what other term would be better.
Comments on specific points follow.
This seems to need minor rebasing in the wake of the join pushdown patch.
+ /* Likewise for ForeignScan that has pushed down INSERT/UPDATE/DELETE */
+ if (IsA(plan, ForeignScan) &&
+ (((ForeignScan *) plan)->operation == CMD_INSERT ||
+ ((ForeignScan *) plan)->operation == CMD_UPDATE ||
+ ((ForeignScan *) plan)->operation == CMD_DELETE))
+ return;
I don't really understand why this is a good idea. Since target lists
are only displayed with EXPLAIN (VERBOSE), I don't really understand
what is to be gained by suppressing them in any case at all, and
therefore I don't understand why it's a good idea to do it here,
either. If there is a good reason, maybe the comment should explain
what it is.
+ /* Check point 1 */
+ if (operation == CMD_INSERT)
+ return false;
+
+ /* Check point 2 */
+ if (nodeTag(subplan) != T_ForeignScan)
+ return false;
+
+ /* Check point 3 */
+ if (subplan->qual != NIL)
+ return false;
+
+ /* Check point 4 */
+ if (operation == CMD_UPDATE)
These comments are referring to something in the function header
further up, but you could instead just delete the stuff from the
header and mention the actual conditions here. Also:
- If the first condition is supposed accept only CMD_UPDATE or
CMD_DELETE, say if (operation != CMD_UPDATE || operation !=
CMD_DELETE) rather than doing it this way. Is-not-insert may in this
context be functionally equivalent to is-update-or-delete, but it's
better to write the comment and the code so that they exactly match
rather than kinda-match.
- For point 2, use IsA(subplan, ForiegnScan).
+ /*
+ * ignore subplan if the FDW pushes down the
command to the remote
+ * server
+ */
This comment states what the code does, instead of explaining why it
does it. Please update it so that it explains why it does it.
+ List *fdwPushdowns; /* per-target-table FDW
pushdown flags */
Isn't a list of booleans an awfully inefficient representation? How
about a Bitmapset?
+ /*
+ * Prepare remote-parameter expressions for evaluation. (Note: in
+ * practice, we expect that all these expressions will be just
Params, so
+ * we could possibly do something more efficient than using the full
+ * expression-eval machinery for this. But probably there
would be little
+ * benefit, and it'd require postgres_fdw to know more than is desirable
+ * about Param evaluation.)
+ */
+ dpstate->param_exprs = (List *)
+ ExecInitExpr((Expr *) fsplan->fdw_exprs,
+ (PlanState *) node);
This is an exact copy of an existing piece of code and its associated
comment. A good bit of the surrounding code is copied, too. You need
to refactor to avoid duplication, like by putting some of the code in
a new function that both postgresBeginForeignScan and
postgresBeginForeignModify can call.
execute_dml_stmt() has some of the same disease.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hi Rushabh and Thom,
Thanks for the review!
On 2016/02/10 22:37, Thom Brown wrote:
On 10 February 2016 at 08:00, Rushabh Lathia <rushabh.lathia@gmail.com> wrote:
Fujita-san, I am attaching update version of the patch, which added
the documentation update.
Thanks for updating the patch!
Once we finalize this, I feel good with the patch and think that we
could mark this as ready for committer.
I find this wording a bit confusing:
"If the IsForeignRelUpdatable pointer is set to NULL, foreign tables
are assumed to be insertable, updatable, or deletable either the FDW
provides ExecForeignInsert,ExecForeignUpdate, or ExecForeignDelete
respectively or if the FDW optimizes a foreign table update on a
foreign table using PlanDMLPushdown (PlanDMLPushdown still needs to
provide BeginDMLPushdown, IterateDMLPushdown and EndDMLPushdown to
execute the optimized update.)."This is a very long sentence, and the word "either" doesn't work here.
Agreed.
As a result of our discussions, we reached a conclusion that the DML
pushdown APIs should be provided together with existing APIs such as
ExecForeignInsert, ExecForeignUpdate or ExecForeignDelete, IIUC. So,
how about (1) leaving the description for the existing APIs as-is and
(2) adding a new description for the DML pushdown APIs in parenthesis,
like this?:
If the <function>IsForeignRelUpdatable</> pointer is set to
<literal>NULL</>, foreign tables are assumed to be insertable,
updatable,
or deletable if the FDW provides <function>ExecForeignInsert</>,
<function>ExecForeignUpdate</>, or <function>ExecForeignDelete</>
respectively.
(If the FDW attempts to optimize a foreign table update, it still
needs to provide PlanDMLPushdown, BeginDMLPushdown,
IterateDMLPushdown and EndDMLPushdown.)
Actually, if the FDW provides the DML pushdown APIs, (pushdown-able)
foreign table updates can be done without ExecForeignInsert,
ExecForeignUpdate or ExecForeignDelete. So, the above docs are not
necessarily correct. But we don't recommend to do that without the
existing APIs, so I'm not sure it's worth complicating the docs.
Also:
"When the query doesn't has the clause, the FDW must also increment
the row count for the ForeignScanState node in the EXPLAIN ANALYZE
case."Should read "doesn't have"
Will fix.
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hi Robert,
Thanks for the review!
On 2016/02/11 5:43, Robert Haas wrote:
On Wed, Feb 10, 2016 at 3:00 AM, Rushabh Lathia
<rushabh.lathia@gmail.com> wrote:Fujita-san, I am attaching update version of the patch, which added
the documentation update.Once we finalize this, I feel good with the patch and think that we
could mark this as ready for committer.
It would be nice to hear from Tom and/or Stephen whether the changes
that have been made since they last commented on it. I feel like the
design is reasonably OK, but I don't want to push this through if
they're still not happy with it. One thing I'm not altogether keen on
is the use of "pushdown" or "dml pushdown" as a term of art; on the
other hand, I'm not sure what other term would be better.
I'm open to that naming. Proposals are welcome!
Comments on specific points follow.
This seems to need minor rebasing in the wake of the join pushdown patch.
Will do.
+ /* Likewise for ForeignScan that has pushed down INSERT/UPDATE/DELETE */ + if (IsA(plan, ForeignScan) && + (((ForeignScan *) plan)->operation == CMD_INSERT || + ((ForeignScan *) plan)->operation == CMD_UPDATE || + ((ForeignScan *) plan)->operation == CMD_DELETE)) + return;I don't really understand why this is a good idea. Since target lists
are only displayed with EXPLAIN (VERBOSE), I don't really understand
what is to be gained by suppressing them in any case at all, and
therefore I don't understand why it's a good idea to do it here,
either. If there is a good reason, maybe the comment should explain
what it is.
I think that displaying target lists would be confusing for users. Here
is an example:
EXPLAIN (verbose, costs off)
DELETE FROM rem1; -- can be pushed down
QUERY PLAN
---------------------------------------------
Delete on public.rem1
-> Foreign Delete on public.rem1
Output: ctid
Remote SQL: DELETE FROM public.loc1
(4 rows)
Should we output the "Output" line?
+ /* Check point 1 */ + if (operation == CMD_INSERT) + return false; + + /* Check point 2 */ + if (nodeTag(subplan) != T_ForeignScan) + return false; + + /* Check point 3 */ + if (subplan->qual != NIL) + return false; + + /* Check point 4 */ + if (operation == CMD_UPDATE)These comments are referring to something in the function header
further up, but you could instead just delete the stuff from the
header and mention the actual conditions here. Also:
Will fix.
- If the first condition is supposed accept only CMD_UPDATE or
CMD_DELETE, say if (operation != CMD_UPDATE || operation !=
CMD_DELETE) rather than doing it this way. Is-not-insert may in this
context be functionally equivalent to is-update-or-delete, but it's
better to write the comment and the code so that they exactly match
rather than kinda-match.- For point 2, use IsA(subplan, ForiegnScan).
Will fix.
+ /* + * ignore subplan if the FDW pushes down the command to the remote + * server + */This comment states what the code does, instead of explaining why it
does it. Please update it so that it explains why it does it.
Will update.
+ List *fdwPushdowns; /* per-target-table FDW
pushdown flags */Isn't a list of booleans an awfully inefficient representation? How
about a Bitmapset?
OK, will do.
+ /* + * Prepare remote-parameter expressions for evaluation. (Note: in + * practice, we expect that all these expressions will be just Params, so + * we could possibly do something more efficient than using the full + * expression-eval machinery for this. But probably there would be little + * benefit, and it'd require postgres_fdw to know more than is desirable + * about Param evaluation.) + */ + dpstate->param_exprs = (List *) + ExecInitExpr((Expr *) fsplan->fdw_exprs, + (PlanState *) node);This is an exact copy of an existing piece of code and its associated
comment. A good bit of the surrounding code is copied, too. You need
to refactor to avoid duplication, like by putting some of the code in
a new function that both postgresBeginForeignScan and
postgresBeginForeignModify can call.execute_dml_stmt() has some of the same disease.
Will do.
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Feb 12, 2016 at 7:19 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
I think that displaying target lists would be confusing for users. Here is
an example:EXPLAIN (verbose, costs off)
DELETE FROM rem1; -- can be pushed down
QUERY PLAN
---------------------------------------------
Delete on public.rem1
-> Foreign Delete on public.rem1
Output: ctid
Remote SQL: DELETE FROM public.loc1
(4 rows)Should we output the "Output" line?
I see your point, but what if there's a RETURNING clause?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/02/12 21:46, Robert Haas wrote:
On Fri, Feb 12, 2016 at 7:19 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:I think that displaying target lists would be confusing for users. Here is
an example:EXPLAIN (verbose, costs off)
DELETE FROM rem1; -- can be pushed down
QUERY PLAN
---------------------------------------------
Delete on public.rem1
-> Foreign Delete on public.rem1
Output: ctid
Remote SQL: DELETE FROM public.loc1
(4 rows)Should we output the "Output" line?
I see your point, but what if there's a RETURNING clause?
IMO I think that would be confusing in that case. Here is an example:
EXPLAIN (verbose, costs off)
DELETE FROM rem1 RETURNING *; -- can be pushed down
QUERY PLAN
--------------------------------------------------------------
Delete on public.rem1
Output: f1, f2
-> Foreign Delete on public.rem1
Output: ctid
Remote SQL: DELETE FROM public.loc1 RETURNING f1, f2
(5 rows)
The Output line beneath the ForeignScan node doesn't match the RETURNING
expressions in the remote query as the Output line beneath the
ModifyTable node does, so I think displaying that would be confusing
even in that case.
Another example:
postgres=# explain verbose update foo set a = a + 1 returning *;
QUERY PLAN
------------------------------------------------------------------------------
Update on public.foo (cost=100.00..137.50 rows=1000 width=10)
Output: a
-> Foreign Update on public.foo (cost=100.00..137.50 rows=1000
width=10)
Output: (a + 1), ctid
Remote SQL: UPDATE public.foo SET a = (a + 1) RETURNING a
(5 rows)
Same above.
As for case of INSERT .. RETURNING .., I guess there is not such a
mismatch, but I'm not sure that displaying that is that helpful,
honestly, so I'd vote for suppressing that in all cases, for consistency.
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Feb 12, 2016 at 5:40 PM, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>
wrote:
Hi Rushabh and Thom,
Thanks for the review!
On 2016/02/10 22:37, Thom Brown wrote:
On 10 February 2016 at 08:00, Rushabh Lathia <rushabh.lathia@gmail.com>
wrote:Fujita-san, I am attaching update version of the patch, which added
the documentation update.Thanks for updating the patch!
Once we finalize this, I feel good with the patch and think that we
could mark this as ready for committer.
I find this wording a bit confusing:
"If the IsForeignRelUpdatable pointer is set to NULL, foreign tables
are assumed to be insertable, updatable, or deletable either the FDW
provides ExecForeignInsert,ExecForeignUpdate, or ExecForeignDelete
respectively or if the FDW optimizes a foreign table update on a
foreign table using PlanDMLPushdown (PlanDMLPushdown still needs to
provide BeginDMLPushdown, IterateDMLPushdown and EndDMLPushdown to
execute the optimized update.)."This is a very long sentence, and the word "either" doesn't work here.
Agreed.
As a result of our discussions, we reached a conclusion that the DML
pushdown APIs should be provided together with existing APIs such as
ExecForeignInsert, ExecForeignUpdate or ExecForeignDelete, IIUC. So, how
about (1) leaving the description for the existing APIs as-is and (2)
adding a new description for the DML pushdown APIs in parenthesis, like
this?:If the <function>IsForeignRelUpdatable</> pointer is set to
<literal>NULL</>, foreign tables are assumed to be insertable,
updatable,
or deletable if the FDW provides <function>ExecForeignInsert</>,
<function>ExecForeignUpdate</>, or <function>ExecForeignDelete</>
respectively.
(If the FDW attempts to optimize a foreign table update, it still
needs to provide PlanDMLPushdown, BeginDMLPushdown,
IterateDMLPushdown and EndDMLPushdown.)Actually, if the FDW provides the DML pushdown APIs, (pushdown-able)
foreign table updates can be done without ExecForeignInsert,
ExecForeignUpdate or ExecForeignDelete. So, the above docs are not
necessarily correct. But we don't recommend to do that without the
existing APIs, so I'm not sure it's worth complicating the docs.
Adding a new description for DML pushdown API seems good idea. I would
suggest to add that as separate paragraph rather then into brackets.
Also:
"When the query doesn't has the clause, the FDW must also increment
the row count for the ForeignScanState node in the EXPLAIN ANALYZE
case."Should read "doesn't have"
Will fix.
Best regards,
Etsuro Fujita
--
Rushabh Lathia
On 2016/02/15 15:20, Rushabh Lathia wrote:
On Fri, Feb 12, 2016 at 5:40 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp <mailto:fujita.etsuro@lab.ntt.co.jp>> wrote:
As a result of our discussions, we reached a conclusion that the DML
pushdown APIs should be provided together with existing APIs such as
ExecForeignInsert, ExecForeignUpdate or ExecForeignDelete, IIUC.
So, how about (1) leaving the description for the existing APIs
as-is and (2) adding a new description for the DML pushdown APIs in
parenthesis, like this?:If the <function>IsForeignRelUpdatable</> pointer is set to
<literal>NULL</>, foreign tables are assumed to be insertable,
updatable,
or deletable if the FDW provides <function>ExecForeignInsert</>,
<function>ExecForeignUpdate</>, or <function>ExecForeignDelete</>
respectively.
(If the FDW attempts to optimize a foreign table update, it still
needs to provide PlanDMLPushdown, BeginDMLPushdown,
IterateDMLPushdown and EndDMLPushdown.)Actually, if the FDW provides the DML pushdown APIs, (pushdown-able)
foreign table updates can be done without ExecForeignInsert,
ExecForeignUpdate or ExecForeignDelete. So, the above docs are not
necessarily correct. But we don't recommend to do that without the
existing APIs, so I'm not sure it's worth complicating the docs.
Adding a new description for DML pushdown API seems good idea. I would
suggest to add that as separate paragraph rather then into brackets.
OK, will do.
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/02/12 21:19, Etsuro Fujita wrote:
Comments on specific points follow.
This seems to need minor rebasing in the wake of the join pushdown patch.
Will do.
Done.
+ /* Likewise for ForeignScan that has pushed down INSERT/UPDATE/DELETE */ + if (IsA(plan, ForeignScan) && + (((ForeignScan *) plan)->operation == CMD_INSERT || + ((ForeignScan *) plan)->operation == CMD_UPDATE || + ((ForeignScan *) plan)->operation == CMD_DELETE)) + return;I don't really understand why this is a good idea. Since target lists
are only displayed with EXPLAIN (VERBOSE), I don't really understand
what is to be gained by suppressing them in any case at all, and
therefore I don't understand why it's a good idea to do it here,
either. If there is a good reason, maybe the comment should explain
what it is.
I think that displaying target lists would be confusing for users.
There seems no objection from you (or anyone), I left that as proposed,
and added more comments.
+ /* Check point 1 */ + if (operation == CMD_INSERT) + return false; + + /* Check point 2 */ + if (nodeTag(subplan) != T_ForeignScan) + return false; + + /* Check point 3 */ + if (subplan->qual != NIL) + return false; + + /* Check point 4 */ + if (operation == CMD_UPDATE)These comments are referring to something in the function header
further up, but you could instead just delete the stuff from the
header and mention the actual conditions here.
Will fix.
Done.
The patch doesn't allow the postgres_fdw to push down an UPDATE/DELETE
on a foreign join, so I added one more condition here not to handle such
cases. (I'm planning to propose a patch to handle such cases, in the
next CF.)
- If the first condition is supposed accept only CMD_UPDATE or
CMD_DELETE, say if (operation != CMD_UPDATE || operation !=
CMD_DELETE) rather than doing it this way. Is-not-insert may in this
context be functionally equivalent to is-update-or-delete, but it's
better to write the comment and the code so that they exactly match
rather than kinda-match.- For point 2, use IsA(subplan, ForiegnScan).
Will fix.
Done.
+ /* + * ignore subplan if the FDW pushes down the command to the remote + * server + */This comment states what the code does, instead of explaining why it
does it. Please update it so that it explains why it does it.
Will update.
Done.
+ List *fdwPushdowns; /* per-target-table FDW
pushdown flags */Isn't a list of booleans an awfully inefficient representation? How
about a Bitmapset?
OK, will do.
Done.
+ /* + * Prepare remote-parameter expressions for evaluation. (Note: in + * practice, we expect that all these expressions will be just Params, so + * we could possibly do something more efficient than using the full + * expression-eval machinery for this. But probably there would be little + * benefit, and it'd require postgres_fdw to know more than is desirable + * about Param evaluation.) + */ + dpstate->param_exprs = (List *) + ExecInitExpr((Expr *) fsplan->fdw_exprs, + (PlanState *) node);This is an exact copy of an existing piece of code and its associated
comment. A good bit of the surrounding code is copied, too. You need
to refactor to avoid duplication, like by putting some of the code in
a new function that both postgresBeginForeignScan and
postgresBeginForeignModify can call.execute_dml_stmt() has some of the same disease.
Will do.
Done.
Other changes:
* I fixed docs as discussed before with Rushabh Lathia and Thom Brown.
* I keep Rushabh's code change that we call PlanDMLPushdown only when
all the required APIs are available with FDW, but for
CheckValidResultRel, I left the code as-is (no changes to that
function), to match the docs saying that the FDW needs to provide the
DML pushdown callback functions together with existing table-updating
functions such as ExecForeignInsert, ExecForeignUpdate and
ExecForeignDelete.
Best regards,
Etsuro Fujita
Attachments:
fdw-dml-pushdown-v8.patchapplication/x-patch; name=fdw-dml-pushdown-v8.patchDownload
*** a/contrib/postgres_fdw/deparse.c
--- b/contrib/postgres_fdw/deparse.c
***************
*** 1316,1321 **** deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 1316,1384 ----
}
/*
+ * deparse remote UPDATE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ deparse_expr_cxt context;
+ int nestlevel;
+ bool first;
+ ListCell *lc;
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = params_list;
+
+ appendStringInfoString(buf, "UPDATE ");
+ deparseRelation(buf, rel);
+ appendStringInfoString(buf, " SET ");
+
+ /* Make sure any constants in the exprs are printed portably */
+ nestlevel = set_transmission_modes();
+
+ first = true;
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(targetlist, attnum);
+
+ if (!first)
+ appendStringInfoString(buf, ", ");
+ first = false;
+
+ deparseColumnRef(buf, rtindex, attnum, root, false);
+ appendStringInfoString(buf, " = ");
+ deparseExpr((Expr *) tle->expr, &context);
+ }
+
+ reset_transmission_modes(nestlevel);
+
+ if (remote_conds)
+ {
+ appendStringInfo(buf, " WHERE ");
+ appendConditions(remote_conds, &context);
+ }
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* deparse remote DELETE statement
*
* The statement text is appended to buf, and we also create an integer List
***************
*** 1338,1343 **** deparseDeleteSql(StringInfo buf, PlannerInfo *root,
--- 1401,1443 ----
}
/*
+ * deparse remote DELETE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ deparse_expr_cxt context;
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = params_list;
+
+ appendStringInfoString(buf, "DELETE FROM ");
+ deparseRelation(buf, rel);
+
+ if (remote_conds)
+ {
+ appendStringInfo(buf, " WHERE ");
+ appendConditions(remote_conds, &context);
+ }
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* Add a RETURNING clause, if needed, to an INSERT/UPDATE/DELETE.
*/
static void
*** a/contrib/postgres_fdw/expected/postgres_fdw.out
--- b/contrib/postgres_fdw/expected/postgres_fdw.out
***************
*** 2258,2264 **** INSERT INTO ft2 (c1,c2,c3)
--- 2258,2283 ----
(3 rows)
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 300), c3 = (c3 || '_update3'::text) WHERE ((("C 1" % 10) = 3))
+ (3 rows)
+
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Output: c1, c2, c3, c4, c5, c6, c7, c8
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 400), c3 = (c3 || '_update7'::text) WHERE ((("C 1" % 10) = 7)) RETURNING "C 1", c2, c3, c4, c5, c6, c7, c8
+ (4 rows)
+
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
c1 | c2 | c3 | c4 | c5 | c6 | c7 | c8
------+-----+--------------------+------------------------------+--------------------------+----+------------+-----
***************
*** 2368,2374 **** UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
--- 2387,2393 ----
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
***************
*** 2393,2408 **** UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
! QUERY PLAN
! ----------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1 RETURNING "C 1", c4
! -> Foreign Scan on public.ft2
! Output: ctid
! Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) FOR UPDATE
! (6 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
--- 2412,2425 ----
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
! QUERY PLAN
! --------------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
! -> Foreign Delete on public.ft2
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) RETURNING "C 1", c4
! (4 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
***************
*** 2513,2519 **** DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
--- 2530,2536 ----
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
***************
*** 3378,3393 **** INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
(1 row)
EXPLAIN (verbose, costs off)
! UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
! QUERY PLAN
! -------------------------------------------------------------------------------------------------------------------
Update on public.ft2
Output: (tableoid)::regclass
! Remote SQL: UPDATE "S 1"."T 1" SET c3 = $2 WHERE ctid = $1
! -> Foreign Scan on public.ft2
! Output: c1, c2, NULL::integer, 'bar'::text, c4, c5, c6, c7, c8, ctid
! Remote SQL: SELECT "C 1", c2, c4, c5, c6, c7, c8, ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
! (6 rows)
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
--- 3395,3408 ----
(1 row)
EXPLAIN (verbose, costs off)
! UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
! QUERY PLAN
! ------------------------------------------------------------------------------------
Update on public.ft2
Output: (tableoid)::regclass
! -> Foreign Update on public.ft2
! Remote SQL: UPDATE "S 1"."T 1" SET c3 = 'bar'::text WHERE (("C 1" = 9999))
! (4 rows)
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
***************
*** 3396,3411 **** UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
(1 row)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
! QUERY PLAN
! ------------------------------------------------------------------------------------
Delete on public.ft2
Output: (tableoid)::regclass
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1
! -> Foreign Scan on public.ft2
! Output: ctid
! Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
! (6 rows)
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
--- 3411,3424 ----
(1 row)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
! QUERY PLAN
! --------------------------------------------------------------------
Delete on public.ft2
Output: (tableoid)::regclass
! -> Foreign Delete on public.ft2
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE (("C 1" = 9999))
! (4 rows)
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
***************
*** 3559,3565 **** CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 3572,3578 ----
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 3718,3724 **** savepoint s3;
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 3731,3737 ----
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (-2) WHERE ((c2 = 42)) AND (("C 1" = 10))
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 3858,3864 **** CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2positive;
-- But inconsistent check constraints provide inconsistent results
ALTER FOREIGN TABLE ft1 ADD CONSTRAINT ft1_c2negative CHECK (c2 < 0);
--- 3871,3877 ----
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2positive;
-- But inconsistent check constraints provide inconsistent results
ALTER FOREIGN TABLE ft1 ADD CONSTRAINT ft1_c2negative CHECK (c2 < 0);
***************
*** 4251,4256 **** NOTICE: NEW: (13,"test triggered !")
--- 4264,4462 ----
(0,27)
(1 row)
+ -- cleanup
+ DROP TRIGGER trig_row_before ON rem1;
+ DROP TRIGGER trig_row_after ON rem1;
+ DROP TRIGGER trig_local_before ON loc1;
+ -- Test DML pushdown functionality
+ -- Test with statement-level triggers
+ CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_stmt_before ON rem1;
+ CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_stmt_after ON rem1;
+ -- Test with row-level ON INSERT triggers
+ CREATE TRIGGER trig_row_before_insert
+ BEFORE INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_before_insert ON rem1;
+ CREATE TRIGGER trig_row_after_insert
+ AFTER INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_after_insert ON rem1;
+ -- Test with row-level ON UPDATE triggers
+ CREATE TRIGGER trig_row_before_update
+ BEFORE UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_before_update ON rem1;
+ CREATE TRIGGER trig_row_after_update
+ AFTER UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+ -------------------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_after_update ON rem1;
+ -- Test with row-level ON DELETE triggers
+ CREATE TRIGGER trig_row_before_delete
+ BEFORE DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ DROP TRIGGER trig_row_before_delete ON rem1;
+ CREATE TRIGGER trig_row_after_delete
+ AFTER DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+ ------------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ DROP TRIGGER trig_row_after_delete ON rem1;
-- ===================================================================
-- test inheritance features
-- ===================================================================
***************
*** 4720,4725 **** fetch from c;
--- 4926,4981 ----
update bar set f2 = null where current of c;
ERROR: WHERE CURRENT OF is not supported for this table type
rollback;
+ explain (verbose, costs off)
+ delete from foo where f1 < 5 returning *;
+ QUERY PLAN
+ --------------------------------------------------------------------------------
+ Delete on public.foo
+ Output: foo.f1, foo.f2
+ Delete on public.foo
+ Foreign Delete on public.foo2
+ -> Index Scan using i_foo_f1 on public.foo
+ Output: foo.ctid
+ Index Cond: (foo.f1 < 5)
+ -> Foreign Delete on public.foo2
+ Remote SQL: DELETE FROM public.loct1 WHERE ((f1 < 5)) RETURNING f1, f2
+ (9 rows)
+
+ delete from foo where f1 < 5 returning *;
+ f1 | f2
+ ----+----
+ 1 | 1
+ 3 | 3
+ 0 | 0
+ 2 | 2
+ 4 | 4
+ (5 rows)
+
+ explain (verbose, costs off)
+ update bar set f2 = f2 + 100 returning *;
+ QUERY PLAN
+ ------------------------------------------------------------------------------
+ Update on public.bar
+ Output: bar.f1, bar.f2
+ Update on public.bar
+ Foreign Update on public.bar2
+ -> Seq Scan on public.bar
+ Output: bar.f1, (bar.f2 + 100), bar.ctid
+ -> Foreign Update on public.bar2
+ Remote SQL: UPDATE public.loct2 SET f2 = (f2 + 100) RETURNING f1, f2
+ (8 rows)
+
+ update bar set f2 = f2 + 100 returning *;
+ f1 | f2
+ ----+-----
+ 1 | 311
+ 2 | 322
+ 6 | 266
+ 3 | 333
+ 4 | 344
+ 7 | 277
+ (6 rows)
+
drop table foo cascade;
NOTICE: drop cascades to foreign table foo2
drop table bar cascade;
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
***************
*** 61,66 **** enum FdwScanPrivateIndex
--- 61,68 ----
{
/* SQL statement to execute remotely (as a String node) */
FdwScanPrivateSelectSql,
+ /* List of restriction clauses that can be executed remotely */
+ FdwScanPrivateRemoteConds,
/* Integer list of attribute numbers retrieved by the SELECT */
FdwScanPrivateRetrievedAttrs,
/* Integer representing the desired fetch_size */
***************
*** 98,103 **** enum FdwModifyPrivateIndex
--- 100,127 ----
};
/*
+ * Similarly, this enum describes what's kept in the fdw_private list for
+ * a ForeignScan node that has pushed down an UPDATE/DELETE to the remote
+ * server. We store:
+ *
+ * 1) UPDATE/DELETE statement text to be sent to the remote server
+ * 2) Boolean flag showing if the remote query has a RETURNING clause
+ * 3) Integer list of attribute numbers retrieved by RETURNING, if any
+ * 4) Boolean flag showing if we set the command es_processed
+ */
+ enum FdwDmlPushdownPrivateIndex
+ {
+ /* SQL statement to execute remotely (as a String node) */
+ FdwDmlPushdownPrivateUpdateSql,
+ /* has-returning flag (as an integer Value node) */
+ FdwDmlPushdownPrivateHasReturning,
+ /* Integer list of attribute numbers retrieved by RETURNING */
+ FdwDmlPushdownPrivateRetrievedAttrs,
+ /* set-processed flag (as an integer Value node) */
+ FdwDmlPushdownPrivateSetProcessed
+ };
+
+ /*
* Execution state of a foreign scan using postgres_fdw.
*/
typedef struct PgFdwScanState
***************
*** 164,169 **** typedef struct PgFdwModifyState
--- 188,224 ----
} PgFdwModifyState;
/*
+ * Execution state of a foreign scan that has pushed down a foreign table
+ * modification to the remote server
+ */
+ typedef struct PgFdwDmlPushdownState
+ {
+ Relation rel; /* relcache entry for the foreign table */
+ AttInMetadata *attinmeta; /* attribute datatype conversion metadata */
+
+ /* extracted fdw_private data */
+ char *query; /* text of UPDATE/DELETE command */
+ bool has_returning; /* is there a RETURNING clause? */
+ List *retrieved_attrs; /* attr numbers retrieved by RETURNING */
+ bool set_processed; /* do we set the command es_processed? */
+
+ /* for remote query execution */
+ PGconn *conn; /* connection for the update */
+ int numParams; /* number of parameters passed to query */
+ FmgrInfo *param_flinfo; /* output conversion functions for them */
+ List *param_exprs; /* executable expressions for param values */
+ const char **param_values; /* textual values of query parameters */
+
+ /* for storing result tuples */
+ PGresult *result; /* result for query */
+ int num_tuples; /* # of result tuples */
+ int next_tuple; /* index of next one to return */
+
+ /* working memory context */
+ MemoryContext temp_cxt; /* context for per-tuple temporary data */
+ } PgFdwDmlPushdownState;
+
+ /*
* Workspace for analyzing a foreign table.
*/
typedef struct PgFdwAnalyzeState
***************
*** 263,268 **** static TupleTableSlot *postgresExecForeignDelete(EState *estate,
--- 318,330 ----
static void postgresEndForeignModify(EState *estate,
ResultRelInfo *resultRelInfo);
static int postgresIsForeignRelUpdatable(Relation rel);
+ static bool postgresPlanDMLPushdown(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+ static void postgresBeginDMLPushdown(ForeignScanState *node, int eflags);
+ static TupleTableSlot *postgresIterateDMLPushdown(ForeignScanState *node);
+ static void postgresEndDMLPushdown(ForeignScanState *node);
static void postgresExplainForeignScan(ForeignScanState *node,
ExplainState *es);
static void postgresExplainForeignModify(ModifyTableState *mtstate,
***************
*** 270,275 **** static void postgresExplainForeignModify(ModifyTableState *mtstate,
--- 332,339 ----
List *fdw_private,
int subplan_index,
ExplainState *es);
+ static void postgresExplainDMLPushdown(ForeignScanState *node,
+ ExplainState *es);
static bool postgresAnalyzeForeignTable(Relation relation,
AcquireSampleRowsFunc *func,
BlockNumber *totalpages);
***************
*** 314,319 **** static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
--- 378,395 ----
TupleTableSlot *slot);
static void store_returning_result(PgFdwModifyState *fmstate,
TupleTableSlot *slot, PGresult *res);
+ static void execute_dml_stmt(ForeignScanState *node);
+ static TupleTableSlot *get_returning_data(ForeignScanState *node);
+ static void prepare_query_params(PlanState *node,
+ List *fdw_exprs,
+ int numParams,
+ FmgrInfo **param_flinfo,
+ List **param_exprs,
+ const char ***param_values);
+ static void process_query_params(ExprContext *econtext,
+ FmgrInfo *param_flinfo,
+ List *param_exprs,
+ const char **param_values);
static int postgresAcquireSampleRowsFunc(Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
***************
*** 360,371 **** postgres_fdw_handler(PG_FUNCTION_ARGS)
--- 436,452 ----
routine->ExecForeignDelete = postgresExecForeignDelete;
routine->EndForeignModify = postgresEndForeignModify;
routine->IsForeignRelUpdatable = postgresIsForeignRelUpdatable;
+ routine->PlanDMLPushdown = postgresPlanDMLPushdown;
+ routine->BeginDMLPushdown = postgresBeginDMLPushdown;
+ routine->IterateDMLPushdown = postgresIterateDMLPushdown;
+ routine->EndDMLPushdown = postgresEndDMLPushdown;
/* Function for EvalPlanQual rechecks */
routine->RecheckForeignScan = postgresRecheckForeignScan;
/* Support functions for EXPLAIN */
routine->ExplainForeignScan = postgresExplainForeignScan;
routine->ExplainForeignModify = postgresExplainForeignModify;
+ routine->ExplainDMLPushdown = postgresExplainDMLPushdown;
/* Support functions for ANALYZE */
routine->AnalyzeForeignTable = postgresAnalyzeForeignTable;
***************
*** 1132,1138 **** postgresGetForeignPlan(PlannerInfo *root,
* Build the fdw_private list that will be available to the executor.
* Items in the list must match order in enum FdwScanPrivateIndex.
*/
! fdw_private = list_make4(makeString(sql.data),
retrieved_attrs,
makeInteger(fpinfo->fetch_size),
makeInteger(foreignrel->umid));
--- 1213,1220 ----
* Build the fdw_private list that will be available to the executor.
* Items in the list must match order in enum FdwScanPrivateIndex.
*/
! fdw_private = list_make5(makeString(sql.data),
! remote_conds,
retrieved_attrs,
makeInteger(fpinfo->fetch_size),
makeInteger(foreignrel->umid));
***************
*** 1169,1176 **** postgresBeginForeignScan(ForeignScanState *node, int eflags)
PgFdwScanState *fsstate;
UserMapping *user;
int numParams;
- int i;
- ListCell *lc;
/*
* Do nothing in EXPLAIN (no ANALYZE) case. node->fdw_state stays NULL.
--- 1251,1256 ----
***************
*** 1257,1298 **** postgresBeginForeignScan(ForeignScanState *node, int eflags)
fsstate->attinmeta = TupleDescGetAttInMetadata(fsstate->tupdesc);
- /* Prepare for output conversion of parameters used in remote query. */
- numParams = list_length(fsplan->fdw_exprs);
- fsstate->numParams = numParams;
- fsstate->param_flinfo = (FmgrInfo *) palloc0(sizeof(FmgrInfo) * numParams);
-
- i = 0;
- foreach(lc, fsplan->fdw_exprs)
- {
- Node *param_expr = (Node *) lfirst(lc);
- Oid typefnoid;
- bool isvarlena;
-
- getTypeOutputInfo(exprType(param_expr), &typefnoid, &isvarlena);
- fmgr_info(typefnoid, &fsstate->param_flinfo[i]);
- i++;
- }
-
- /*
- * Prepare remote-parameter expressions for evaluation. (Note: in
- * practice, we expect that all these expressions will be just Params, so
- * we could possibly do something more efficient than using the full
- * expression-eval machinery for this. But probably there would be little
- * benefit, and it'd require postgres_fdw to know more than is desirable
- * about Param evaluation.)
- */
- fsstate->param_exprs = (List *)
- ExecInitExpr((Expr *) fsplan->fdw_exprs,
- (PlanState *) node);
-
/*
! * Allocate buffer for text form of query parameters, if any.
*/
if (numParams > 0)
! fsstate->param_values = (const char **) palloc0(numParams * sizeof(char *));
! else
! fsstate->param_values = NULL;
}
/*
--- 1337,1354 ----
fsstate->attinmeta = TupleDescGetAttInMetadata(fsstate->tupdesc);
/*
! * Prepare for processing of parameters used in remote query, if any.
*/
+ numParams = list_length(fsplan->fdw_exprs);
+ fsstate->numParams = numParams;
if (numParams > 0)
! prepare_query_params((PlanState *) node,
! fsplan->fdw_exprs,
! numParams,
! &fsstate->param_flinfo,
! &fsstate->param_exprs,
! &fsstate->param_values);
}
/*
***************
*** 1457,1469 **** postgresAddForeignUpdateTargets(Query *parsetree,
/*
* postgresPlanForeignModify
* Plan an insert/update/delete operation on a foreign table
- *
- * Note: currently, the plan tree generated for UPDATE/DELETE will always
- * include a ForeignScan that retrieves ctids (using SELECT FOR UPDATE)
- * and then the ModifyTable node will have to execute individual remote
- * UPDATE/DELETE commands. If there are no local conditions or joins
- * needed, it'd be better to let the scan node do UPDATE/DELETE RETURNING
- * and then do nothing at ModifyTable. Room for future optimization ...
*/
static List *
postgresPlanForeignModify(PlannerInfo *root,
--- 1513,1518 ----
***************
*** 2002,2007 **** postgresRecheckForeignScan(ForeignScanState *node, TupleTableSlot *slot)
--- 2051,2364 ----
}
/*
+ * postgresPlanDMLPushdown
+ * Consider pushing down a foreign table modification to the remote server
+ *
+ * Decide whether the table modification is safe to push down to the remote end,
+ * and if so, modify subplan so as to do that.
+ */
+ static bool
+ postgresPlanDMLPushdown(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index)
+ {
+ CmdType operation = plan->operation;
+ Plan *subplan = (Plan *) list_nth(plan->plans, subplan_index);
+ RangeTblEntry *rte = planner_rt_fetch(resultRelation, root);
+ Relation rel;
+ StringInfoData sql;
+ ForeignScan *fscan;
+ List *targetAttrs = NIL;
+ List *remote_conds;
+ List *params_list = NIL;
+ List *returningList = NIL;
+ List *retrieved_attrs = NIL;
+
+ /*
+ * Decide whether the table modification is pushdown-safe.
+ */
+
+ /*
+ * 1. The table modification must be an UPDATE or DELETE.
+ */
+ if (operation != CMD_UPDATE && operation != CMD_DELETE)
+ return false;
+
+ /*
+ * 2. It's unsafe to push down the command if there are any local joins
+ * needed.
+ */
+ if (!IsA(subplan, ForeignScan))
+ return false;
+
+ /*
+ * 3. It's unsafe to push down the command if there are any quals that
+ * can't be evaluated remotely.
+ */
+ if (subplan->qual != NIL)
+ return false;
+
+ /*
+ * 4. We can't push down an UPDATE, if any expressions to assign to the
+ * target columns are unsafe to evaluate on the remote end.
+ */
+ if (operation == CMD_UPDATE)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[resultRelation];
+ int col;
+
+ /*
+ * We transmit only columns that were explicitly targets of the UPDATE,
+ * so as to avoid unnecessary data transmission.
+ */
+ col = -1;
+ while ((col = bms_next_member(rte->updatedCols, col)) >= 0)
+ {
+ /* bit numbers are offset by FirstLowInvalidHeapAttributeNumber */
+ AttrNumber attno = col + FirstLowInvalidHeapAttributeNumber;
+ TargetEntry *tle;
+
+ if (attno <= InvalidAttrNumber) /* shouldn't happen */
+ elog(ERROR, "system-column update is not supported");
+
+ tle = get_tle_by_resno(subplan->targetlist, attno);
+
+ if (!is_foreign_expr(root, baserel, (Expr *) tle->expr))
+ return false;
+
+ targetAttrs = lappend_int(targetAttrs, attno);
+ }
+ }
+
+ /*
+ * 5. We can't push down an UPDATE or DELETE on a foreign join for now.
+ */
+ fscan = (ForeignScan *) subplan;
+ if (fscan->scan.scanrelid == 0)
+ return false;
+
+ /*
+ * Ok, modify subplan so as to push down the command to the remote server.
+ */
+ initStringInfo(&sql);
+
+ /*
+ * Core code already has some lock on each rel being planned, so we can
+ * use NoLock here.
+ */
+ rel = heap_open(rte->relid, NoLock);
+
+ /*
+ * Extract the baserestrictinfo clauses that can be evaluated remotely.
+ */
+ remote_conds = (List *) list_nth(fscan->fdw_private,
+ FdwScanPrivateRemoteConds);
+
+ /*
+ * Extract the relevant RETURNING list if any.
+ */
+ if (plan->returningLists)
+ returningList = (List *) list_nth(plan->returningLists, subplan_index);
+
+ /*
+ * Construct the SQL command string.
+ */
+ switch (operation)
+ {
+ case CMD_UPDATE:
+ deparsePushedDownUpdateSql(&sql, root, resultRelation, rel,
+ ((Plan *) fscan)->targetlist,
+ targetAttrs,
+ remote_conds, ¶ms_list,
+ returningList, &retrieved_attrs);
+ break;
+ case CMD_DELETE:
+ deparsePushedDownDeleteSql(&sql, root, resultRelation, rel,
+ remote_conds, ¶ms_list,
+ returningList, &retrieved_attrs);
+ break;
+ default:
+ elog(ERROR, "unexpected operation: %d", (int) operation);
+ break;
+ }
+
+ /*
+ * Update the operation info.
+ */
+ fscan->operation = operation;
+
+ /*
+ * Update the fdw_exprs list that will be available to the executor.
+ */
+ fscan->fdw_exprs = params_list;
+
+ /*
+ * Update the fdw_private list that will be available to the executor.
+ * Items in the list must match enum FdwDmlPushdownPrivateIndex, above.
+ */
+ fscan->fdw_private = list_make4(makeString(sql.data),
+ makeInteger((retrieved_attrs != NIL)),
+ retrieved_attrs,
+ makeInteger(plan->canSetTag));
+
+ heap_close(rel, NoLock);
+ return true;
+ }
+
+ /*
+ * postgresBeginDMLPushdown
+ * Initiate pushing down a foreign table modification to the remote server
+ */
+ static void
+ postgresBeginDMLPushdown(ForeignScanState *node, int eflags)
+ {
+ ForeignScan *fsplan = (ForeignScan *) node->ss.ps.plan;
+ EState *estate = node->ss.ps.state;
+ PgFdwDmlPushdownState *dpstate;
+ RangeTblEntry *rte;
+ Oid userid;
+ ForeignTable *table;
+ UserMapping *user;
+ int numParams;
+
+ /*
+ * Do nothing in EXPLAIN (no ANALYZE) case. node->fdw_state stays NULL.
+ */
+ if (eflags & EXEC_FLAG_EXPLAIN_ONLY)
+ return;
+
+ /*
+ * We'll save private state in node->fdw_state.
+ */
+ dpstate = (PgFdwDmlPushdownState *) palloc0(sizeof(PgFdwDmlPushdownState));
+ node->fdw_state = (void *) dpstate;
+
+ /*
+ * Identify which user to do the remote access as. This should match what
+ * ExecCheckRTEPerms() does.
+ */
+ rte = rt_fetch(fsplan->scan.scanrelid, estate->es_range_table);
+ userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
+
+ /* Get info about foreign table. */
+ dpstate->rel = node->ss.ss_currentRelation;
+ table = GetForeignTable(RelationGetRelid(dpstate->rel));
+ user = GetUserMapping(userid, table->serverid);
+
+ /*
+ * Get connection to the foreign server. Connection manager will
+ * establish new connection if necessary.
+ */
+ dpstate->conn = GetConnection(user, false);
+
+ /* Initialize state variable */
+ dpstate->num_tuples = -1; /* -1 means not set yet */
+
+ /* Get private info created by planner functions. */
+ dpstate->query = strVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateUpdateSql));
+ dpstate->has_returning = intVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateHasReturning));
+ dpstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateRetrievedAttrs);
+ dpstate->set_processed = intVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateSetProcessed));
+
+ /* Create context for per-tuple temp workspace. */
+ dpstate->temp_cxt = AllocSetContextCreate(estate->es_query_cxt,
+ "postgres_fdw temporary data",
+ ALLOCSET_SMALL_MINSIZE,
+ ALLOCSET_SMALL_INITSIZE,
+ ALLOCSET_SMALL_MAXSIZE);
+
+ /* Prepare for input conversion of RETURNING results. */
+ if (dpstate->has_returning)
+ dpstate->attinmeta = TupleDescGetAttInMetadata(RelationGetDescr(dpstate->rel));
+
+ /*
+ * Prepare for processing of parameters used in remote query, if any.
+ */
+ numParams = list_length(fsplan->fdw_exprs);
+ dpstate->numParams = numParams;
+ if (numParams > 0)
+ prepare_query_params((PlanState *) node,
+ fsplan->fdw_exprs,
+ numParams,
+ &dpstate->param_flinfo,
+ &dpstate->param_exprs,
+ &dpstate->param_values);
+ }
+
+ /*
+ * postgresIterateDMLPushdown
+ * Execute pushing down a foreign table modification to the remote server
+ */
+ static TupleTableSlot *
+ postgresIterateDMLPushdown(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ EState *estate = node->ss.ps.state;
+ ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
+
+ /*
+ * If this is the first call after Begin, execute the statement.
+ */
+ if (dpstate->num_tuples == -1)
+ execute_dml_stmt(node);
+
+ /*
+ * If the local query doesn't specify RETURNING, just clear tuple slot.
+ */
+ if (!resultRelInfo->ri_projectReturning)
+ {
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+ Instrumentation *instr = node->ss.ps.instrument;
+
+ Assert(!dpstate->has_returning);
+
+ /* Increment the command es_processed count if necessary. */
+ if (dpstate->set_processed)
+ estate->es_processed += dpstate->num_tuples;
+
+ /* Increment the tuple count for EXPLAIN ANALYZE if necessary. */
+ if (instr)
+ instr->tuplecount += dpstate->num_tuples;
+
+ return ExecClearTuple(slot);
+ }
+
+ /*
+ * Get the next RETURNING tuple.
+ */
+ return get_returning_data(node);
+ }
+
+ /*
+ * postgresEndDMLPushdown
+ * Finish pushing down a foreign table modification to the remote server
+ */
+ static void
+ postgresEndDMLPushdown(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+
+ /* if dpstate is NULL, we are in EXPLAIN; nothing to do */
+ if (dpstate == NULL)
+ return;
+
+ /* Release PGresult */
+ if (dpstate->result)
+ PQclear(dpstate->result);
+
+ /* Release remote connection */
+ ReleaseConnection(dpstate->conn);
+ dpstate->conn = NULL;
+
+ /* MemoryContext will be deleted automatically. */
+ }
+
+ /*
* postgresExplainForeignScan
* Produce extra output for EXPLAIN of a ForeignScan on a foreign table
*/
***************
*** 2054,2059 **** postgresExplainForeignModify(ModifyTableState *mtstate,
--- 2411,2435 ----
}
}
+ /*
+ * postgresExplainDMLPushdown
+ * Produce extra output for EXPLAIN of a ForeignScan on a foreign table
+ * that has pushed down an UPDATE/DELETE to the remote server
+ */
+ static void
+ postgresExplainDMLPushdown(ForeignScanState *node, ExplainState *es)
+ {
+ List *fdw_private;
+ char *sql;
+
+ if (es->verbose)
+ {
+ fdw_private = ((ForeignScan *) node->ss.ps.plan)->fdw_private;
+ sql = strVal(list_nth(fdw_private, FdwDmlPushdownPrivateUpdateSql));
+ ExplainPropertyText("Remote SQL", sql, es);
+ }
+ }
+
/*
* estimate_path_cost_size
***************
*** 2412,2449 **** create_cursor(ForeignScanState *node)
*/
if (numParams > 0)
{
- int nestlevel;
MemoryContext oldcontext;
- int i;
- ListCell *lc;
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! nestlevel = set_transmission_modes();
!
! i = 0;
! foreach(lc, fsstate->param_exprs)
! {
! ExprState *expr_state = (ExprState *) lfirst(lc);
! Datum expr_value;
! bool isNull;
!
! /* Evaluate the parameter expression */
! expr_value = ExecEvalExpr(expr_state, econtext, &isNull, NULL);
!
! /*
! * Get string representation of each parameter value by invoking
! * type-specific output function, unless the value is null.
! */
! if (isNull)
! values[i] = NULL;
! else
! values[i] = OutputFunctionCall(&fsstate->param_flinfo[i],
! expr_value);
! i++;
! }
!
! reset_transmission_modes(nestlevel);
MemoryContextSwitchTo(oldcontext);
}
--- 2788,2801 ----
*/
if (numParams > 0)
{
MemoryContext oldcontext;
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! process_query_params(econtext,
! fsstate->param_flinfo,
! fsstate->param_exprs,
! values);
MemoryContextSwitchTo(oldcontext);
}
***************
*** 2764,2769 **** store_returning_result(PgFdwModifyState *fmstate,
--- 3116,3312 ----
}
/*
+ * Execute a pushed-down UPDATE/DELETE statement.
+ */
+ static void
+ execute_dml_stmt(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ ExprContext *econtext = node->ss.ps.ps_ExprContext;
+ int numParams = dpstate->numParams;
+ const char **values = dpstate->param_values;
+
+ /*
+ * Construct array of query parameter values in text format.
+ */
+ if (numParams > 0)
+ process_query_params(econtext,
+ dpstate->param_flinfo,
+ dpstate->param_exprs,
+ values);
+
+ /*
+ * Notice that we pass NULL for paramTypes, thus forcing the remote server
+ * to infer types for all parameters. Since we explicitly cast every
+ * parameter (see deparse.c), the "inference" is trivial and will produce
+ * the desired result. This allows us to avoid assuming that the remote
+ * server has the same OIDs we do for the parameters' types.
+ *
+ * We don't use a PG_TRY block here, so be careful not to throw error
+ * without releasing the PGresult.
+ */
+ dpstate->result = PQexecParams(dpstate->conn, dpstate->query,
+ numParams, NULL, values, NULL, NULL, 0);
+ if (PQresultStatus(dpstate->result) !=
+ (dpstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
+ pgfdw_report_error(ERROR, dpstate->result, dpstate->conn, true,
+ dpstate->query);
+
+ /* Get the number of rows affected. */
+ if (dpstate->has_returning)
+ dpstate->num_tuples = PQntuples(dpstate->result);
+ else
+ dpstate->num_tuples = atoi(PQcmdTuples(dpstate->result));
+ }
+
+ /*
+ * Get the result of a RETURNING clause.
+ */
+ static TupleTableSlot *
+ get_returning_data(ForeignScanState *node)
+ {
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ EState *estate = node->ss.ps.state;
+ ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+
+ Assert(resultRelInfo->ri_projectReturning);
+
+ /* If we didn't get any tuples, must be end of data. */
+ if (dpstate->next_tuple >= dpstate->num_tuples)
+ return ExecClearTuple(slot);
+
+ /* Increment the command es_processed count if necessary. */
+ if (dpstate->set_processed)
+ estate->es_processed += 1;
+
+ /*
+ * Store a RETURNING tuple. If has_returning is false, just emit a dummy
+ * tuple. (We have has_returning=false if the local query is of the form
+ * UPDATE/DELETE .. RETURNING 1 for example.)
+ */
+ if (!dpstate->has_returning)
+ ExecStoreAllNullTuple(slot);
+ else
+ {
+ /*
+ * On error, be sure to release the PGresult on the way out. Callers
+ * do not have PG_TRY blocks to ensure this happens.
+ */
+ PG_TRY();
+ {
+ HeapTuple newtup;
+
+ newtup = make_tuple_from_result_row(dpstate->result,
+ dpstate->next_tuple,
+ dpstate->rel,
+ dpstate->attinmeta,
+ dpstate->retrieved_attrs,
+ NULL,
+ dpstate->temp_cxt);
+ ExecStoreTuple(newtup, slot, InvalidBuffer, false);
+ }
+ PG_CATCH();
+ {
+ if (dpstate->result)
+ PQclear(dpstate->result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ }
+ dpstate->next_tuple++;
+
+ /* Make slot available for evaluation of the local query RETURNING list. */
+ resultRelInfo->ri_projectReturning->pi_exprContext->ecxt_scantuple = slot;
+
+ return slot;
+ }
+
+ /*
+ * Prepare for processing of parameters used in remote query.
+ */
+ static void
+ prepare_query_params(PlanState *node,
+ List *fdw_exprs,
+ int numParams,
+ FmgrInfo **param_flinfo,
+ List **param_exprs,
+ const char ***param_values)
+ {
+ int i;
+ ListCell *lc;
+
+ Assert(numParams > 0);
+
+ /* Prepare for output conversion of parameters used in remote query. */
+ *param_flinfo = (FmgrInfo *) palloc0(sizeof(FmgrInfo) * numParams);
+
+ i = 0;
+ foreach(lc, fdw_exprs)
+ {
+ Node *param_expr = (Node *) lfirst(lc);
+ Oid typefnoid;
+ bool isvarlena;
+
+ getTypeOutputInfo(exprType(param_expr), &typefnoid, &isvarlena);
+ fmgr_info(typefnoid, &(*param_flinfo)[i]);
+ i++;
+ }
+
+ /*
+ * Prepare remote-parameter expressions for evaluation. (Note: in
+ * practice, we expect that all these expressions will be just Params, so
+ * we could possibly do something more efficient than using the full
+ * expression-eval machinery for this. But probably there would be little
+ * benefit, and it'd require postgres_fdw to know more than is desirable
+ * about Param evaluation.)
+ */
+ *param_exprs = (List *) ExecInitExpr((Expr *) fdw_exprs, node);
+
+ /* Allocate buffer for text form of query parameters. */
+ *param_values = (const char **) palloc0(numParams * sizeof(char *));
+ }
+
+ /*
+ * Construct array of query parameter values in text format.
+ */
+ static void
+ process_query_params(ExprContext *econtext,
+ FmgrInfo *param_flinfo,
+ List *param_exprs,
+ const char **param_values)
+ {
+ int nestlevel;
+ int i;
+ ListCell *lc;
+
+ nestlevel = set_transmission_modes();
+
+ i = 0;
+ foreach(lc, param_exprs)
+ {
+ ExprState *expr_state = (ExprState *) lfirst(lc);
+ Datum expr_value;
+ bool isNull;
+
+ /* Evaluate the parameter expression */
+ expr_value = ExecEvalExpr(expr_state, econtext, &isNull, NULL);
+
+ /*
+ * Get string representation of each parameter value by invoking
+ * type-specific output function, unless the value is null.
+ */
+ if (isNull)
+ param_values[i] = NULL;
+ else
+ param_values[i] = OutputFunctionCall(¶m_flinfo[i], expr_value);
+ i++;
+ }
+
+ reset_transmission_modes(nestlevel);
+ }
+
+ /*
* postgresAnalyzeForeignTable
* Test whether analyzing this foreign table is supported
*/
*** a/contrib/postgres_fdw/postgres_fdw.h
--- b/contrib/postgres_fdw/postgres_fdw.h
***************
*** 130,139 **** extern void deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 130,153 ----
Index rtindex, Relation rel,
List *targetAttrs, List *returningList,
List **retrieved_attrs);
+ extern void deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseDeleteSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *returningList,
List **retrieved_attrs);
+ extern void deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseAnalyzeSizeSql(StringInfo buf, Relation rel);
extern void deparseAnalyzeSql(StringInfo buf, Relation rel,
List **retrieved_attrs);
*** a/contrib/postgres_fdw/sql/postgres_fdw.sql
--- b/contrib/postgres_fdw/sql/postgres_fdw.sql
***************
*** 599,626 **** INSERT INTO ft2 (c1,c2,c3) SELECT c1+1000,c2+100, c3 || c3 FROM ft2 LIMIT 20;
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
EXPLAIN (verbose, costs off)
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
! UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
-- Test that trigger on remote table works as expected
--- 599,630 ----
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
EXPLAIN (verbose, costs off)
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
! UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
-- Test that trigger on remote table works as expected
***************
*** 937,942 **** UPDATE rem1 SET f2 = 'testo';
--- 941,1030 ----
-- Test returning a system attribute
INSERT INTO rem1(f2) VALUES ('test') RETURNING ctid;
+ -- cleanup
+ DROP TRIGGER trig_row_before ON rem1;
+ DROP TRIGGER trig_row_after ON rem1;
+ DROP TRIGGER trig_local_before ON loc1;
+
+
+ -- Test DML pushdown functionality
+
+ -- Test with statement-level triggers
+ CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_stmt_before ON rem1;
+
+ CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_stmt_after ON rem1;
+
+ -- Test with row-level ON INSERT triggers
+ CREATE TRIGGER trig_row_before_insert
+ BEFORE INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_before_insert ON rem1;
+
+ CREATE TRIGGER trig_row_after_insert
+ AFTER INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_after_insert ON rem1;
+
+ -- Test with row-level ON UPDATE triggers
+ CREATE TRIGGER trig_row_before_update
+ BEFORE UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_before_update ON rem1;
+
+ CREATE TRIGGER trig_row_after_update
+ AFTER UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_after_update ON rem1;
+
+ -- Test with row-level ON DELETE triggers
+ CREATE TRIGGER trig_row_before_delete
+ BEFORE DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ DROP TRIGGER trig_row_before_delete ON rem1;
+
+ CREATE TRIGGER trig_row_after_delete
+ AFTER DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ DROP TRIGGER trig_row_after_delete ON rem1;
+
-- ===================================================================
-- test inheritance features
-- ===================================================================
***************
*** 1068,1073 **** fetch from c;
--- 1156,1168 ----
update bar set f2 = null where current of c;
rollback;
+ explain (verbose, costs off)
+ delete from foo where f1 < 5 returning *;
+ delete from foo where f1 < 5 returning *;
+ explain (verbose, costs off)
+ update bar set f2 = f2 + 100 returning *;
+ update bar set f2 = f2 + 100 returning *;
+
drop table foo cascade;
drop table bar cascade;
drop table loct1;
*** a/doc/src/sgml/fdwhandler.sgml
--- b/doc/src/sgml/fdwhandler.sgml
***************
*** 678,683 **** IsForeignRelUpdatable (Relation rel);
--- 678,828 ----
updatability for display in the <literal>information_schema</> views.)
</para>
+ <para>
+ If an FDW supports optimizing foreign table updates, it still needs to
+ provide <function>PlanDMLPushdown</>, <function>BeginDMLPushdown</>,
+ <function>IterateDMLPushdown</> and <function>EndDMLPushdown</>
+ described below.
+ </para>
+
+ <para>
+ <programlisting>
+ bool
+ PlanDMLPushdown (PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+ </programlisting>
+
+ Decide whether it is safe to execute a foreign table update directly
+ on the remote server. If so, return <literal>true</> after performing
+ planning actions needed for that. Otherwise, return <literal>false</>.
+ This optional function is called during query planning.
+ If this function succeeds, <function>BeginDMLPushdown</>,
+ <function>IterateDMLPushdown</> and <function>EndDMLPushdown</> will be
+ called at the execution stage, instead. Otherwise, the table update
+ will be executed using the table-updating functions described above.
+ The parameters are the same as for <function>PlanForeignModify</>.
+ </para>
+
+ <para>
+ To execute the table update directly on the remote server, this function
+ must rewrite the target subplan with a <structname>ForeignScan</> plan
+ node that executes the table update directly on the remote server. The
+ <structfield>operation</> field of the <structname>ForeignScan</> must
+ be set to the <literal>CmdType</> enumeration appropriately; that is,
+ <literal>CMD_UPDATE</> for <command>UPDATE</>,
+ <literal>CMD_INSERT</> for <command>INSERT</>, and
+ <literal>CMD_DELETE</> for <command>DELETE</>.
+ </para>
+
+ <para>
+ See <xref linkend="fdw-planning"> for additional information.
+ </para>
+
+ <para>
+ If the <function>PlanDMLPushdown</> pointer is set to
+ <literal>NULL</>, no attempts to execute the table update directly on
+ the remote server are taken.
+ </para>
+
+ <para>
+ <programlisting>
+ void
+ BeginDMLPushdown (ForeignScanState *node,
+ int eflags);
+ </programlisting>
+
+ Begin executing a foreign table update directly on the remote server.
+ This is called during executor startup. It should perform any
+ initialization needed prior to the actual table update (that should be
+ done upon the first call to <function>IterateDMLPushdown</>).
+ The <structname>ForeignScanState</> node has already been created, but
+ its <structfield>fdw_state</> field is still NULL. Information about
+ the table to update is accessible through the
+ <structname>ForeignScanState</> node (in particular, from the underlying
+ <structname>ForeignScan</> plan node, which contains any FDW-private
+ information provided by <function>PlanDMLPushdown</>).
+ <literal>eflags</> contains flag bits describing the executor's
+ operating mode for this plan node.
+ </para>
+
+ <para>
+ Note that when <literal>(eflags & EXEC_FLAG_EXPLAIN_ONLY)</> is
+ true, this function should not perform any externally-visible actions;
+ it should only do the minimum required to make the node state valid
+ for <function>ExplainDMLPushdown</> and <function>EndDMLPushdown</>.
+ </para>
+
+ <para>
+ If the <function>BeginDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to execute the table update directly on
+ the remote server will fail with an error message.
+ </para>
+
+ <para>
+ <programlisting>
+ TupleTableSlot *
+ IterateDMLPushdown (ForeignScanState *node);
+ </programlisting>
+
+ When the <command>INSERT</>, <command>UPDATE</> or <command>DELETE</>
+ query doesn't have a <literal>RETURNING</> clause, just return NULL
+ after the actual table update directly executed on the remote server.
+ When the query has the clause, fetch one result containing the data
+ needed for the <literal>RETURNING</> calculation, returning it in a
+ tuple table slot (the node's <structfield>ScanTupleSlot</> should be
+ used for this purpose). The data that was actually inserted, updated
+ or deleted must be stored in the
+ <literal>es_result_relation_info->ri_projectReturning->pi_exprContext->ecxt_scantuple</>
+ of the node's <structname>EState</>.
+ Return NULL if no more rows are available.
+ Note that this is called in a short-lived memory context that will be
+ reset between invocations. Create a memory context in
+ <function>BeginDMLPushdown</> if you need longer-lived storage, or use
+ the <structfield>es_query_cxt</> of the node's <structname>EState</>.
+ </para>
+
+ <para>
+ The rows returned must match the <structfield>fdw_scan_tlist</> target
+ list if one was supplied, otherwise they must match the row type of the
+ foreign table being updated. If you choose to optimize away fetching
+ columns that are not needed for the <literal>RETURNING</> calculation,
+ you should insert nulls in those column positions, or else generate a
+ <structfield>fdw_scan_tlist</> list with those columns omitted.
+ </para>
+
+ <para>
+ Whether the query has the clause or not, the query's reported row count
+ must be incremented by the FDW itself. When the query doesn't have the
+ clause, the FDW must also increment the row count for the
+ <structname>ForeignScanState</> node in the <command>EXPLAIN ANALYZE</>
+ case.
+ </para>
+
+ <para>
+ If the <function>IterateDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to execute the table update directly on
+ the remote server will fail with an error message.
+ </para>
+
+ <para>
+ <programlisting>
+ void
+ EndDMLPushdown (ForeignScanState *node);
+ </programlisting>
+
+ End the table update and release resources. It is normally not important
+ to release palloc'd memory, but for example open files and connections
+ to remote servers should be cleaned up.
+ </para>
+
+ <para>
+ If the <function>EndDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to execute the table update directly on
+ the remote server will fail with an error message.
+ </para>
+
</sect2>
<sect2 id="fdw-callbacks-row-locking">
***************
*** 865,870 **** ExplainForeignModify (ModifyTableState *mtstate,
--- 1010,1038 ----
<command>EXPLAIN</>.
</para>
+ <para>
+ <programlisting>
+ void
+ ExplainDMLPushdown (ForeignScanState *node,
+ ExplainState *es);
+ </programlisting>
+
+ Print additional <command>EXPLAIN</> output for a foreign table update
+ that is executed directly on the remote server.
+ This function can call <function>ExplainPropertyText</> and
+ related functions to add fields to the <command>EXPLAIN</> output.
+ The flag fields in <literal>es</> can be used to determine what to
+ print, and the state of the <structname>ForeignScanState</> node
+ can be inspected to provide run-time statistics in the <command>EXPLAIN
+ ANALYZE</> case.
+ </para>
+
+ <para>
+ If the <function>ExplainDMLPushdown</> pointer is set to
+ <literal>NULL</>, no additional information is printed during
+ <command>EXPLAIN</>.
+ </para>
+
</sect2>
<sect2 id="fdw-callbacks-analyze">
***************
*** 1146,1152 **** GetForeignServerByName(const char *name, bool missing_ok);
<para>
The FDW callback functions <function>GetForeignRelSize</>,
<function>GetForeignPaths</>, <function>GetForeignPlan</>,
! <function>PlanForeignModify</>, and <function>GetForeignJoinPaths</>
must fit into the workings of the <productname>PostgreSQL</> planner.
Here are some notes about what they must do.
</para>
--- 1314,1321 ----
<para>
The FDW callback functions <function>GetForeignRelSize</>,
<function>GetForeignPaths</>, <function>GetForeignPlan</>,
! <function>PlanForeignModify</>, <function>GetForeignJoinPaths</>, and
! <function>PlanDMLPushdown</>
must fit into the workings of the <productname>PostgreSQL</> planner.
Here are some notes about what they must do.
</para>
***************
*** 1306,1312 **** GetForeignServerByName(const char *name, bool missing_ok);
<para>
When planning an <command>UPDATE</> or <command>DELETE</>,
! <function>PlanForeignModify</> can look up the <structname>RelOptInfo</>
struct for the foreign table and make use of the
<literal>baserel->fdw_private</> data previously created by the
scan-planning functions. However, in <command>INSERT</> the target
--- 1475,1482 ----
<para>
When planning an <command>UPDATE</> or <command>DELETE</>,
! <function>PlanForeignModify</> and <function>PlanDMLPushdown</>
! can look up the <structname>RelOptInfo</>
struct for the foreign table and make use of the
<literal>baserel->fdw_private</> data previously created by the
scan-planning functions. However, in <command>INSERT</> the target
*** a/doc/src/sgml/postgres-fdw.sgml
--- b/doc/src/sgml/postgres-fdw.sgml
***************
*** 484,489 ****
--- 484,498 ----
extension that's listed in the foreign server's <literal>extensions</>
option. Operators and functions in such clauses must
be <literal>IMMUTABLE</> as well.
+ For an <command>UPDATE</> or <command>DELETE</> query,
+ <filename>postgres_fdw</> attempts to optimize the query execution by
+ sending the whole query to the remote server if there are no query
+ <literal>WHERE</> clauses that cannot be sent to the remote server,
+ no local joins for the query, and no row-level local <literal>BEFORE</> or
+ <literal>AFTER</> triggers on the target table. In <command>UPDATE</>,
+ expressions to assign to target columns must use only built-in data types,
+ <literal>IMMUTABLE</> operators, or <literal>IMMUTABLE</> functions,
+ to reduce the risk of misexecution of the query.
</para>
<para>
*** a/src/backend/commands/explain.c
--- b/src/backend/commands/explain.c
***************
*** 900,906 **** ExplainNode(PlanState *planstate, List *ancestors,
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
! pname = sname = "Foreign Scan";
break;
case T_CustomScan:
sname = "Custom Scan";
--- 900,928 ----
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
! sname = "Foreign Scan";
! switch (((ForeignScan *) plan)->operation)
! {
! case CMD_SELECT:
! pname = "Foreign Scan";
! operation = "Select";
! break;
! case CMD_INSERT:
! pname = "Foreign Insert";
! operation = "Insert";
! break;
! case CMD_UPDATE:
! pname = "Foreign Update";
! operation = "Update";
! break;
! case CMD_DELETE:
! pname = "Foreign Delete";
! operation = "Delete";
! break;
! default:
! pname = "???";
! break;
! }
break;
case T_CustomScan:
sname = "Custom Scan";
***************
*** 1648,1653 **** show_plan_tlist(PlanState *planstate, List *ancestors, ExplainState *es)
--- 1670,1690 ----
return;
if (IsA(plan, RecursiveUnion))
return;
+ /*
+ * Likewise for ForeignScan that has pushed down INSERT/UPDATE/DELETE
+ *
+ * Note: the tlist for an ForeignScan that has pushed down an INSERT/
+ * UPDATE might contain subplan output expressions that are confusing
+ * in this context. The tlist for an ForeignScan that has pushed down
+ * an UPDATE/DELETE always contains "junk" target columns to identify
+ * the exact row to update or delete, which would be confusing in this
+ * context. So, we suppress it in all the cases.
+ */
+ if (IsA(plan, ForeignScan) &&
+ (((ForeignScan *) plan)->operation == CMD_INSERT ||
+ ((ForeignScan *) plan)->operation == CMD_UPDATE ||
+ ((ForeignScan *) plan)->operation == CMD_DELETE))
+ return;
/* Set up deparsing context */
context = set_deparse_context_planstate(es->deparse_cxt,
***************
*** 2236,2243 **** show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es)
FdwRoutine *fdwroutine = fsstate->fdwroutine;
/* Let the FDW emit whatever fields it wants */
! if (fdwroutine->ExplainForeignScan != NULL)
! fdwroutine->ExplainForeignScan(fsstate, es);
}
/*
--- 2273,2288 ----
FdwRoutine *fdwroutine = fsstate->fdwroutine;
/* Let the FDW emit whatever fields it wants */
! if (((ForeignScan *) fsstate->ss.ps.plan)->operation != CMD_SELECT)
! {
! if (fdwroutine->ExplainDMLPushdown != NULL)
! fdwroutine->ExplainDMLPushdown(fsstate, es);
! }
! else
! {
! if (fdwroutine->ExplainForeignScan != NULL)
! fdwroutine->ExplainForeignScan(fsstate, es);
! }
}
/*
***************
*** 2623,2630 **** show_modifytable_info(ModifyTableState *mtstate, List *ancestors,
}
}
! /* Give FDW a chance */
! if (fdwroutine && fdwroutine->ExplainForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, j);
--- 2668,2677 ----
}
}
! /* Give FDW a chance if needed */
! if (!resultRelInfo->ri_FdwPushdown &&
! fdwroutine != NULL &&
! fdwroutine->ExplainForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, j);
*** a/src/backend/executor/execMain.c
--- b/src/backend/executor/execMain.c
***************
*** 1245,1250 **** InitResultRelInfo(ResultRelInfo *resultRelInfo,
--- 1245,1251 ----
else
resultRelInfo->ri_FdwRoutine = NULL;
resultRelInfo->ri_FdwState = NULL;
+ resultRelInfo->ri_FdwPushdown = false;
resultRelInfo->ri_ConstraintExprs = NULL;
resultRelInfo->ri_junkFilter = NULL;
resultRelInfo->ri_projectReturning = NULL;
*** a/src/backend/executor/nodeForeignscan.c
--- b/src/backend/executor/nodeForeignscan.c
***************
*** 48,54 **** ForeignNext(ForeignScanState *node)
/* Call the Iterate function in short-lived context */
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! slot = node->fdwroutine->IterateForeignScan(node);
MemoryContextSwitchTo(oldcontext);
/*
--- 48,57 ----
/* Call the Iterate function in short-lived context */
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! if (plan->operation != CMD_SELECT)
! slot = node->fdwroutine->IterateDMLPushdown(node);
! else
! slot = node->fdwroutine->IterateForeignScan(node);
MemoryContextSwitchTo(oldcontext);
/*
***************
*** 226,232 **** ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
/*
* Tell the FDW to initialize the scan.
*/
! fdwroutine->BeginForeignScan(scanstate, eflags);
return scanstate;
}
--- 229,238 ----
/*
* Tell the FDW to initialize the scan.
*/
! if (node->operation != CMD_SELECT)
! fdwroutine->BeginDMLPushdown(scanstate, eflags);
! else
! fdwroutine->BeginForeignScan(scanstate, eflags);
return scanstate;
}
***************
*** 240,247 **** ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
void
ExecEndForeignScan(ForeignScanState *node)
{
/* Let the FDW shut down */
! node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
--- 246,258 ----
void
ExecEndForeignScan(ForeignScanState *node)
{
+ ForeignScan *plan = (ForeignScan *) node->ss.ps.plan;
+
/* Let the FDW shut down */
! if (plan->operation != CMD_SELECT)
! node->fdwroutine->EndDMLPushdown(node);
! else
! node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
*** a/src/backend/executor/nodeModifyTable.c
--- b/src/backend/executor/nodeModifyTable.c
***************
*** 138,150 **** ExecCheckPlanOutput(Relation resultRel, List *targetList)
* tupleSlot: slot holding tuple actually inserted/updated/deleted
* planSlot: slot holding tuple returned by top subplan node
*
* Returns a slot holding the result tuple
*/
static TupleTableSlot *
! ExecProcessReturning(ProjectionInfo *projectReturning,
TupleTableSlot *tupleSlot,
TupleTableSlot *planSlot)
{
ExprContext *econtext = projectReturning->pi_exprContext;
/*
--- 138,154 ----
* tupleSlot: slot holding tuple actually inserted/updated/deleted
* planSlot: slot holding tuple returned by top subplan node
*
+ * Note: If tupleSlot is NULL, the FDW should have already provided econtext's
+ * scan tuple.
+ *
* Returns a slot holding the result tuple
*/
static TupleTableSlot *
! ExecProcessReturning(ResultRelInfo *resultRelInfo,
TupleTableSlot *tupleSlot,
TupleTableSlot *planSlot)
{
+ ProjectionInfo *projectReturning = resultRelInfo->ri_projectReturning;
ExprContext *econtext = projectReturning->pi_exprContext;
/*
***************
*** 154,160 **** ExecProcessReturning(ProjectionInfo *projectReturning,
ResetExprContext(econtext);
/* Make tuple and any needed join variables available to ExecProject */
! econtext->ecxt_scantuple = tupleSlot;
econtext->ecxt_outertuple = planSlot;
/* Compute the RETURNING expressions */
--- 158,177 ----
ResetExprContext(econtext);
/* Make tuple and any needed join variables available to ExecProject */
! if (tupleSlot)
! econtext->ecxt_scantuple = tupleSlot;
! else
! {
! HeapTuple tuple;
!
! /*
! * RETURNING expressions might reference the tableoid column, so
! * initialize t_tableOid before evaluating them.
! */
! Assert(!TupIsNull(econtext->ecxt_scantuple));
! tuple = ExecMaterializeSlot(econtext->ecxt_scantuple);
! tuple->t_tableOid = RelationGetRelid(resultRelInfo->ri_RelationDesc);
! }
econtext->ecxt_outertuple = planSlot;
/* Compute the RETURNING expressions */
***************
*** 496,503 **** ExecInsert(ModifyTableState *mtstate,
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo->ri_projectReturning,
! slot, planSlot);
return NULL;
}
--- 513,519 ----
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo, slot, planSlot);
return NULL;
}
***************
*** 738,745 **** ldelete:;
ExecStoreTuple(&deltuple, slot, InvalidBuffer, false);
}
! rslot = ExecProcessReturning(resultRelInfo->ri_projectReturning,
! slot, planSlot);
/*
* Before releasing the target tuple again, make sure rslot has a
--- 754,760 ----
ExecStoreTuple(&deltuple, slot, InvalidBuffer, false);
}
! rslot = ExecProcessReturning(resultRelInfo, slot, planSlot);
/*
* Before releasing the target tuple again, make sure rslot has a
***************
*** 1024,1031 **** lreplace:;
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo->ri_projectReturning,
! slot, planSlot);
return NULL;
}
--- 1039,1045 ----
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo, slot, planSlot);
return NULL;
}
***************
*** 1380,1385 **** ExecModifyTable(ModifyTableState *node)
--- 1394,1414 ----
break;
}
+ /*
+ * If ri_FdwPushdown is true, all we need to do here is compute the
+ * RETURNING expressions.
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ Assert(resultRelInfo->ri_projectReturning);
+
+ /* No need to provide scan tuple to ExecProcessReturning. */
+ slot = ExecProcessReturning(resultRelInfo, NULL, planSlot);
+
+ estate->es_result_relation_info = saved_resultRelInfo;
+ return slot;
+ }
+
EvalPlanQualSetSlot(&node->mt_epqstate, planSlot);
slot = planSlot;
***************
*** 1559,1564 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
--- 1588,1596 ----
{
subplan = (Plan *) lfirst(l);
+ /* Initialize the FdwPushdown flag */
+ resultRelInfo->ri_FdwPushdown = bms_is_member(i, node->fdwPushdowns);
+
/*
* Verify result relation is a valid target for the current operation
*/
***************
*** 1583,1589 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
/* Also let FDWs init themselves for foreign-table result rels */
! if (resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, i);
--- 1615,1622 ----
mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
/* Also let FDWs init themselves for foreign-table result rels */
! if (!resultRelInfo->ri_FdwPushdown &&
! resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, i);
***************
*** 1754,1766 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
--- 1787,1812 ----
erm = ExecFindRowMark(estate, rc->rti, false);
/* build ExecAuxRowMark for each subplan */
+ resultRelInfo = mtstate->resultRelInfo;
for (i = 0; i < nplans; i++)
{
ExecAuxRowMark *aerm;
+ /*
+ * ignore subplan if the FDW pushes down the command to the remote
+ * server; the ModifyTable won't have anything to do except for
+ * evaluation of RETURNING expressions
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ resultRelInfo++;
+ continue;
+ }
+
subplan = mtstate->mt_plans[i]->plan;
aerm = ExecBuildAuxRowMark(erm, subplan->targetlist);
mtstate->mt_arowmarks[i] = lappend(mtstate->mt_arowmarks[i], aerm);
+ resultRelInfo++;
}
}
***************
*** 1821,1826 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
--- 1867,1883 ----
ExecCheckPlanOutput(resultRelInfo->ri_RelationDesc,
subplan->targetlist);
+ /*
+ * ignore subplan if the FDW pushes down the command to the
+ * remote server; the ModifyTable won't have anything to do
+ * except for evaluation of RETURNING expressions
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ resultRelInfo++;
+ continue;
+ }
+
j = ExecInitJunkFilter(subplan->targetlist,
resultRelInfo->ri_RelationDesc->rd_att->tdhasoid,
ExecInitExtraTupleSlot(estate));
***************
*** 1910,1916 **** ExecEndModifyTable(ModifyTableState *node)
{
ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
! if (resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->EndForeignModify != NULL)
resultRelInfo->ri_FdwRoutine->EndForeignModify(node->ps.state,
resultRelInfo);
--- 1967,1974 ----
{
ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
! if (!resultRelInfo->ri_FdwPushdown &&
! resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->EndForeignModify != NULL)
resultRelInfo->ri_FdwRoutine->EndForeignModify(node->ps.state,
resultRelInfo);
*** a/src/backend/nodes/copyfuncs.c
--- b/src/backend/nodes/copyfuncs.c
***************
*** 188,193 **** _copyModifyTable(const ModifyTable *from)
--- 188,194 ----
COPY_NODE_FIELD(withCheckOptionLists);
COPY_NODE_FIELD(returningLists);
COPY_NODE_FIELD(fdwPrivLists);
+ COPY_BITMAPSET_FIELD(fdwPushdowns);
COPY_NODE_FIELD(rowMarks);
COPY_SCALAR_FIELD(epqParam);
COPY_SCALAR_FIELD(onConflictAction);
***************
*** 648,653 **** _copyForeignScan(const ForeignScan *from)
--- 649,655 ----
/*
* copy remainder of node
*/
+ COPY_SCALAR_FIELD(operation);
COPY_SCALAR_FIELD(fs_server);
COPY_NODE_FIELD(fdw_exprs);
COPY_NODE_FIELD(fdw_private);
*** a/src/backend/nodes/outfuncs.c
--- b/src/backend/nodes/outfuncs.c
***************
*** 356,361 **** _outModifyTable(StringInfo str, const ModifyTable *node)
--- 356,362 ----
WRITE_NODE_FIELD(withCheckOptionLists);
WRITE_NODE_FIELD(returningLists);
WRITE_NODE_FIELD(fdwPrivLists);
+ WRITE_BITMAPSET_FIELD(fdwPushdowns);
WRITE_NODE_FIELD(rowMarks);
WRITE_INT_FIELD(epqParam);
WRITE_ENUM_FIELD(onConflictAction, OnConflictAction);
***************
*** 608,613 **** _outForeignScan(StringInfo str, const ForeignScan *node)
--- 609,615 ----
_outScanInfo(str, (const Scan *) node);
+ WRITE_ENUM_FIELD(operation, CmdType);
WRITE_OID_FIELD(fs_server);
WRITE_NODE_FIELD(fdw_exprs);
WRITE_NODE_FIELD(fdw_private);
*** a/src/backend/nodes/readfuncs.c
--- b/src/backend/nodes/readfuncs.c
***************
*** 1481,1486 **** _readModifyTable(void)
--- 1481,1487 ----
READ_NODE_FIELD(withCheckOptionLists);
READ_NODE_FIELD(returningLists);
READ_NODE_FIELD(fdwPrivLists);
+ READ_BITMAPSET_FIELD(fdwPushdowns);
READ_NODE_FIELD(rowMarks);
READ_INT_FIELD(epqParam);
READ_ENUM_FIELD(onConflictAction, OnConflictAction);
*** a/src/backend/optimizer/plan/createplan.c
--- b/src/backend/optimizer/plan/createplan.c
***************
*** 3792,3797 **** make_foreignscan(List *qptlist,
--- 3792,3798 ----
plan->lefttree = outer_plan;
plan->righttree = NULL;
node->scan.scanrelid = scanrelid;
+ node->operation = CMD_SELECT;
/* fs_server will be filled in by create_foreignscan_plan */
node->fs_server = InvalidOid;
node->fdw_exprs = fdw_exprs;
***************
*** 5069,5074 **** make_modifytable(PlannerInfo *root,
--- 5070,5076 ----
Plan *plan = &node->plan;
double total_size;
List *fdw_private_list;
+ Bitmapset *fdwpushdowns;
ListCell *subnode;
ListCell *lc;
int i;
***************
*** 5149,5160 **** make_modifytable(PlannerInfo *root,
--- 5151,5164 ----
* construct private plan data, and accumulate it all into a list.
*/
fdw_private_list = NIL;
+ fdwpushdowns = NULL;
i = 0;
foreach(lc, resultRelations)
{
Index rti = lfirst_int(lc);
FdwRoutine *fdwroutine;
List *fdw_private;
+ bool fdwpushdown;
/*
* If possible, we want to get the FdwRoutine from our RelOptInfo for
***************
*** 5181,5187 **** make_modifytable(PlannerInfo *root,
--- 5185,5207 ----
fdwroutine = NULL;
}
+ /*
+ * If the target relation has any row-level triggers, we can't push
+ * down the command to the remote server.
+ */
+ fdwpushdown = false;
if (fdwroutine != NULL &&
+ fdwroutine->PlanDMLPushdown != NULL &&
+ fdwroutine->BeginDMLPushdown != NULL &&
+ fdwroutine->IterateDMLPushdown != NULL &&
+ fdwroutine->EndDMLPushdown != NULL &&
+ !has_row_triggers(root, rti, operation))
+ fdwpushdown = fdwroutine->PlanDMLPushdown(root, node, rti, i);
+ if (fdwpushdown)
+ fdwpushdowns = bms_add_member(fdwpushdowns, i);
+
+ if (!fdwpushdown &&
+ fdwroutine != NULL &&
fdwroutine->PlanForeignModify != NULL)
fdw_private = fdwroutine->PlanForeignModify(root, node, rti, i);
else
***************
*** 5190,5195 **** make_modifytable(PlannerInfo *root,
--- 5210,5216 ----
i++;
}
node->fdwPrivLists = fdw_private_list;
+ node->fdwPushdowns = fdwpushdowns;
return node;
}
*** a/src/backend/optimizer/util/plancat.c
--- b/src/backend/optimizer/util/plancat.c
***************
*** 1520,1522 **** has_unique_index(RelOptInfo *rel, AttrNumber attno)
--- 1520,1569 ----
}
return false;
}
+
+
+ /*
+ * has_row_triggers
+ *
+ * Detect whether the specified relation has any row-level triggers for event.
+ */
+ bool
+ has_row_triggers(PlannerInfo *root, Index rti, CmdType event)
+ {
+ RangeTblEntry *rte = planner_rt_fetch(rti, root);
+ Relation relation;
+ TriggerDesc *trigDesc;
+ bool result = false;
+
+ /* Assume we already have adequate lock */
+ relation = heap_open(rte->relid, NoLock);
+
+ trigDesc = relation->trigdesc;
+ switch (event)
+ {
+ case CMD_INSERT:
+ if (trigDesc &&
+ (trigDesc->trig_insert_after_row ||
+ trigDesc->trig_insert_before_row))
+ result = true;
+ break;
+ case CMD_UPDATE:
+ if (trigDesc &&
+ (trigDesc->trig_update_after_row ||
+ trigDesc->trig_update_before_row))
+ result = true;
+ break;
+ case CMD_DELETE:
+ if (trigDesc &&
+ (trigDesc->trig_delete_after_row ||
+ trigDesc->trig_delete_before_row))
+ result = true;
+ break;
+ default:
+ elog(ERROR, "unrecognized CmdType: %d", (int) event);
+ break;
+ }
+
+ heap_close(relation, NoLock);
+ return result;
+ }
*** a/src/include/foreign/fdwapi.h
--- b/src/include/foreign/fdwapi.h
***************
*** 94,99 **** typedef void (*EndForeignModify_function) (EState *estate,
--- 94,111 ----
typedef int (*IsForeignRelUpdatable_function) (Relation rel);
+ typedef bool (*PlanDMLPushdown_function) (PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+
+ typedef void (*BeginDMLPushdown_function) (ForeignScanState *node,
+ int eflags);
+
+ typedef TupleTableSlot *(*IterateDMLPushdown_function) (ForeignScanState *node);
+
+ typedef void (*EndDMLPushdown_function) (ForeignScanState *node);
+
typedef RowMarkType (*GetForeignRowMarkType_function) (RangeTblEntry *rte,
LockClauseStrength strength);
***************
*** 111,116 **** typedef void (*ExplainForeignModify_function) (ModifyTableState *mtstate,
--- 123,131 ----
int subplan_index,
struct ExplainState *es);
+ typedef void (*ExplainDMLPushdown_function) (ForeignScanState *node,
+ struct ExplainState *es);
+
typedef int (*AcquireSampleRowsFunc) (Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
***************
*** 171,176 **** typedef struct FdwRoutine
--- 186,195 ----
ExecForeignDelete_function ExecForeignDelete;
EndForeignModify_function EndForeignModify;
IsForeignRelUpdatable_function IsForeignRelUpdatable;
+ PlanDMLPushdown_function PlanDMLPushdown;
+ BeginDMLPushdown_function BeginDMLPushdown;
+ IterateDMLPushdown_function IterateDMLPushdown;
+ EndDMLPushdown_function EndDMLPushdown;
/* Functions for SELECT FOR UPDATE/SHARE row locking */
GetForeignRowMarkType_function GetForeignRowMarkType;
***************
*** 180,185 **** typedef struct FdwRoutine
--- 199,205 ----
/* Support functions for EXPLAIN */
ExplainForeignScan_function ExplainForeignScan;
ExplainForeignModify_function ExplainForeignModify;
+ ExplainDMLPushdown_function ExplainDMLPushdown;
/* Support functions for ANALYZE */
AnalyzeForeignTable_function AnalyzeForeignTable;
*** a/src/include/nodes/execnodes.h
--- b/src/include/nodes/execnodes.h
***************
*** 311,316 **** typedef struct JunkFilter
--- 311,317 ----
* TrigInstrument optional runtime measurements for triggers
* FdwRoutine FDW callback functions, if foreign table
* FdwState available to save private state of FDW
+ * FdwPushdown true when the command is pushed down
* WithCheckOptions list of WithCheckOption's to be checked
* WithCheckOptionExprs list of WithCheckOption expr states
* ConstraintExprs array of constraint-checking expr states
***************
*** 334,339 **** typedef struct ResultRelInfo
--- 335,341 ----
Instrumentation *ri_TrigInstrument;
struct FdwRoutine *ri_FdwRoutine;
void *ri_FdwState;
+ bool ri_FdwPushdown;
List *ri_WithCheckOptions;
List *ri_WithCheckOptionExprs;
List **ri_ConstraintExprs;
*** a/src/include/nodes/pg_list.h
--- b/src/include/nodes/pg_list.h
***************
*** 134,149 **** list_length(const List *l)
--- 134,152 ----
#define list_make2(x1,x2) lcons(x1, list_make1(x2))
#define list_make3(x1,x2,x3) lcons(x1, list_make2(x2, x3))
#define list_make4(x1,x2,x3,x4) lcons(x1, list_make3(x2, x3, x4))
+ #define list_make5(x1,x2,x3,x4,x5) lcons(x1, list_make4(x2, x3, x4, x5))
#define list_make1_int(x1) lcons_int(x1, NIL)
#define list_make2_int(x1,x2) lcons_int(x1, list_make1_int(x2))
#define list_make3_int(x1,x2,x3) lcons_int(x1, list_make2_int(x2, x3))
#define list_make4_int(x1,x2,x3,x4) lcons_int(x1, list_make3_int(x2, x3, x4))
+ #define list_make5_int(x1,x2,x3,x4,x5) lcons_int(x1, list_make4_int(x2, x3, x4, x5))
#define list_make1_oid(x1) lcons_oid(x1, NIL)
#define list_make2_oid(x1,x2) lcons_oid(x1, list_make1_oid(x2))
#define list_make3_oid(x1,x2,x3) lcons_oid(x1, list_make2_oid(x2, x3))
#define list_make4_oid(x1,x2,x3,x4) lcons_oid(x1, list_make3_oid(x2, x3, x4))
+ #define list_make5_oid(x1,x2,x3,x4,x5) lcons_oid(x1, list_make4_oid(x2, x3, x4, x5))
/*
* foreach -
*** a/src/include/nodes/plannodes.h
--- b/src/include/nodes/plannodes.h
***************
*** 189,194 **** typedef struct ModifyTable
--- 189,195 ----
List *withCheckOptionLists; /* per-target-table WCO lists */
List *returningLists; /* per-target-table RETURNING tlists */
List *fdwPrivLists; /* per-target-table FDW private data lists */
+ Bitmapset *fdwPushdowns; /* indices of plans that push down command */
List *rowMarks; /* PlanRowMarks (non-locking only) */
int epqParam; /* ID of Param for EvalPlanQual re-eval */
OnConflictAction onConflictAction; /* ON CONFLICT action */
***************
*** 531,536 **** typedef struct WorkTableScan
--- 532,538 ----
typedef struct ForeignScan
{
Scan scan;
+ CmdType operation; /* SELECT/INSERT/UPDATE/DELETE */
Oid fs_server; /* OID of foreign server */
List *fdw_exprs; /* expressions that FDW may evaluate */
List *fdw_private; /* private data for FDW */
*** a/src/include/optimizer/plancat.h
--- b/src/include/optimizer/plancat.h
***************
*** 55,58 **** extern Selectivity join_selectivity(PlannerInfo *root,
--- 55,60 ----
JoinType jointype,
SpecialJoinInfo *sjinfo);
+ extern bool has_row_triggers(PlannerInfo *root, Index rti, CmdType event);
+
#endif /* PLANCAT_H */
I did another round of review for the latest patch and well as performed
the sanity test, and
haven't found any functional issues. Found couple of issue, see in-line
comments
for the same.
On Thu, Feb 18, 2016 at 3:15 PM, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>
wrote:
On 2016/02/12 21:19, Etsuro Fujita wrote:
Comments on specific points follow.
This seems to need minor rebasing in the wake of the join pushdown patch.
Will do.
Done.
+ /* Likewise for ForeignScan that has pushed down
INSERT/UPDATE/DELETE */ + if (IsA(plan, ForeignScan) && + (((ForeignScan *) plan)->operation == CMD_INSERT || + ((ForeignScan *) plan)->operation == CMD_UPDATE || + ((ForeignScan *) plan)->operation == CMD_DELETE)) + return;I don't really understand why this is a good idea. Since target lists
are only displayed with EXPLAIN (VERBOSE), I don't really understand
what is to be gained by suppressing them in any case at all, and
therefore I don't understand why it's a good idea to do it here,
either. If there is a good reason, maybe the comment should explain
what it is.I think that displaying target lists would be confusing for users.
There seems no objection from you (or anyone), I left that as proposed,
and added more comments.+ /* Check point 1 */
+ if (operation == CMD_INSERT) + return false; + + /* Check point 2 */ + if (nodeTag(subplan) != T_ForeignScan) + return false; + + /* Check point 3 */ + if (subplan->qual != NIL) + return false; + + /* Check point 4 */ + if (operation == CMD_UPDATE)These comments are referring to something in the function header
further up, but you could instead just delete the stuff from the
header and mention the actual conditions here.Will fix.
Done.
The patch doesn't allow the postgres_fdw to push down an UPDATE/DELETE on
a foreign join, so I added one more condition here not to handle such
cases. (I'm planning to propose a patch to handle such cases, in the next
CF.)
I think we should place the checking foreign join condition before the
target columns, as foreign join condition is less costly then the target
columns.
- If the first condition is supposed accept only CMD_UPDATE or
CMD_DELETE, say if (operation != CMD_UPDATE || operation !=
CMD_DELETE) rather than doing it this way. Is-not-insert may in this
context be functionally equivalent to is-update-or-delete, but it's
better to write the comment and the code so that they exactly match
rather than kinda-match.- For point 2, use IsA(subplan, ForiegnScan).
Will fix.
Done.
+ /*
+ * ignore subplan if the FDW pushes down the command to the remote + * server + */This comment states what the code does, instead of explaining why it
does it. Please update it so that it explains why it does it.Will update.
Done.
+ List *fdwPushdowns; /* per-target-table FDW
pushdown flags */
Isn't a list of booleans an awfully inefficient representation? How
about a Bitmapset?OK, will do.
Done.
+ /*
+ * Prepare remote-parameter expressions for evaluation. (Note: in + * practice, we expect that all these expressions will be just Params, so + * we could possibly do something more efficient than using the full + * expression-eval machinery for this. But probably there would be little + * benefit, and it'd require postgres_fdw to know more than is desirable + * about Param evaluation.) + */ + dpstate->param_exprs = (List *) + ExecInitExpr((Expr *) fsplan->fdw_exprs, + (PlanState *) node);This is an exact copy of an existing piece of code and its associated
comment. A good bit of the surrounding code is copied, too. You need
to refactor to avoid duplication, like by putting some of the code in
a new function that both postgresBeginForeignScan and
postgresBeginForeignModify can call.execute_dml_stmt() has some of the same disease.
Will do.
Done.
Other changes:
* I fixed docs as discussed before with Rushabh Lathia and Thom Brown.
* I keep Rushabh's code change that we call PlanDMLPushdown only when all
the required APIs are available with FDW, but for CheckValidResultRel, I
left the code as-is (no changes to that function), to match the docs saying
that the FDW needs to provide the DML pushdown callback functions together
with existing table-updating functions such as ExecForeignInsert,
ExecForeignUpdate and ExecForeignDelete.
I think we should also update the CheckValidResultRel(), because even
though ExecForeignInsert,
ExecForeignUpdate and ExecForeignDelete not present, FDW still can perform
UPDATE/DELETE/INSERT using DML Pushdown APIs. Lets take committer's view on
this.
PFA update patch, which includes changes into postgresPlanDMLPushdown() to
check for join
condition before target columns and also fixed couple of whitespace issues.
Regards
Rushabh Lathia
www.EnterpriseDB.com
Attachments:
fdw-dml-pushdown-v9.patchtext/x-diff; charset=US-ASCII; name=fdw-dml-pushdown-v9.patchDownload
diff --git a/contrib/postgres_fdw/deparse.c b/contrib/postgres_fdw/deparse.c
index ef8eab6..c022ff4 100644
--- a/contrib/postgres_fdw/deparse.c
+++ b/contrib/postgres_fdw/deparse.c
@@ -1316,6 +1316,69 @@ deparseUpdateSql(StringInfo buf, PlannerInfo *root,
}
/*
+ * deparse remote UPDATE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+void
+deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+{
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ deparse_expr_cxt context;
+ int nestlevel;
+ bool first;
+ ListCell *lc;
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = params_list;
+
+ appendStringInfoString(buf, "UPDATE ");
+ deparseRelation(buf, rel);
+ appendStringInfoString(buf, " SET ");
+
+ /* Make sure any constants in the exprs are printed portably */
+ nestlevel = set_transmission_modes();
+
+ first = true;
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(targetlist, attnum);
+
+ if (!first)
+ appendStringInfoString(buf, ", ");
+ first = false;
+
+ deparseColumnRef(buf, rtindex, attnum, root, false);
+ appendStringInfoString(buf, " = ");
+ deparseExpr((Expr *) tle->expr, &context);
+ }
+
+ reset_transmission_modes(nestlevel);
+
+ if (remote_conds)
+ {
+ appendStringInfo(buf, " WHERE ");
+ appendConditions(remote_conds, &context);
+ }
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+}
+
+/*
* deparse remote DELETE statement
*
* The statement text is appended to buf, and we also create an integer List
@@ -1338,6 +1401,43 @@ deparseDeleteSql(StringInfo buf, PlannerInfo *root,
}
/*
+ * deparse remote DELETE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+void
+deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+{
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ deparse_expr_cxt context;
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = params_list;
+
+ appendStringInfoString(buf, "DELETE FROM ");
+ deparseRelation(buf, rel);
+
+ if (remote_conds)
+ {
+ appendStringInfo(buf, " WHERE ");
+ appendConditions(remote_conds, &context);
+ }
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+}
+
+/*
* Add a RETURNING clause, if needed, to an INSERT/UPDATE/DELETE.
*/
static void
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index 280c377..007e1a8 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -2258,7 +2258,26 @@ INSERT INTO ft2 (c1,c2,c3)
(3 rows)
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+EXPLAIN (verbose, costs off)
+UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
+ QUERY PLAN
+----------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 300), c3 = (c3 || '_update3'::text) WHERE ((("C 1" % 10) = 3))
+(3 rows)
+
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+EXPLAIN (verbose, costs off)
+UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
+ QUERY PLAN
+------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Output: c1, c2, c3, c4, c5, c6, c7, c8
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 400), c3 = (c3 || '_update7'::text) WHERE ((("C 1" % 10) = 7)) RETURNING "C 1", c2, c3, c4, c5, c6, c7, c8
+(4 rows)
+
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
c1 | c2 | c3 | c4 | c5 | c6 | c7 | c8
------+-----+--------------------+------------------------------+--------------------------+----+------------+-----
@@ -2368,7 +2387,7 @@ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
- FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
+ FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
@@ -2393,16 +2412,14 @@ UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
- DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
- QUERY PLAN
-----------------------------------------------------------------------------------------
+ DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
+ QUERY PLAN
+--------------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
- Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1 RETURNING "C 1", c4
- -> Foreign Scan on public.ft2
- Output: ctid
- Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) FOR UPDATE
-(6 rows)
+ -> Foreign Delete on public.ft2
+ Remote SQL: DELETE FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) RETURNING "C 1", c4
+(4 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
@@ -2513,7 +2530,7 @@ DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
(103 rows)
EXPLAIN (verbose, costs off)
-DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
+DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
@@ -3378,16 +3395,14 @@ INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
(1 row)
EXPLAIN (verbose, costs off)
-UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
- QUERY PLAN
--------------------------------------------------------------------------------------------------------------------
+UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
+ QUERY PLAN
+------------------------------------------------------------------------------------
Update on public.ft2
Output: (tableoid)::regclass
- Remote SQL: UPDATE "S 1"."T 1" SET c3 = $2 WHERE ctid = $1
- -> Foreign Scan on public.ft2
- Output: c1, c2, NULL::integer, 'bar'::text, c4, c5, c6, c7, c8, ctid
- Remote SQL: SELECT "C 1", c2, c4, c5, c6, c7, c8, ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
-(6 rows)
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c3 = 'bar'::text WHERE (("C 1" = 9999))
+(4 rows)
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
@@ -3396,16 +3411,14 @@ UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
(1 row)
EXPLAIN (verbose, costs off)
-DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
- QUERY PLAN
-------------------------------------------------------------------------------------
+DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
+ QUERY PLAN
+--------------------------------------------------------------------
Delete on public.ft2
Output: (tableoid)::regclass
- Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1
- -> Foreign Scan on public.ft2
- Output: ctid
- Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
-(6 rows)
+ -> Foreign Delete on public.ft2
+ Remote SQL: DELETE FROM "S 1"."T 1" WHERE (("C 1" = 9999))
+(4 rows)
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
@@ -3559,7 +3572,7 @@ CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
-CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
+CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
@@ -3718,7 +3731,7 @@ savepoint s3;
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
-CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
+CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (-2) WHERE ((c2 = 42)) AND (("C 1" = 10))
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
@@ -3858,7 +3871,7 @@ CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
-CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
+CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2positive;
-- But inconsistent check constraints provide inconsistent results
ALTER FOREIGN TABLE ft1 ADD CONSTRAINT ft1_c2negative CHECK (c2 < 0);
@@ -4251,6 +4264,199 @@ NOTICE: NEW: (13,"test triggered !")
(0,27)
(1 row)
+-- cleanup
+DROP TRIGGER trig_row_before ON rem1;
+DROP TRIGGER trig_row_after ON rem1;
+DROP TRIGGER trig_local_before ON loc1;
+-- Test DML pushdown functionality
+-- Test with statement-level triggers
+CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+(3 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+(3 rows)
+
+DROP TRIGGER trig_stmt_before ON rem1;
+CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+(3 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+(3 rows)
+
+DROP TRIGGER trig_stmt_after ON rem1;
+-- Test with row-level ON INSERT triggers
+CREATE TRIGGER trig_row_before_insert
+BEFORE INSERT ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+(3 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+(3 rows)
+
+DROP TRIGGER trig_row_before_insert ON rem1;
+CREATE TRIGGER trig_row_after_insert
+AFTER INSERT ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+(3 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+(3 rows)
+
+DROP TRIGGER trig_row_after_insert ON rem1;
+-- Test with row-level ON UPDATE triggers
+CREATE TRIGGER trig_row_before_update
+BEFORE UPDATE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+---------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+(5 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+(3 rows)
+
+DROP TRIGGER trig_row_before_update ON rem1;
+CREATE TRIGGER trig_row_after_update
+AFTER UPDATE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+-------------------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+(5 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+(3 rows)
+
+DROP TRIGGER trig_row_after_update ON rem1;
+-- Test with row-level ON DELETE triggers
+CREATE TRIGGER trig_row_before_delete
+BEFORE DELETE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+(3 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+---------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+(5 rows)
+
+DROP TRIGGER trig_row_before_delete ON rem1;
+CREATE TRIGGER trig_row_after_delete
+AFTER DELETE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+(3 rows)
+
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+------------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+(5 rows)
+
+DROP TRIGGER trig_row_after_delete ON rem1;
-- ===================================================================
-- test inheritance features
-- ===================================================================
@@ -4720,6 +4926,56 @@ fetch from c;
update bar set f2 = null where current of c;
ERROR: WHERE CURRENT OF is not supported for this table type
rollback;
+explain (verbose, costs off)
+delete from foo where f1 < 5 returning *;
+ QUERY PLAN
+--------------------------------------------------------------------------------
+ Delete on public.foo
+ Output: foo.f1, foo.f2
+ Delete on public.foo
+ Foreign Delete on public.foo2
+ -> Index Scan using i_foo_f1 on public.foo
+ Output: foo.ctid
+ Index Cond: (foo.f1 < 5)
+ -> Foreign Delete on public.foo2
+ Remote SQL: DELETE FROM public.loct1 WHERE ((f1 < 5)) RETURNING f1, f2
+(9 rows)
+
+delete from foo where f1 < 5 returning *;
+ f1 | f2
+----+----
+ 1 | 1
+ 3 | 3
+ 0 | 0
+ 2 | 2
+ 4 | 4
+(5 rows)
+
+explain (verbose, costs off)
+update bar set f2 = f2 + 100 returning *;
+ QUERY PLAN
+------------------------------------------------------------------------------
+ Update on public.bar
+ Output: bar.f1, bar.f2
+ Update on public.bar
+ Foreign Update on public.bar2
+ -> Seq Scan on public.bar
+ Output: bar.f1, (bar.f2 + 100), bar.ctid
+ -> Foreign Update on public.bar2
+ Remote SQL: UPDATE public.loct2 SET f2 = (f2 + 100) RETURNING f1, f2
+(8 rows)
+
+update bar set f2 = f2 + 100 returning *;
+ f1 | f2
+----+-----
+ 1 | 311
+ 2 | 322
+ 6 | 266
+ 3 | 333
+ 4 | 344
+ 7 | 277
+(6 rows)
+
drop table foo cascade;
NOTICE: drop cascades to foreign table foo2
drop table bar cascade;
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 465f43c..f88d796 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -61,6 +61,8 @@ enum FdwScanPrivateIndex
{
/* SQL statement to execute remotely (as a String node) */
FdwScanPrivateSelectSql,
+ /* List of restriction clauses that can be executed remotely */
+ FdwScanPrivateRemoteConds,
/* Integer list of attribute numbers retrieved by the SELECT */
FdwScanPrivateRetrievedAttrs,
/* Integer representing the desired fetch_size */
@@ -98,6 +100,28 @@ enum FdwModifyPrivateIndex
};
/*
+ * Similarly, this enum describes what's kept in the fdw_private list for
+ * a ForeignScan node that has pushed down an UPDATE/DELETE to the remote
+ * server. We store:
+ *
+ * 1) UPDATE/DELETE statement text to be sent to the remote server
+ * 2) Boolean flag showing if the remote query has a RETURNING clause
+ * 3) Integer list of attribute numbers retrieved by RETURNING, if any
+ * 4) Boolean flag showing if we set the command es_processed
+ */
+enum FdwDmlPushdownPrivateIndex
+{
+ /* SQL statement to execute remotely (as a String node) */
+ FdwDmlPushdownPrivateUpdateSql,
+ /* has-returning flag (as an integer Value node) */
+ FdwDmlPushdownPrivateHasReturning,
+ /* Integer list of attribute numbers retrieved by RETURNING */
+ FdwDmlPushdownPrivateRetrievedAttrs,
+ /* set-processed flag (as an integer Value node) */
+ FdwDmlPushdownPrivateSetProcessed
+};
+
+/*
* Execution state of a foreign scan using postgres_fdw.
*/
typedef struct PgFdwScanState
@@ -164,6 +188,37 @@ typedef struct PgFdwModifyState
} PgFdwModifyState;
/*
+ * Execution state of a foreign scan that has pushed down a foreign table
+ * modification to the remote server
+ */
+typedef struct PgFdwDmlPushdownState
+{
+ Relation rel; /* relcache entry for the foreign table */
+ AttInMetadata *attinmeta; /* attribute datatype conversion metadata */
+
+ /* extracted fdw_private data */
+ char *query; /* text of UPDATE/DELETE command */
+ bool has_returning; /* is there a RETURNING clause? */
+ List *retrieved_attrs; /* attr numbers retrieved by RETURNING */
+ bool set_processed; /* do we set the command es_processed? */
+
+ /* for remote query execution */
+ PGconn *conn; /* connection for the update */
+ int numParams; /* number of parameters passed to query */
+ FmgrInfo *param_flinfo; /* output conversion functions for them */
+ List *param_exprs; /* executable expressions for param values */
+ const char **param_values; /* textual values of query parameters */
+
+ /* for storing result tuples */
+ PGresult *result; /* result for query */
+ int num_tuples; /* # of result tuples */
+ int next_tuple; /* index of next one to return */
+
+ /* working memory context */
+ MemoryContext temp_cxt; /* context for per-tuple temporary data */
+} PgFdwDmlPushdownState;
+
+/*
* Workspace for analyzing a foreign table.
*/
typedef struct PgFdwAnalyzeState
@@ -263,6 +318,13 @@ static TupleTableSlot *postgresExecForeignDelete(EState *estate,
static void postgresEndForeignModify(EState *estate,
ResultRelInfo *resultRelInfo);
static int postgresIsForeignRelUpdatable(Relation rel);
+static bool postgresPlanDMLPushdown(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+static void postgresBeginDMLPushdown(ForeignScanState *node, int eflags);
+static TupleTableSlot *postgresIterateDMLPushdown(ForeignScanState *node);
+static void postgresEndDMLPushdown(ForeignScanState *node);
static void postgresExplainForeignScan(ForeignScanState *node,
ExplainState *es);
static void postgresExplainForeignModify(ModifyTableState *mtstate,
@@ -270,6 +332,8 @@ static void postgresExplainForeignModify(ModifyTableState *mtstate,
List *fdw_private,
int subplan_index,
ExplainState *es);
+static void postgresExplainDMLPushdown(ForeignScanState *node,
+ ExplainState *es);
static bool postgresAnalyzeForeignTable(Relation relation,
AcquireSampleRowsFunc *func,
BlockNumber *totalpages);
@@ -314,6 +378,18 @@ static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
TupleTableSlot *slot);
static void store_returning_result(PgFdwModifyState *fmstate,
TupleTableSlot *slot, PGresult *res);
+static void execute_dml_stmt(ForeignScanState *node);
+static TupleTableSlot *get_returning_data(ForeignScanState *node);
+static void prepare_query_params(PlanState *node,
+ List *fdw_exprs,
+ int numParams,
+ FmgrInfo **param_flinfo,
+ List **param_exprs,
+ const char ***param_values);
+static void process_query_params(ExprContext *econtext,
+ FmgrInfo *param_flinfo,
+ List *param_exprs,
+ const char **param_values);
static int postgresAcquireSampleRowsFunc(Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
@@ -360,12 +436,17 @@ postgres_fdw_handler(PG_FUNCTION_ARGS)
routine->ExecForeignDelete = postgresExecForeignDelete;
routine->EndForeignModify = postgresEndForeignModify;
routine->IsForeignRelUpdatable = postgresIsForeignRelUpdatable;
+ routine->PlanDMLPushdown = postgresPlanDMLPushdown;
+ routine->BeginDMLPushdown = postgresBeginDMLPushdown;
+ routine->IterateDMLPushdown = postgresIterateDMLPushdown;
+ routine->EndDMLPushdown = postgresEndDMLPushdown;
/* Function for EvalPlanQual rechecks */
routine->RecheckForeignScan = postgresRecheckForeignScan;
/* Support functions for EXPLAIN */
routine->ExplainForeignScan = postgresExplainForeignScan;
routine->ExplainForeignModify = postgresExplainForeignModify;
+ routine->ExplainDMLPushdown = postgresExplainDMLPushdown;
/* Support functions for ANALYZE */
routine->AnalyzeForeignTable = postgresAnalyzeForeignTable;
@@ -1133,7 +1214,8 @@ postgresGetForeignPlan(PlannerInfo *root,
* Build the fdw_private list that will be available to the executor.
* Items in the list must match order in enum FdwScanPrivateIndex.
*/
- fdw_private = list_make4(makeString(sql.data),
+ fdw_private = list_make5(makeString(sql.data),
+ remote_conds,
retrieved_attrs,
makeInteger(fpinfo->fetch_size),
makeInteger(foreignrel->umid));
@@ -1170,8 +1252,6 @@ postgresBeginForeignScan(ForeignScanState *node, int eflags)
PgFdwScanState *fsstate;
UserMapping *user;
int numParams;
- int i;
- ListCell *lc;
/*
* Do nothing in EXPLAIN (no ANALYZE) case. node->fdw_state stays NULL.
@@ -1258,42 +1338,18 @@ postgresBeginForeignScan(ForeignScanState *node, int eflags)
fsstate->attinmeta = TupleDescGetAttInMetadata(fsstate->tupdesc);
- /* Prepare for output conversion of parameters used in remote query. */
- numParams = list_length(fsplan->fdw_exprs);
- fsstate->numParams = numParams;
- fsstate->param_flinfo = (FmgrInfo *) palloc0(sizeof(FmgrInfo) * numParams);
-
- i = 0;
- foreach(lc, fsplan->fdw_exprs)
- {
- Node *param_expr = (Node *) lfirst(lc);
- Oid typefnoid;
- bool isvarlena;
-
- getTypeOutputInfo(exprType(param_expr), &typefnoid, &isvarlena);
- fmgr_info(typefnoid, &fsstate->param_flinfo[i]);
- i++;
- }
-
- /*
- * Prepare remote-parameter expressions for evaluation. (Note: in
- * practice, we expect that all these expressions will be just Params, so
- * we could possibly do something more efficient than using the full
- * expression-eval machinery for this. But probably there would be little
- * benefit, and it'd require postgres_fdw to know more than is desirable
- * about Param evaluation.)
- */
- fsstate->param_exprs = (List *)
- ExecInitExpr((Expr *) fsplan->fdw_exprs,
- (PlanState *) node);
-
/*
- * Allocate buffer for text form of query parameters, if any.
+ * Prepare for processing of parameters used in remote query, if any.
*/
+ numParams = list_length(fsplan->fdw_exprs);
+ fsstate->numParams = numParams;
if (numParams > 0)
- fsstate->param_values = (const char **) palloc0(numParams * sizeof(char *));
- else
- fsstate->param_values = NULL;
+ prepare_query_params((PlanState *) node,
+ fsplan->fdw_exprs,
+ numParams,
+ &fsstate->param_flinfo,
+ &fsstate->param_exprs,
+ &fsstate->param_values);
}
/*
@@ -1458,13 +1514,6 @@ postgresAddForeignUpdateTargets(Query *parsetree,
/*
* postgresPlanForeignModify
* Plan an insert/update/delete operation on a foreign table
- *
- * Note: currently, the plan tree generated for UPDATE/DELETE will always
- * include a ForeignScan that retrieves ctids (using SELECT FOR UPDATE)
- * and then the ModifyTable node will have to execute individual remote
- * UPDATE/DELETE commands. If there are no local conditions or joins
- * needed, it'd be better to let the scan node do UPDATE/DELETE RETURNING
- * and then do nothing at ModifyTable. Room for future optimization ...
*/
static List *
postgresPlanForeignModify(PlannerInfo *root,
@@ -2003,6 +2052,314 @@ postgresRecheckForeignScan(ForeignScanState *node, TupleTableSlot *slot)
}
/*
+ * postgresPlanDMLPushdown
+ * Consider pushing down a foreign table modification to the remote server
+ *
+ * Decide whether the table modification is safe to push down to the remote end,
+ * and if so, modify subplan so as to do that.
+ */
+static bool
+postgresPlanDMLPushdown(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index)
+{
+ CmdType operation = plan->operation;
+ Plan *subplan = (Plan *) list_nth(plan->plans, subplan_index);
+ RangeTblEntry *rte = planner_rt_fetch(resultRelation, root);
+ Relation rel;
+ StringInfoData sql;
+ ForeignScan *fscan;
+ List *targetAttrs = NIL;
+ List *remote_conds;
+ List *params_list = NIL;
+ List *returningList = NIL;
+ List *retrieved_attrs = NIL;
+
+ /*
+ * Decide whether the table modification is pushdown-safe.
+ */
+
+ /*
+ * 1. The table modification must be an UPDATE or DELETE.
+ */
+ if (operation != CMD_UPDATE && operation != CMD_DELETE)
+ return false;
+
+ /*
+ * 2. It's unsafe to push down the command if there are any local joins
+ * needed.
+ */
+ if (!IsA(subplan, ForeignScan))
+ return false;
+
+ /*
+ * 3. It's unsafe to push down the command if there are any quals that
+ * can't be evaluated remotely.
+ */
+ if (subplan->qual != NIL)
+ return false;
+
+ /*
+ * 4. We can't push down an UPDATE or DELETE on a foreign join for now.
+ */
+ fscan = (ForeignScan *) subplan;
+ if (fscan->scan.scanrelid == 0)
+ return false;
+
+ /*
+ * 5. We can't push down an UPDATE, if any expressions to assign to the
+ * target columns are unsafe to evaluate on the remote end.
+ */
+ if (operation == CMD_UPDATE)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[resultRelation];
+ int col;
+
+ /*
+ * We transmit only columns that were explicitly targets of the UPDATE,
+ * so as to avoid unnecessary data transmission.
+ */
+ col = -1;
+ while ((col = bms_next_member(rte->updatedCols, col)) >= 0)
+ {
+ /* bit numbers are offset by FirstLowInvalidHeapAttributeNumber */
+ AttrNumber attno = col + FirstLowInvalidHeapAttributeNumber;
+ TargetEntry *tle;
+
+ if (attno <= InvalidAttrNumber) /* shouldn't happen */
+ elog(ERROR, "system-column update is not supported");
+
+ tle = get_tle_by_resno(subplan->targetlist, attno);
+
+ if (!is_foreign_expr(root, baserel, (Expr *) tle->expr))
+ return false;
+
+ targetAttrs = lappend_int(targetAttrs, attno);
+ }
+ }
+
+ /*
+ * Ok, modify subplan so as to push down the command to the remote server.
+ */
+ initStringInfo(&sql);
+
+ /*
+ * Core code already has some lock on each rel being planned, so we can
+ * use NoLock here.
+ */
+ rel = heap_open(rte->relid, NoLock);
+
+ /*
+ * Extract the baserestrictinfo clauses that can be evaluated remotely.
+ */
+ remote_conds = (List *) list_nth(fscan->fdw_private,
+ FdwScanPrivateRemoteConds);
+
+ /*
+ * Extract the relevant RETURNING list if any.
+ */
+ if (plan->returningLists)
+ returningList = (List *) list_nth(plan->returningLists, subplan_index);
+
+ /*
+ * Construct the SQL command string.
+ */
+ switch (operation)
+ {
+ case CMD_UPDATE:
+ deparsePushedDownUpdateSql(&sql, root, resultRelation, rel,
+ ((Plan *) fscan)->targetlist,
+ targetAttrs,
+ remote_conds, ¶ms_list,
+ returningList, &retrieved_attrs);
+ break;
+ case CMD_DELETE:
+ deparsePushedDownDeleteSql(&sql, root, resultRelation, rel,
+ remote_conds, ¶ms_list,
+ returningList, &retrieved_attrs);
+ break;
+ default:
+ elog(ERROR, "unexpected operation: %d", (int) operation);
+ break;
+ }
+
+ /*
+ * Update the operation info.
+ */
+ fscan->operation = operation;
+
+ /*
+ * Update the fdw_exprs list that will be available to the executor.
+ */
+ fscan->fdw_exprs = params_list;
+
+ /*
+ * Update the fdw_private list that will be available to the executor.
+ * Items in the list must match enum FdwDmlPushdownPrivateIndex, above.
+ */
+ fscan->fdw_private = list_make4(makeString(sql.data),
+ makeInteger((retrieved_attrs != NIL)),
+ retrieved_attrs,
+ makeInteger(plan->canSetTag));
+
+ heap_close(rel, NoLock);
+ return true;
+}
+
+/*
+ * postgresBeginDMLPushdown
+ * Initiate pushing down a foreign table modification to the remote server
+ */
+static void
+postgresBeginDMLPushdown(ForeignScanState *node, int eflags)
+{
+ ForeignScan *fsplan = (ForeignScan *) node->ss.ps.plan;
+ EState *estate = node->ss.ps.state;
+ PgFdwDmlPushdownState *dpstate;
+ RangeTblEntry *rte;
+ Oid userid;
+ ForeignTable *table;
+ UserMapping *user;
+ int numParams;
+
+ /*
+ * Do nothing in EXPLAIN (no ANALYZE) case. node->fdw_state stays NULL.
+ */
+ if (eflags & EXEC_FLAG_EXPLAIN_ONLY)
+ return;
+
+ /*
+ * We'll save private state in node->fdw_state.
+ */
+ dpstate = (PgFdwDmlPushdownState *) palloc0(sizeof(PgFdwDmlPushdownState));
+ node->fdw_state = (void *) dpstate;
+
+ /*
+ * Identify which user to do the remote access as. This should match what
+ * ExecCheckRTEPerms() does.
+ */
+ rte = rt_fetch(fsplan->scan.scanrelid, estate->es_range_table);
+ userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
+
+ /* Get info about foreign table. */
+ dpstate->rel = node->ss.ss_currentRelation;
+ table = GetForeignTable(RelationGetRelid(dpstate->rel));
+ user = GetUserMapping(userid, table->serverid);
+
+ /*
+ * Get connection to the foreign server. Connection manager will
+ * establish new connection if necessary.
+ */
+ dpstate->conn = GetConnection(user, false);
+
+ /* Initialize state variable */
+ dpstate->num_tuples = -1; /* -1 means not set yet */
+
+ /* Get private info created by planner functions. */
+ dpstate->query = strVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateUpdateSql));
+ dpstate->has_returning = intVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateHasReturning));
+ dpstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateRetrievedAttrs);
+ dpstate->set_processed = intVal(list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateSetProcessed));
+
+ /* Create context for per-tuple temp workspace. */
+ dpstate->temp_cxt = AllocSetContextCreate(estate->es_query_cxt,
+ "postgres_fdw temporary data",
+ ALLOCSET_SMALL_MINSIZE,
+ ALLOCSET_SMALL_INITSIZE,
+ ALLOCSET_SMALL_MAXSIZE);
+
+ /* Prepare for input conversion of RETURNING results. */
+ if (dpstate->has_returning)
+ dpstate->attinmeta = TupleDescGetAttInMetadata(RelationGetDescr(dpstate->rel));
+
+ /*
+ * Prepare for processing of parameters used in remote query, if any.
+ */
+ numParams = list_length(fsplan->fdw_exprs);
+ dpstate->numParams = numParams;
+ if (numParams > 0)
+ prepare_query_params((PlanState *) node,
+ fsplan->fdw_exprs,
+ numParams,
+ &dpstate->param_flinfo,
+ &dpstate->param_exprs,
+ &dpstate->param_values);
+}
+
+/*
+ * postgresIterateDMLPushdown
+ * Execute pushing down a foreign table modification to the remote server
+ */
+static TupleTableSlot *
+postgresIterateDMLPushdown(ForeignScanState *node)
+{
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ EState *estate = node->ss.ps.state;
+ ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
+
+ /*
+ * If this is the first call after Begin, execute the statement.
+ */
+ if (dpstate->num_tuples == -1)
+ execute_dml_stmt(node);
+
+ /*
+ * If the local query doesn't specify RETURNING, just clear tuple slot.
+ */
+ if (!resultRelInfo->ri_projectReturning)
+ {
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+ Instrumentation *instr = node->ss.ps.instrument;
+
+ Assert(!dpstate->has_returning);
+
+ /* Increment the command es_processed count if necessary. */
+ if (dpstate->set_processed)
+ estate->es_processed += dpstate->num_tuples;
+
+ /* Increment the tuple count for EXPLAIN ANALYZE if necessary. */
+ if (instr)
+ instr->tuplecount += dpstate->num_tuples;
+
+ return ExecClearTuple(slot);
+ }
+
+ /*
+ * Get the next RETURNING tuple.
+ */
+ return get_returning_data(node);
+}
+
+/*
+ * postgresEndDMLPushdown
+ * Finish pushing down a foreign table modification to the remote server
+ */
+static void
+postgresEndDMLPushdown(ForeignScanState *node)
+{
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+
+ /* if dpstate is NULL, we are in EXPLAIN; nothing to do */
+ if (dpstate == NULL)
+ return;
+
+ /* Release PGresult */
+ if (dpstate->result)
+ PQclear(dpstate->result);
+
+ /* Release remote connection */
+ ReleaseConnection(dpstate->conn);
+ dpstate->conn = NULL;
+
+ /* MemoryContext will be deleted automatically. */
+}
+
+/*
* postgresExplainForeignScan
* Produce extra output for EXPLAIN of a ForeignScan on a foreign table
*/
@@ -2055,6 +2412,25 @@ postgresExplainForeignModify(ModifyTableState *mtstate,
}
}
+/*
+ * postgresExplainDMLPushdown
+ * Produce extra output for EXPLAIN of a ForeignScan on a foreign table
+ * that has pushed down an UPDATE/DELETE to the remote server
+ */
+static void
+postgresExplainDMLPushdown(ForeignScanState *node, ExplainState *es)
+{
+ List *fdw_private;
+ char *sql;
+
+ if (es->verbose)
+ {
+ fdw_private = ((ForeignScan *) node->ss.ps.plan)->fdw_private;
+ sql = strVal(list_nth(fdw_private, FdwDmlPushdownPrivateUpdateSql));
+ ExplainPropertyText("Remote SQL", sql, es);
+ }
+}
+
/*
* estimate_path_cost_size
@@ -2413,38 +2789,14 @@ create_cursor(ForeignScanState *node)
*/
if (numParams > 0)
{
- int nestlevel;
MemoryContext oldcontext;
- int i;
- ListCell *lc;
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
- nestlevel = set_transmission_modes();
-
- i = 0;
- foreach(lc, fsstate->param_exprs)
- {
- ExprState *expr_state = (ExprState *) lfirst(lc);
- Datum expr_value;
- bool isNull;
-
- /* Evaluate the parameter expression */
- expr_value = ExecEvalExpr(expr_state, econtext, &isNull, NULL);
-
- /*
- * Get string representation of each parameter value by invoking
- * type-specific output function, unless the value is null.
- */
- if (isNull)
- values[i] = NULL;
- else
- values[i] = OutputFunctionCall(&fsstate->param_flinfo[i],
- expr_value);
- i++;
- }
-
- reset_transmission_modes(nestlevel);
+ process_query_params(econtext,
+ fsstate->param_flinfo,
+ fsstate->param_exprs,
+ values);
MemoryContextSwitchTo(oldcontext);
}
@@ -2765,6 +3117,197 @@ store_returning_result(PgFdwModifyState *fmstate,
}
/*
+ * Execute a pushed-down UPDATE/DELETE statement.
+ */
+static void
+execute_dml_stmt(ForeignScanState *node)
+{
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ ExprContext *econtext = node->ss.ps.ps_ExprContext;
+ int numParams = dpstate->numParams;
+ const char **values = dpstate->param_values;
+
+ /*
+ * Construct array of query parameter values in text format.
+ */
+ if (numParams > 0)
+ process_query_params(econtext,
+ dpstate->param_flinfo,
+ dpstate->param_exprs,
+ values);
+
+ /*
+ * Notice that we pass NULL for paramTypes, thus forcing the remote server
+ * to infer types for all parameters. Since we explicitly cast every
+ * parameter (see deparse.c), the "inference" is trivial and will produce
+ * the desired result. This allows us to avoid assuming that the remote
+ * server has the same OIDs we do for the parameters' types.
+ *
+ * We don't use a PG_TRY block here, so be careful not to throw error
+ * without releasing the PGresult.
+ */
+ dpstate->result = PQexecParams(dpstate->conn, dpstate->query,
+ numParams, NULL, values, NULL, NULL, 0);
+ if (PQresultStatus(dpstate->result) !=
+ (dpstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
+ pgfdw_report_error(ERROR, dpstate->result, dpstate->conn, true,
+ dpstate->query);
+
+ /* Get the number of rows affected. */
+ if (dpstate->has_returning)
+ dpstate->num_tuples = PQntuples(dpstate->result);
+ else
+ dpstate->num_tuples = atoi(PQcmdTuples(dpstate->result));
+}
+
+/*
+ * Get the result of a RETURNING clause.
+ */
+static TupleTableSlot *
+get_returning_data(ForeignScanState *node)
+{
+ PgFdwDmlPushdownState *dpstate = (PgFdwDmlPushdownState *) node->fdw_state;
+ EState *estate = node->ss.ps.state;
+ ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+
+ Assert(resultRelInfo->ri_projectReturning);
+
+ /* If we didn't get any tuples, must be end of data. */
+ if (dpstate->next_tuple >= dpstate->num_tuples)
+ return ExecClearTuple(slot);
+
+ /* Increment the command es_processed count if necessary. */
+ if (dpstate->set_processed)
+ estate->es_processed += 1;
+
+ /*
+ * Store a RETURNING tuple. If has_returning is false, just emit a dummy
+ * tuple. (We have has_returning=false if the local query is of the form
+ * UPDATE/DELETE .. RETURNING 1 for example.)
+ */
+ if (!dpstate->has_returning)
+ ExecStoreAllNullTuple(slot);
+ else
+ {
+ /*
+ * On error, be sure to release the PGresult on the way out. Callers
+ * do not have PG_TRY blocks to ensure this happens.
+ */
+ PG_TRY();
+ {
+ HeapTuple newtup;
+
+ newtup = make_tuple_from_result_row(dpstate->result,
+ dpstate->next_tuple,
+ dpstate->rel,
+ dpstate->attinmeta,
+ dpstate->retrieved_attrs,
+ NULL,
+ dpstate->temp_cxt);
+ ExecStoreTuple(newtup, slot, InvalidBuffer, false);
+ }
+ PG_CATCH();
+ {
+ if (dpstate->result)
+ PQclear(dpstate->result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ }
+ dpstate->next_tuple++;
+
+ /* Make slot available for evaluation of the local query RETURNING list. */
+ resultRelInfo->ri_projectReturning->pi_exprContext->ecxt_scantuple = slot;
+
+ return slot;
+}
+
+/*
+ * Prepare for processing of parameters used in remote query.
+ */
+static void
+prepare_query_params(PlanState *node,
+ List *fdw_exprs,
+ int numParams,
+ FmgrInfo **param_flinfo,
+ List **param_exprs,
+ const char ***param_values)
+{
+ int i;
+ ListCell *lc;
+
+ Assert(numParams > 0);
+
+ /* Prepare for output conversion of parameters used in remote query. */
+ *param_flinfo = (FmgrInfo *) palloc0(sizeof(FmgrInfo) * numParams);
+
+ i = 0;
+ foreach(lc, fdw_exprs)
+ {
+ Node *param_expr = (Node *) lfirst(lc);
+ Oid typefnoid;
+ bool isvarlena;
+
+ getTypeOutputInfo(exprType(param_expr), &typefnoid, &isvarlena);
+ fmgr_info(typefnoid, &(*param_flinfo)[i]);
+ i++;
+ }
+
+ /*
+ * Prepare remote-parameter expressions for evaluation. (Note: in
+ * practice, we expect that all these expressions will be just Params, so
+ * we could possibly do something more efficient than using the full
+ * expression-eval machinery for this. But probably there would be little
+ * benefit, and it'd require postgres_fdw to know more than is desirable
+ * about Param evaluation.)
+ */
+ *param_exprs = (List *) ExecInitExpr((Expr *) fdw_exprs, node);
+
+ /* Allocate buffer for text form of query parameters. */
+ *param_values = (const char **) palloc0(numParams * sizeof(char *));
+}
+
+/*
+ * Construct array of query parameter values in text format.
+ */
+static void
+process_query_params(ExprContext *econtext,
+ FmgrInfo *param_flinfo,
+ List *param_exprs,
+ const char **param_values)
+{
+ int nestlevel;
+ int i;
+ ListCell *lc;
+
+ nestlevel = set_transmission_modes();
+
+ i = 0;
+ foreach(lc, param_exprs)
+ {
+ ExprState *expr_state = (ExprState *) lfirst(lc);
+ Datum expr_value;
+ bool isNull;
+
+ /* Evaluate the parameter expression */
+ expr_value = ExecEvalExpr(expr_state, econtext, &isNull, NULL);
+
+ /*
+ * Get string representation of each parameter value by invoking
+ * type-specific output function, unless the value is null.
+ */
+ if (isNull)
+ param_values[i] = NULL;
+ else
+ param_values[i] = OutputFunctionCall(¶m_flinfo[i], expr_value);
+ i++;
+ }
+
+ reset_transmission_modes(nestlevel);
+}
+
+/*
* postgresAnalyzeForeignTable
* Test whether analyzing this foreign table is supported
*/
diff --git a/contrib/postgres_fdw/postgres_fdw.h b/contrib/postgres_fdw/postgres_fdw.h
index 4c731be..9fd2415 100644
--- a/contrib/postgres_fdw/postgres_fdw.h
+++ b/contrib/postgres_fdw/postgres_fdw.h
@@ -130,10 +130,24 @@ extern void deparseUpdateSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *targetAttrs, List *returningList,
List **retrieved_attrs);
+extern void deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseDeleteSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *returningList,
List **retrieved_attrs);
+extern void deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseAnalyzeSizeSql(StringInfo buf, Relation rel);
extern void deparseAnalyzeSql(StringInfo buf, Relation rel,
List **retrieved_attrs);
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index 885a5fb..5a65ea5 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -599,28 +599,32 @@ INSERT INTO ft2 (c1,c2,c3) SELECT c1+1000,c2+100, c3 || c3 FROM ft2 LIMIT 20;
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+EXPLAIN (verbose, costs off)
+UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+EXPLAIN (verbose, costs off)
+UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
- FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
+ FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
- DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
+ DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
-DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
+DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
EXPLAIN (verbose, costs off)
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
-UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
+UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
-DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
+DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
-- Test that trigger on remote table works as expected
@@ -937,6 +941,90 @@ UPDATE rem1 SET f2 = 'testo';
-- Test returning a system attribute
INSERT INTO rem1(f2) VALUES ('test') RETURNING ctid;
+-- cleanup
+DROP TRIGGER trig_row_before ON rem1;
+DROP TRIGGER trig_row_after ON rem1;
+DROP TRIGGER trig_local_before ON loc1;
+
+
+-- Test DML pushdown functionality
+
+-- Test with statement-level triggers
+CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+DROP TRIGGER trig_stmt_before ON rem1;
+
+CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+DROP TRIGGER trig_stmt_after ON rem1;
+
+-- Test with row-level ON INSERT triggers
+CREATE TRIGGER trig_row_before_insert
+BEFORE INSERT ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+DROP TRIGGER trig_row_before_insert ON rem1;
+
+CREATE TRIGGER trig_row_after_insert
+AFTER INSERT ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+DROP TRIGGER trig_row_after_insert ON rem1;
+
+-- Test with row-level ON UPDATE triggers
+CREATE TRIGGER trig_row_before_update
+BEFORE UPDATE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can't be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+DROP TRIGGER trig_row_before_update ON rem1;
+
+CREATE TRIGGER trig_row_after_update
+AFTER UPDATE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can't be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can be pushed down
+DROP TRIGGER trig_row_after_update ON rem1;
+
+-- Test with row-level ON DELETE triggers
+CREATE TRIGGER trig_row_before_delete
+BEFORE DELETE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can't be pushed down
+DROP TRIGGER trig_row_before_delete ON rem1;
+
+CREATE TRIGGER trig_row_after_delete
+AFTER DELETE ON rem1
+FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+EXPLAIN (verbose, costs off)
+UPDATE rem1 set f2 = ''; -- can be pushed down
+EXPLAIN (verbose, costs off)
+DELETE FROM rem1; -- can't be pushed down
+DROP TRIGGER trig_row_after_delete ON rem1;
+
-- ===================================================================
-- test inheritance features
-- ===================================================================
@@ -1068,6 +1156,13 @@ fetch from c;
update bar set f2 = null where current of c;
rollback;
+explain (verbose, costs off)
+delete from foo where f1 < 5 returning *;
+delete from foo where f1 < 5 returning *;
+explain (verbose, costs off)
+update bar set f2 = f2 + 100 returning *;
+update bar set f2 = f2 + 100 returning *;
+
drop table foo cascade;
drop table bar cascade;
drop table loct1;
diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml
index c24ddfc..087948f 100644
--- a/doc/src/sgml/fdwhandler.sgml
+++ b/doc/src/sgml/fdwhandler.sgml
@@ -678,6 +678,151 @@ IsForeignRelUpdatable (Relation rel);
updatability for display in the <literal>information_schema</> views.)
</para>
+ <para>
+ If an FDW supports optimizing foreign table updates, it still needs to
+ provide <function>PlanDMLPushdown</>, <function>BeginDMLPushdown</>,
+ <function>IterateDMLPushdown</> and <function>EndDMLPushdown</>
+ described below.
+ </para>
+
+ <para>
+<programlisting>
+bool
+PlanDMLPushdown (PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+</programlisting>
+
+ Decide whether it is safe to execute a foreign table update directly
+ on the remote server. If so, return <literal>true</> after performing
+ planning actions needed for that. Otherwise, return <literal>false</>.
+ This optional function is called during query planning.
+ If this function succeeds, <function>BeginDMLPushdown</>,
+ <function>IterateDMLPushdown</> and <function>EndDMLPushdown</> will be
+ called at the execution stage, instead. Otherwise, the table update
+ will be executed using the table-updating functions described above.
+ The parameters are the same as for <function>PlanForeignModify</>.
+ </para>
+
+ <para>
+ To execute the table update directly on the remote server, this function
+ must rewrite the target subplan with a <structname>ForeignScan</> plan
+ node that executes the table update directly on the remote server. The
+ <structfield>operation</> field of the <structname>ForeignScan</> must
+ be set to the <literal>CmdType</> enumeration appropriately; that is,
+ <literal>CMD_UPDATE</> for <command>UPDATE</>,
+ <literal>CMD_INSERT</> for <command>INSERT</>, and
+ <literal>CMD_DELETE</> for <command>DELETE</>.
+ </para>
+
+ <para>
+ See <xref linkend="fdw-planning"> for additional information.
+ </para>
+
+ <para>
+ If the <function>PlanDMLPushdown</> pointer is set to
+ <literal>NULL</>, no attempts to execute the table update directly on
+ the remote server are taken.
+ </para>
+
+ <para>
+<programlisting>
+void
+BeginDMLPushdown (ForeignScanState *node,
+ int eflags);
+</programlisting>
+
+ Begin executing a foreign table update directly on the remote server.
+ This is called during executor startup. It should perform any
+ initialization needed prior to the actual table update (that should be
+ done upon the first call to <function>IterateDMLPushdown</>).
+ The <structname>ForeignScanState</> node has already been created, but
+ its <structfield>fdw_state</> field is still NULL. Information about
+ the table to update is accessible through the
+ <structname>ForeignScanState</> node (in particular, from the underlying
+ <structname>ForeignScan</> plan node, which contains any FDW-private
+ information provided by <function>PlanDMLPushdown</>).
+ <literal>eflags</> contains flag bits describing the executor's
+ operating mode for this plan node.
+ </para>
+
+ <para>
+ Note that when <literal>(eflags & EXEC_FLAG_EXPLAIN_ONLY)</> is
+ true, this function should not perform any externally-visible actions;
+ it should only do the minimum required to make the node state valid
+ for <function>ExplainDMLPushdown</> and <function>EndDMLPushdown</>.
+ </para>
+
+ <para>
+ If the <function>BeginDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to execute the table update directly on
+ the remote server will fail with an error message.
+ </para>
+
+ <para>
+<programlisting>
+TupleTableSlot *
+IterateDMLPushdown (ForeignScanState *node);
+</programlisting>
+
+ When the <command>INSERT</>, <command>UPDATE</> or <command>DELETE</>
+ query doesn't have a <literal>RETURNING</> clause, just return NULL
+ after the actual table update directly executed on the remote server.
+ When the query has the clause, fetch one result containing the data
+ needed for the <literal>RETURNING</> calculation, returning it in a
+ tuple table slot (the node's <structfield>ScanTupleSlot</> should be
+ used for this purpose). The data that was actually inserted, updated
+ or deleted must be stored in the
+ <literal>es_result_relation_info->ri_projectReturning->pi_exprContext->ecxt_scantuple</>
+ of the node's <structname>EState</>.
+ Return NULL if no more rows are available.
+ Note that this is called in a short-lived memory context that will be
+ reset between invocations. Create a memory context in
+ <function>BeginDMLPushdown</> if you need longer-lived storage, or use
+ the <structfield>es_query_cxt</> of the node's <structname>EState</>.
+ </para>
+
+ <para>
+ The rows returned must match the <structfield>fdw_scan_tlist</> target
+ list if one was supplied, otherwise they must match the row type of the
+ foreign table being updated. If you choose to optimize away fetching
+ columns that are not needed for the <literal>RETURNING</> calculation,
+ you should insert nulls in those column positions, or else generate a
+ <structfield>fdw_scan_tlist</> list with those columns omitted.
+ </para>
+
+ <para>
+ Whether the query has the clause or not, the query's reported row count
+ must be incremented by the FDW itself. When the query doesn't have the
+ clause, the FDW must also increment the row count for the
+ <structname>ForeignScanState</> node in the <command>EXPLAIN ANALYZE</>
+ case.
+ </para>
+
+ <para>
+ If the <function>IterateDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to execute the table update directly on
+ the remote server will fail with an error message.
+ </para>
+
+ <para>
+<programlisting>
+void
+EndDMLPushdown (ForeignScanState *node);
+</programlisting>
+
+ End the table update and release resources. It is normally not important
+ to release palloc'd memory, but for example open files and connections
+ to remote servers should be cleaned up.
+ </para>
+
+ <para>
+ If the <function>EndDMLPushdown</> pointer is set to
+ <literal>NULL</>, attempts to execute the table update directly on
+ the remote server will fail with an error message.
+ </para>
+
</sect2>
<sect2 id="fdw-callbacks-row-locking">
@@ -865,6 +1010,29 @@ ExplainForeignModify (ModifyTableState *mtstate,
<command>EXPLAIN</>.
</para>
+ <para>
+<programlisting>
+void
+ExplainDMLPushdown (ForeignScanState *node,
+ ExplainState *es);
+</programlisting>
+
+ Print additional <command>EXPLAIN</> output for a foreign table update
+ that is executed directly on the remote server.
+ This function can call <function>ExplainPropertyText</> and
+ related functions to add fields to the <command>EXPLAIN</> output.
+ The flag fields in <literal>es</> can be used to determine what to
+ print, and the state of the <structname>ForeignScanState</> node
+ can be inspected to provide run-time statistics in the <command>EXPLAIN
+ ANALYZE</> case.
+ </para>
+
+ <para>
+ If the <function>ExplainDMLPushdown</> pointer is set to
+ <literal>NULL</>, no additional information is printed during
+ <command>EXPLAIN</>.
+ </para>
+
</sect2>
<sect2 id="fdw-callbacks-analyze">
@@ -1146,7 +1314,8 @@ GetForeignServerByName(const char *name, bool missing_ok);
<para>
The FDW callback functions <function>GetForeignRelSize</>,
<function>GetForeignPaths</>, <function>GetForeignPlan</>,
- <function>PlanForeignModify</>, and <function>GetForeignJoinPaths</>
+ <function>PlanForeignModify</>, <function>GetForeignJoinPaths</>, and
+ <function>PlanDMLPushdown</>
must fit into the workings of the <productname>PostgreSQL</> planner.
Here are some notes about what they must do.
</para>
@@ -1306,7 +1475,8 @@ GetForeignServerByName(const char *name, bool missing_ok);
<para>
When planning an <command>UPDATE</> or <command>DELETE</>,
- <function>PlanForeignModify</> can look up the <structname>RelOptInfo</>
+ <function>PlanForeignModify</> and <function>PlanDMLPushdown</>
+ can look up the <structname>RelOptInfo</>
struct for the foreign table and make use of the
<literal>baserel->fdw_private</> data previously created by the
scan-planning functions. However, in <command>INSERT</> the target
diff --git a/doc/src/sgml/postgres-fdw.sgml b/doc/src/sgml/postgres-fdw.sgml
index c408ba6..31547b5 100644
--- a/doc/src/sgml/postgres-fdw.sgml
+++ b/doc/src/sgml/postgres-fdw.sgml
@@ -484,6 +484,15 @@
extension that's listed in the foreign server's <literal>extensions</>
option. Operators and functions in such clauses must
be <literal>IMMUTABLE</> as well.
+ For an <command>UPDATE</> or <command>DELETE</> query,
+ <filename>postgres_fdw</> attempts to optimize the query execution by
+ sending the whole query to the remote server if there are no query
+ <literal>WHERE</> clauses that cannot be sent to the remote server,
+ no local joins for the query, and no row-level local <literal>BEFORE</> or
+ <literal>AFTER</> triggers on the target table. In <command>UPDATE</>,
+ expressions to assign to target columns must use only built-in data types,
+ <literal>IMMUTABLE</> operators, or <literal>IMMUTABLE</> functions,
+ to reduce the risk of misexecution of the query.
</para>
<para>
diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c
index ee13136..2b2ee3c 100644
--- a/src/backend/commands/explain.c
+++ b/src/backend/commands/explain.c
@@ -900,7 +900,29 @@ ExplainNode(PlanState *planstate, List *ancestors,
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
- pname = sname = "Foreign Scan";
+ sname = "Foreign Scan";
+ switch (((ForeignScan *) plan)->operation)
+ {
+ case CMD_SELECT:
+ pname = "Foreign Scan";
+ operation = "Select";
+ break;
+ case CMD_INSERT:
+ pname = "Foreign Insert";
+ operation = "Insert";
+ break;
+ case CMD_UPDATE:
+ pname = "Foreign Update";
+ operation = "Update";
+ break;
+ case CMD_DELETE:
+ pname = "Foreign Delete";
+ operation = "Delete";
+ break;
+ default:
+ pname = "???";
+ break;
+ }
break;
case T_CustomScan:
sname = "Custom Scan";
@@ -1648,6 +1670,21 @@ show_plan_tlist(PlanState *planstate, List *ancestors, ExplainState *es)
return;
if (IsA(plan, RecursiveUnion))
return;
+ /*
+ * Likewise for ForeignScan that has pushed down INSERT/UPDATE/DELETE
+ *
+ * Note: the tlist for an ForeignScan that has pushed down an INSERT/
+ * UPDATE might contain subplan output expressions that are confusing
+ * in this context. The tlist for an ForeignScan that has pushed down
+ * an UPDATE/DELETE always contains "junk" target columns to identify
+ * the exact row to update or delete, which would be confusing in this
+ * context. So, we suppress it in all the cases.
+ */
+ if (IsA(plan, ForeignScan) &&
+ (((ForeignScan *) plan)->operation == CMD_INSERT ||
+ ((ForeignScan *) plan)->operation == CMD_UPDATE ||
+ ((ForeignScan *) plan)->operation == CMD_DELETE))
+ return;
/* Set up deparsing context */
context = set_deparse_context_planstate(es->deparse_cxt,
@@ -2236,8 +2273,16 @@ show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es)
FdwRoutine *fdwroutine = fsstate->fdwroutine;
/* Let the FDW emit whatever fields it wants */
- if (fdwroutine->ExplainForeignScan != NULL)
- fdwroutine->ExplainForeignScan(fsstate, es);
+ if (((ForeignScan *) fsstate->ss.ps.plan)->operation != CMD_SELECT)
+ {
+ if (fdwroutine->ExplainDMLPushdown != NULL)
+ fdwroutine->ExplainDMLPushdown(fsstate, es);
+ }
+ else
+ {
+ if (fdwroutine->ExplainForeignScan != NULL)
+ fdwroutine->ExplainForeignScan(fsstate, es);
+ }
}
/*
@@ -2623,8 +2668,10 @@ show_modifytable_info(ModifyTableState *mtstate, List *ancestors,
}
}
- /* Give FDW a chance */
- if (fdwroutine && fdwroutine->ExplainForeignModify != NULL)
+ /* Give FDW a chance if needed */
+ if (!resultRelInfo->ri_FdwPushdown &&
+ fdwroutine != NULL &&
+ fdwroutine->ExplainForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, j);
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 76f7297..4883396 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1245,6 +1245,7 @@ InitResultRelInfo(ResultRelInfo *resultRelInfo,
else
resultRelInfo->ri_FdwRoutine = NULL;
resultRelInfo->ri_FdwState = NULL;
+ resultRelInfo->ri_FdwPushdown = false;
resultRelInfo->ri_ConstraintExprs = NULL;
resultRelInfo->ri_junkFilter = NULL;
resultRelInfo->ri_projectReturning = NULL;
diff --git a/src/backend/executor/nodeForeignscan.c b/src/backend/executor/nodeForeignscan.c
index 388c922..2cfe256 100644
--- a/src/backend/executor/nodeForeignscan.c
+++ b/src/backend/executor/nodeForeignscan.c
@@ -48,7 +48,10 @@ ForeignNext(ForeignScanState *node)
/* Call the Iterate function in short-lived context */
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
- slot = node->fdwroutine->IterateForeignScan(node);
+ if (plan->operation != CMD_SELECT)
+ slot = node->fdwroutine->IterateDMLPushdown(node);
+ else
+ slot = node->fdwroutine->IterateForeignScan(node);
MemoryContextSwitchTo(oldcontext);
/*
@@ -226,7 +229,10 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
/*
* Tell the FDW to initialize the scan.
*/
- fdwroutine->BeginForeignScan(scanstate, eflags);
+ if (node->operation != CMD_SELECT)
+ fdwroutine->BeginDMLPushdown(scanstate, eflags);
+ else
+ fdwroutine->BeginForeignScan(scanstate, eflags);
return scanstate;
}
@@ -240,8 +246,13 @@ ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
void
ExecEndForeignScan(ForeignScanState *node)
{
+ ForeignScan *plan = (ForeignScan *) node->ss.ps.plan;
+
/* Let the FDW shut down */
- node->fdwroutine->EndForeignScan(node);
+ if (plan->operation != CMD_SELECT)
+ node->fdwroutine->EndDMLPushdown(node);
+ else
+ node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 27051e8..890ed59 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -138,13 +138,17 @@ ExecCheckPlanOutput(Relation resultRel, List *targetList)
* tupleSlot: slot holding tuple actually inserted/updated/deleted
* planSlot: slot holding tuple returned by top subplan node
*
+ * Note: If tupleSlot is NULL, the FDW should have already provided econtext's
+ * scan tuple.
+ *
* Returns a slot holding the result tuple
*/
static TupleTableSlot *
-ExecProcessReturning(ProjectionInfo *projectReturning,
+ExecProcessReturning(ResultRelInfo *resultRelInfo,
TupleTableSlot *tupleSlot,
TupleTableSlot *planSlot)
{
+ ProjectionInfo *projectReturning = resultRelInfo->ri_projectReturning;
ExprContext *econtext = projectReturning->pi_exprContext;
/*
@@ -154,7 +158,20 @@ ExecProcessReturning(ProjectionInfo *projectReturning,
ResetExprContext(econtext);
/* Make tuple and any needed join variables available to ExecProject */
- econtext->ecxt_scantuple = tupleSlot;
+ if (tupleSlot)
+ econtext->ecxt_scantuple = tupleSlot;
+ else
+ {
+ HeapTuple tuple;
+
+ /*
+ * RETURNING expressions might reference the tableoid column, so
+ * initialize t_tableOid before evaluating them.
+ */
+ Assert(!TupIsNull(econtext->ecxt_scantuple));
+ tuple = ExecMaterializeSlot(econtext->ecxt_scantuple);
+ tuple->t_tableOid = RelationGetRelid(resultRelInfo->ri_RelationDesc);
+ }
econtext->ecxt_outertuple = planSlot;
/* Compute the RETURNING expressions */
@@ -496,8 +513,7 @@ ExecInsert(ModifyTableState *mtstate,
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
- return ExecProcessReturning(resultRelInfo->ri_projectReturning,
- slot, planSlot);
+ return ExecProcessReturning(resultRelInfo, slot, planSlot);
return NULL;
}
@@ -738,8 +754,7 @@ ldelete:;
ExecStoreTuple(&deltuple, slot, InvalidBuffer, false);
}
- rslot = ExecProcessReturning(resultRelInfo->ri_projectReturning,
- slot, planSlot);
+ rslot = ExecProcessReturning(resultRelInfo, slot, planSlot);
/*
* Before releasing the target tuple again, make sure rslot has a
@@ -1024,8 +1039,7 @@ lreplace:;
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
- return ExecProcessReturning(resultRelInfo->ri_projectReturning,
- slot, planSlot);
+ return ExecProcessReturning(resultRelInfo, slot, planSlot);
return NULL;
}
@@ -1380,6 +1394,21 @@ ExecModifyTable(ModifyTableState *node)
break;
}
+ /*
+ * If ri_FdwPushdown is true, all we need to do here is compute the
+ * RETURNING expressions.
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ Assert(resultRelInfo->ri_projectReturning);
+
+ /* No need to provide scan tuple to ExecProcessReturning. */
+ slot = ExecProcessReturning(resultRelInfo, NULL, planSlot);
+
+ estate->es_result_relation_info = saved_resultRelInfo;
+ return slot;
+ }
+
EvalPlanQualSetSlot(&node->mt_epqstate, planSlot);
slot = planSlot;
@@ -1559,6 +1588,9 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
{
subplan = (Plan *) lfirst(l);
+ /* Initialize the FdwPushdown flag */
+ resultRelInfo->ri_FdwPushdown = bms_is_member(i, node->fdwPushdowns);
+
/*
* Verify result relation is a valid target for the current operation
*/
@@ -1583,7 +1615,8 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
/* Also let FDWs init themselves for foreign-table result rels */
- if (resultRelInfo->ri_FdwRoutine != NULL &&
+ if (!resultRelInfo->ri_FdwPushdown &&
+ resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, i);
@@ -1754,13 +1787,26 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
erm = ExecFindRowMark(estate, rc->rti, false);
/* build ExecAuxRowMark for each subplan */
+ resultRelInfo = mtstate->resultRelInfo;
for (i = 0; i < nplans; i++)
{
ExecAuxRowMark *aerm;
+ /*
+ * ignore subplan if the FDW pushes down the command to the remote
+ * server; the ModifyTable won't have anything to do except for
+ * evaluation of RETURNING expressions
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ resultRelInfo++;
+ continue;
+ }
+
subplan = mtstate->mt_plans[i]->plan;
aerm = ExecBuildAuxRowMark(erm, subplan->targetlist);
mtstate->mt_arowmarks[i] = lappend(mtstate->mt_arowmarks[i], aerm);
+ resultRelInfo++;
}
}
@@ -1821,6 +1867,17 @@ ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
ExecCheckPlanOutput(resultRelInfo->ri_RelationDesc,
subplan->targetlist);
+ /*
+ * ignore subplan if the FDW pushes down the command to the
+ * remote server; the ModifyTable won't have anything to do
+ * except for evaluation of RETURNING expressions
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ resultRelInfo++;
+ continue;
+ }
+
j = ExecInitJunkFilter(subplan->targetlist,
resultRelInfo->ri_RelationDesc->rd_att->tdhasoid,
ExecInitExtraTupleSlot(estate));
@@ -1910,7 +1967,8 @@ ExecEndModifyTable(ModifyTableState *node)
{
ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
- if (resultRelInfo->ri_FdwRoutine != NULL &&
+ if (!resultRelInfo->ri_FdwPushdown &&
+ resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->EndForeignModify != NULL)
resultRelInfo->ri_FdwRoutine->EndForeignModify(node->ps.state,
resultRelInfo);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index a9e9cc3..2604fa3 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -188,6 +188,7 @@ _copyModifyTable(const ModifyTable *from)
COPY_NODE_FIELD(withCheckOptionLists);
COPY_NODE_FIELD(returningLists);
COPY_NODE_FIELD(fdwPrivLists);
+ COPY_BITMAPSET_FIELD(fdwPushdowns);
COPY_NODE_FIELD(rowMarks);
COPY_SCALAR_FIELD(epqParam);
COPY_SCALAR_FIELD(onConflictAction);
@@ -648,6 +649,7 @@ _copyForeignScan(const ForeignScan *from)
/*
* copy remainder of node
*/
+ COPY_SCALAR_FIELD(operation);
COPY_SCALAR_FIELD(fs_server);
COPY_NODE_FIELD(fdw_exprs);
COPY_NODE_FIELD(fdw_private);
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 85acce8..e1113d5 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -356,6 +356,7 @@ _outModifyTable(StringInfo str, const ModifyTable *node)
WRITE_NODE_FIELD(withCheckOptionLists);
WRITE_NODE_FIELD(returningLists);
WRITE_NODE_FIELD(fdwPrivLists);
+ WRITE_BITMAPSET_FIELD(fdwPushdowns);
WRITE_NODE_FIELD(rowMarks);
WRITE_INT_FIELD(epqParam);
WRITE_ENUM_FIELD(onConflictAction, OnConflictAction);
@@ -608,6 +609,7 @@ _outForeignScan(StringInfo str, const ForeignScan *node)
_outScanInfo(str, (const Scan *) node);
+ WRITE_ENUM_FIELD(operation, CmdType);
WRITE_OID_FIELD(fs_server);
WRITE_NODE_FIELD(fdw_exprs);
WRITE_NODE_FIELD(fdw_private);
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index e6e6f29..beefa8e 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -1481,6 +1481,7 @@ _readModifyTable(void)
READ_NODE_FIELD(withCheckOptionLists);
READ_NODE_FIELD(returningLists);
READ_NODE_FIELD(fdwPrivLists);
+ READ_BITMAPSET_FIELD(fdwPushdowns);
READ_NODE_FIELD(rowMarks);
READ_INT_FIELD(epqParam);
READ_ENUM_FIELD(onConflictAction, OnConflictAction);
diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 198b06b..48d161c 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -3791,6 +3791,7 @@ make_foreignscan(List *qptlist,
plan->lefttree = outer_plan;
plan->righttree = NULL;
node->scan.scanrelid = scanrelid;
+ node->operation = CMD_SELECT;
/* fs_server will be filled in by create_foreignscan_plan */
node->fs_server = InvalidOid;
node->fdw_exprs = fdw_exprs;
@@ -5068,6 +5069,7 @@ make_modifytable(PlannerInfo *root,
Plan *plan = &node->plan;
double total_size;
List *fdw_private_list;
+ Bitmapset *fdwpushdowns;
ListCell *subnode;
ListCell *lc;
int i;
@@ -5148,12 +5150,14 @@ make_modifytable(PlannerInfo *root,
* construct private plan data, and accumulate it all into a list.
*/
fdw_private_list = NIL;
+ fdwpushdowns = NULL;
i = 0;
foreach(lc, resultRelations)
{
Index rti = lfirst_int(lc);
FdwRoutine *fdwroutine;
List *fdw_private;
+ bool fdwpushdown;
/*
* If possible, we want to get the FdwRoutine from our RelOptInfo for
@@ -5180,7 +5184,23 @@ make_modifytable(PlannerInfo *root,
fdwroutine = NULL;
}
+ /*
+ * If the target relation has any row-level triggers, we can't push
+ * down the command to the remote server.
+ */
+ fdwpushdown = false;
if (fdwroutine != NULL &&
+ fdwroutine->PlanDMLPushdown != NULL &&
+ fdwroutine->BeginDMLPushdown != NULL &&
+ fdwroutine->IterateDMLPushdown != NULL &&
+ fdwroutine->EndDMLPushdown != NULL &&
+ !has_row_triggers(root, rti, operation))
+ fdwpushdown = fdwroutine->PlanDMLPushdown(root, node, rti, i);
+ if (fdwpushdown)
+ fdwpushdowns = bms_add_member(fdwpushdowns, i);
+
+ if (!fdwpushdown &&
+ fdwroutine != NULL &&
fdwroutine->PlanForeignModify != NULL)
fdw_private = fdwroutine->PlanForeignModify(root, node, rti, i);
else
@@ -5189,6 +5209,7 @@ make_modifytable(PlannerInfo *root,
i++;
}
node->fdwPrivLists = fdw_private_list;
+ node->fdwPushdowns = fdwpushdowns;
return node;
}
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 0ea9fcf..8d19b37 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -1520,3 +1520,50 @@ has_unique_index(RelOptInfo *rel, AttrNumber attno)
}
return false;
}
+
+
+/*
+ * has_row_triggers
+ *
+ * Detect whether the specified relation has any row-level triggers for event.
+ */
+bool
+has_row_triggers(PlannerInfo *root, Index rti, CmdType event)
+{
+ RangeTblEntry *rte = planner_rt_fetch(rti, root);
+ Relation relation;
+ TriggerDesc *trigDesc;
+ bool result = false;
+
+ /* Assume we already have adequate lock */
+ relation = heap_open(rte->relid, NoLock);
+
+ trigDesc = relation->trigdesc;
+ switch (event)
+ {
+ case CMD_INSERT:
+ if (trigDesc &&
+ (trigDesc->trig_insert_after_row ||
+ trigDesc->trig_insert_before_row))
+ result = true;
+ break;
+ case CMD_UPDATE:
+ if (trigDesc &&
+ (trigDesc->trig_update_after_row ||
+ trigDesc->trig_update_before_row))
+ result = true;
+ break;
+ case CMD_DELETE:
+ if (trigDesc &&
+ (trigDesc->trig_delete_after_row ||
+ trigDesc->trig_delete_before_row))
+ result = true;
+ break;
+ default:
+ elog(ERROR, "unrecognized CmdType: %d", (int) event);
+ break;
+ }
+
+ heap_close(relation, NoLock);
+ return result;
+}
diff --git a/src/include/foreign/fdwapi.h b/src/include/foreign/fdwapi.h
index 9fafab0..e150bea 100644
--- a/src/include/foreign/fdwapi.h
+++ b/src/include/foreign/fdwapi.h
@@ -94,6 +94,18 @@ typedef void (*EndForeignModify_function) (EState *estate,
typedef int (*IsForeignRelUpdatable_function) (Relation rel);
+typedef bool (*PlanDMLPushdown_function) (PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+
+typedef void (*BeginDMLPushdown_function) (ForeignScanState *node,
+ int eflags);
+
+typedef TupleTableSlot *(*IterateDMLPushdown_function) (ForeignScanState *node);
+
+typedef void (*EndDMLPushdown_function) (ForeignScanState *node);
+
typedef RowMarkType (*GetForeignRowMarkType_function) (RangeTblEntry *rte,
LockClauseStrength strength);
@@ -111,6 +123,9 @@ typedef void (*ExplainForeignModify_function) (ModifyTableState *mtstate,
int subplan_index,
struct ExplainState *es);
+typedef void (*ExplainDMLPushdown_function) (ForeignScanState *node,
+ struct ExplainState *es);
+
typedef int (*AcquireSampleRowsFunc) (Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
@@ -171,6 +186,10 @@ typedef struct FdwRoutine
ExecForeignDelete_function ExecForeignDelete;
EndForeignModify_function EndForeignModify;
IsForeignRelUpdatable_function IsForeignRelUpdatable;
+ PlanDMLPushdown_function PlanDMLPushdown;
+ BeginDMLPushdown_function BeginDMLPushdown;
+ IterateDMLPushdown_function IterateDMLPushdown;
+ EndDMLPushdown_function EndDMLPushdown;
/* Functions for SELECT FOR UPDATE/SHARE row locking */
GetForeignRowMarkType_function GetForeignRowMarkType;
@@ -180,6 +199,7 @@ typedef struct FdwRoutine
/* Support functions for EXPLAIN */
ExplainForeignScan_function ExplainForeignScan;
ExplainForeignModify_function ExplainForeignModify;
+ ExplainDMLPushdown_function ExplainDMLPushdown;
/* Support functions for ANALYZE */
AnalyzeForeignTable_function AnalyzeForeignTable;
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 064a050..fe160db 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -311,6 +311,7 @@ typedef struct JunkFilter
* TrigInstrument optional runtime measurements for triggers
* FdwRoutine FDW callback functions, if foreign table
* FdwState available to save private state of FDW
+ * FdwPushdown true when the command is pushed down
* WithCheckOptions list of WithCheckOption's to be checked
* WithCheckOptionExprs list of WithCheckOption expr states
* ConstraintExprs array of constraint-checking expr states
@@ -334,6 +335,7 @@ typedef struct ResultRelInfo
Instrumentation *ri_TrigInstrument;
struct FdwRoutine *ri_FdwRoutine;
void *ri_FdwState;
+ bool ri_FdwPushdown;
List *ri_WithCheckOptions;
List *ri_WithCheckOptionExprs;
List **ri_ConstraintExprs;
diff --git a/src/include/nodes/pg_list.h b/src/include/nodes/pg_list.h
index 1cb95ee..77b50ff 100644
--- a/src/include/nodes/pg_list.h
+++ b/src/include/nodes/pg_list.h
@@ -134,16 +134,19 @@ list_length(const List *l)
#define list_make2(x1,x2) lcons(x1, list_make1(x2))
#define list_make3(x1,x2,x3) lcons(x1, list_make2(x2, x3))
#define list_make4(x1,x2,x3,x4) lcons(x1, list_make3(x2, x3, x4))
+#define list_make5(x1,x2,x3,x4,x5) lcons(x1, list_make4(x2, x3, x4, x5))
#define list_make1_int(x1) lcons_int(x1, NIL)
#define list_make2_int(x1,x2) lcons_int(x1, list_make1_int(x2))
#define list_make3_int(x1,x2,x3) lcons_int(x1, list_make2_int(x2, x3))
#define list_make4_int(x1,x2,x3,x4) lcons_int(x1, list_make3_int(x2, x3, x4))
+#define list_make5_int(x1,x2,x3,x4,x5) lcons_int(x1, list_make4_int(x2, x3, x4, x5))
#define list_make1_oid(x1) lcons_oid(x1, NIL)
#define list_make2_oid(x1,x2) lcons_oid(x1, list_make1_oid(x2))
#define list_make3_oid(x1,x2,x3) lcons_oid(x1, list_make2_oid(x2, x3))
#define list_make4_oid(x1,x2,x3,x4) lcons_oid(x1, list_make3_oid(x2, x3, x4))
+#define list_make5_oid(x1,x2,x3,x4,x5) lcons_oid(x1, list_make4_oid(x2, x3, x4, x5))
/*
* foreach -
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index ae224cf..9f8185a 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -189,6 +189,7 @@ typedef struct ModifyTable
List *withCheckOptionLists; /* per-target-table WCO lists */
List *returningLists; /* per-target-table RETURNING tlists */
List *fdwPrivLists; /* per-target-table FDW private data lists */
+ Bitmapset *fdwPushdowns; /* indices of plans that push down command */
List *rowMarks; /* PlanRowMarks (non-locking only) */
int epqParam; /* ID of Param for EvalPlanQual re-eval */
OnConflictAction onConflictAction; /* ON CONFLICT action */
@@ -531,6 +532,7 @@ typedef struct WorkTableScan
typedef struct ForeignScan
{
Scan scan;
+ CmdType operation; /* SELECT/INSERT/UPDATE/DELETE */
Oid fs_server; /* OID of foreign server */
List *fdw_exprs; /* expressions that FDW may evaluate */
List *fdw_private; /* private data for FDW */
diff --git a/src/include/optimizer/plancat.h b/src/include/optimizer/plancat.h
index 52335fa..125274e 100644
--- a/src/include/optimizer/plancat.h
+++ b/src/include/optimizer/plancat.h
@@ -55,4 +55,6 @@ extern Selectivity join_selectivity(PlannerInfo *root,
JoinType jointype,
SpecialJoinInfo *sjinfo);
+extern bool has_row_triggers(PlannerInfo *root, Index rti, CmdType event);
+
#endif /* PLANCAT_H */
On 2016/02/22 20:13, Rushabh Lathia wrote:
I did another round of review for the latest patch and well as performed
the sanity test, and
haven't found any functional issues. Found couple of issue, see in-line
comments
for the same.
Thanks!
On Thu, Feb 18, 2016 at 3:15 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp <mailto:fujita.etsuro@lab.ntt.co.jp>> wrote:On 2016/02/12 21:19, Etsuro Fujita wrote:
+ /* Check point 1 */ + if (operation == CMD_INSERT) + return false; + + /* Check point 2 */ + if (nodeTag(subplan) != T_ForeignScan) + return false; + + /* Check point 3 */ + if (subplan->qual != NIL) + return false; + + /* Check point 4 */ + if (operation == CMD_UPDATE)These comments are referring to something in the function header
further up, but you could instead just delete the stuff from the
header and mention the actual conditions here.
Will fix.
Done.
The patch doesn't allow the postgres_fdw to push down an
UPDATE/DELETE on a foreign join, so I added one more condition here
not to handle such cases. (I'm planning to propose a patch to
handle such cases, in the next CF.)
I think we should place the checking foreign join condition before the
target columns, as foreign join condition is less costly then the target
columns.
Agreed.
Other changes:
* I keep Rushabh's code change that we call PlanDMLPushdown only
when all the required APIs are available with FDW, but for
CheckValidResultRel, I left the code as-is (no changes to that
function), to match the docs saying that the FDW needs to provide
the DML pushdown callback functions together with existing
table-updating functions such as ExecForeignInsert,
ExecForeignUpdate and ExecForeignDelete.
I think we should also update the CheckValidResultRel(), because even
though ExecForeignInsert,
ExecForeignUpdate and ExecForeignDelete not present, FDW still can perform
UPDATE/DELETE/INSERT using DML Pushdown APIs. Lets take committer's view
on this.
OK
PFA update patch, which includes changes into postgresPlanDMLPushdown()
to check for join
condition before target columns and also fixed couple of whitespace issues.
Thanks again for updating the patch and fixing the issues!
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/02/22 20:13, Rushabh Lathia wrote:
PFA update patch, which includes changes into postgresPlanDMLPushdown()
to check for join
condition before target columns and also fixed couple of whitespace issues.
For pushing down an UPDATE/DELETE on a foreign join to the remote, I
created a WIP patch on top of the latest version of the DML pushdown
patch. Attached is the WIP patch. I'd like to propose this as part of
(I'd like to discuss this as a separate patch, though):
https://commitfest.postgresql.org/9/453/
The patch doesn't correctly evaluate the values of system columns of
joined relations in RETURNING, other than ctid. I'll fix that ASAP.
Best regards,
Etsuro Fujita
Attachments:
fdw-dml-pushdown-ext-v1.patchtext/x-diff; name=fdw-dml-pushdown-ext-v1.patchDownload
*** a/contrib/postgres_fdw/deparse.c
--- b/contrib/postgres_fdw/deparse.c
***************
*** 130,135 **** static void deparseReturningList(StringInfo buf, PlannerInfo *root,
--- 130,143 ----
bool trig_after_row,
List *returningList,
List **retrieved_attrs);
+ static void deparseJoinedReturningList(List *fdw_scan_tlist,
+ Index rtindex,
+ Bitmapset *attrs_used,
+ List *returningList,
+ List **retrieved_attrs,
+ List **result_attrs,
+ List **result_attrnos,
+ deparse_expr_cxt *context);
static void deparseColumnRef(StringInfo buf, int varno, int varattno,
PlannerInfo *root, bool qualify_col);
static void deparseRelation(StringInfo buf, Relation rel);
***************
*** 158,164 **** static void deparseLockingClause(deparse_expr_cxt *context);
static void appendOrderByClause(List *pathkeys, deparse_expr_cxt *context);
static void appendConditions(List *exprs, deparse_expr_cxt *context);
static void deparseFromExprForRel(StringInfo buf, PlannerInfo *root,
! RelOptInfo *joinrel, bool use_alias, List **params_list);
/*
--- 166,175 ----
static void appendOrderByClause(List *pathkeys, deparse_expr_cxt *context);
static void appendConditions(List *exprs, deparse_expr_cxt *context);
static void deparseFromExprForRel(StringInfo buf, PlannerInfo *root,
! RelOptInfo *joinrel,
! Index ignore_rel,
! bool use_alias,
! List **params_list);
/*
***************
*** 850,856 **** deparseSelectSql(List *tlist, List **retrieved_attrs, deparse_expr_cxt *context)
* Construct FROM clause
*/
appendStringInfoString(buf, " FROM ");
! deparseFromExprForRel(buf, root, foreignrel,
(foreignrel->reloptkind == RELOPT_JOINREL),
context->params_list);
}
--- 861,867 ----
* Construct FROM clause
*/
appendStringInfoString(buf, " FROM ");
! deparseFromExprForRel(buf, root, foreignrel, (Index) 0,
(foreignrel->reloptkind == RELOPT_JOINREL),
context->params_list);
}
***************
*** 1135,1144 **** deparseExplicitTargetList(List *tlist, List **retrieved_attrs,
* The function constructs ... JOIN ... ON ... for join relation. For a base
* relation it just returns schema-qualified tablename, with the appropriate
* alias if so requested.
*/
static void
! deparseFromExprForRel(StringInfo buf, PlannerInfo *root, RelOptInfo *foreignrel,
! bool use_alias, List **params_list)
{
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) foreignrel->fdw_private;
--- 1146,1166 ----
* The function constructs ... JOIN ... ON ... for join relation. For a base
* relation it just returns schema-qualified tablename, with the appropriate
* alias if so requested.
+ *
+ * If constructing FROM clause of UPDATE statement or USING clause of DELETE
+ * statement, we simply ignore the ignore_rel target relation when deparsing
+ * the join to the target relation. Note that the join is safely interchanged
+ * with higher-level outer joins (if any) by outer-join identity 1 since that
+ * the join won't appear on the nullable side of such outer joins (we currently
+ * don't allow the result relation to appear on the nullable side of an outer
+ * join) and that the target relation won't be outer-joined to other relations.
*/
static void
! deparseFromExprForRel(StringInfo buf, PlannerInfo *root,
! RelOptInfo *foreignrel,
! Index ignore_rel,
! bool use_alias,
! List **params_list)
{
PgFdwRelationInfo *fpinfo = (PgFdwRelationInfo *) foreignrel->fdw_private;
***************
*** 1148,1169 **** deparseFromExprForRel(StringInfo buf, PlannerInfo *root, RelOptInfo *foreignrel,
RelOptInfo *rel_i = fpinfo->innerrel;
StringInfoData join_sql_o;
StringInfoData join_sql_i;
/* Deparse outer relation */
! initStringInfo(&join_sql_o);
! deparseFromExprForRel(&join_sql_o, root, rel_o, true, params_list);
/* Deparse inner relation */
! initStringInfo(&join_sql_i);
! deparseFromExprForRel(&join_sql_i, root, rel_i, true, params_list);
/*
* For a join relation FROM clause entry is deparsed as
*
* ((outer relation) <join type> (inner relation) ON (joinclauses)
*/
! appendStringInfo(buf, "(%s %s JOIN %s ON ", join_sql_o.data,
! get_jointype_name(fpinfo->jointype), join_sql_i.data);
/* Append join clause; (TRUE) if no join clause */
if (fpinfo->joinclauses)
--- 1170,1260 ----
RelOptInfo *rel_i = fpinfo->innerrel;
StringInfoData join_sql_o;
StringInfoData join_sql_i;
+ bool do_deparse_o = true;
+ bool do_deparse_i = true;
+ Relids result = NULL;
+ Relids target = NULL;
+
+ if (ignore_rel > 0)
+ {
+ int varno_o = -1;
+ int varno_i = -1;
+
+ do_deparse_o =
+ !(bms_get_singleton_member(rel_o->relids, &varno_o) &&
+ (varno_o == ignore_rel));
+ do_deparse_i =
+ !(bms_get_singleton_member(rel_i->relids, &varno_i) &&
+ (varno_i == ignore_rel));
+ }
/* Deparse outer relation */
! if (do_deparse_o)
! {
! initStringInfo(&join_sql_o);
! deparseFromExprForRel(&join_sql_o, root, rel_o, ignore_rel, true,
! params_list);
! }
/* Deparse inner relation */
! if (do_deparse_i)
! {
! initStringInfo(&join_sql_i);
! deparseFromExprForRel(&join_sql_i, root, rel_i, ignore_rel, true,
! params_list);
! }
!
! if (!do_deparse_o || !do_deparse_i)
! {
! /* This should be for UPDATE/DELETE */
! Assert(ignore_rel > 0);
! /* The join should be an inner join */
! Assert(fpinfo->joinclauses == NIL);
!
! /* Don't parenthesize the expression */
! if (!do_deparse_o)
! appendStringInfo(buf, "%s", join_sql_i.data);
! else
! appendStringInfo(buf, "%s", join_sql_o.data);
! return;
! }
/*
* For a join relation FROM clause entry is deparsed as
*
* ((outer relation) <join type> (inner relation) ON (joinclauses)
+ *
+ * Note: if constructing FROM clause of UPDATE or USING clause of DELETE,
+ * don't parenthesize the topmost expression.
*/
!
! /* Begin the FROM clause entry. */
! if (ignore_rel == 0 && bms_equal(foreignrel->relids, root->all_baserels))
! appendStringInfoChar(buf, '(');
!
! target = bms_make_singleton(ignore_rel);
!
! if (ignore_rel == 0)
! result = rel_o->relids;
! else
! result = bms_difference(rel_o->relids, target);
! if (bms_num_members(result) > 1)
! appendStringInfo(buf, "(%s)", join_sql_o.data);
! else
! appendStringInfo(buf, "%s", join_sql_o.data);
!
! appendStringInfo(buf, " %s JOIN ", get_jointype_name(fpinfo->jointype));
!
! if (ignore_rel == 0)
! result = rel_i->relids;
! else
! result = bms_difference(rel_i->relids, target);
! if (bms_num_members(result) > 1)
! appendStringInfo(buf, "(%s)", join_sql_i.data);
! else
! appendStringInfo(buf, "%s", join_sql_i.data);
!
! appendStringInfoString(buf, " ON ");
/* Append join clause; (TRUE) if no join clause */
if (fpinfo->joinclauses)
***************
*** 1183,1189 **** deparseFromExprForRel(StringInfo buf, PlannerInfo *root, RelOptInfo *foreignrel,
appendStringInfoString(buf, "(TRUE)");
/* End the FROM clause entry. */
! appendStringInfo(buf, ")");
}
else
{
--- 1274,1281 ----
appendStringInfoString(buf, "(TRUE)");
/* End the FROM clause entry. */
! if (ignore_rel == 0 && bms_equal(foreignrel->relids, root->all_baserels))
! appendStringInfoChar(buf, ')');
}
else
{
***************
*** 1207,1213 **** deparseFromExprForRel(StringInfo buf, PlannerInfo *root, RelOptInfo *foreignrel,
heap_close(rel, NoLock);
}
- return;
}
/*
--- 1299,1304 ----
***************
*** 1325,1338 **** deparseUpdateSql(StringInfo buf, PlannerInfo *root,
void
deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *targetlist,
List *targetAttrs,
List *remote_conds,
List **params_list,
List *returningList,
! List **retrieved_attrs)
{
- RelOptInfo *baserel = root->simple_rel_array[rtindex];
deparse_expr_cxt context;
int nestlevel;
bool first;
--- 1416,1433 ----
void
deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
+ RelOptInfo *foreignrel,
+ List *fdw_scan_tlist,
+ Bitmapset *attrs_used,
List *targetlist,
List *targetAttrs,
List *remote_conds,
List **params_list,
List *returningList,
! List **retrieved_attrs,
! List **result_attrs,
! List **result_attrnos)
{
deparse_expr_cxt context;
int nestlevel;
bool first;
***************
*** 1340,1351 **** deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
/* Set up context struct for recursion */
context.root = root;
! context.foreignrel = baserel;
context.buf = buf;
context.params_list = params_list;
appendStringInfoString(buf, "UPDATE ");
deparseRelation(buf, rel);
appendStringInfoString(buf, " SET ");
/* Make sure any constants in the exprs are printed portably */
--- 1435,1448 ----
/* Set up context struct for recursion */
context.root = root;
! context.foreignrel = foreignrel;
context.buf = buf;
context.params_list = params_list;
appendStringInfoString(buf, "UPDATE ");
deparseRelation(buf, rel);
+ if (foreignrel->reloptkind == RELOPT_JOINREL)
+ appendStringInfo(buf, " %s%d", REL_ALIAS_PREFIX, rtindex);
appendStringInfoString(buf, " SET ");
/* Make sure any constants in the exprs are printed portably */
***************
*** 1368,1381 **** deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
reset_transmission_modes(nestlevel);
if (remote_conds)
{
appendStringInfo(buf, " WHERE ");
appendConditions(remote_conds, &context);
}
! deparseReturningList(buf, root, rtindex, rel, false,
! returningList, retrieved_attrs);
}
/*
--- 1465,1490 ----
reset_transmission_modes(nestlevel);
+ if (foreignrel->reloptkind == RELOPT_JOINREL)
+ {
+ appendStringInfoString(buf, " FROM ");
+ deparseFromExprForRel(buf, root, foreignrel, rtindex, true,
+ params_list);
+ }
+
if (remote_conds)
{
appendStringInfo(buf, " WHERE ");
appendConditions(remote_conds, &context);
}
! if (foreignrel->reloptkind == RELOPT_JOINREL)
! deparseJoinedReturningList(fdw_scan_tlist, rtindex, attrs_used,
! returningList, retrieved_attrs,
! result_attrs, result_attrnos, &context);
! else
! deparseReturningList(buf, root, rtindex, rel, false,
! returningList, retrieved_attrs);
}
/*
***************
*** 1410,1431 **** deparseDeleteSql(StringInfo buf, PlannerInfo *root,
void
deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *remote_conds,
List **params_list,
List *returningList,
! List **retrieved_attrs)
{
- RelOptInfo *baserel = root->simple_rel_array[rtindex];
deparse_expr_cxt context;
/* Set up context struct for recursion */
context.root = root;
! context.foreignrel = baserel;
context.buf = buf;
context.params_list = params_list;
appendStringInfoString(buf, "DELETE FROM ");
deparseRelation(buf, rel);
if (remote_conds)
{
--- 1519,1553 ----
void
deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
+ RelOptInfo *foreignrel,
+ List *fdw_scan_tlist,
+ Bitmapset *attrs_used,
List *remote_conds,
List **params_list,
List *returningList,
! List **retrieved_attrs,
! List **result_attrs,
! List **result_attrnos)
{
deparse_expr_cxt context;
/* Set up context struct for recursion */
context.root = root;
! context.foreignrel = foreignrel;
context.buf = buf;
context.params_list = params_list;
appendStringInfoString(buf, "DELETE FROM ");
deparseRelation(buf, rel);
+ if (foreignrel->reloptkind == RELOPT_JOINREL)
+ appendStringInfo(buf, " %s%d", REL_ALIAS_PREFIX, rtindex);
+
+ if (foreignrel->reloptkind == RELOPT_JOINREL)
+ {
+ appendStringInfoString(buf, " USING ");
+ deparseFromExprForRel(buf, root, foreignrel, rtindex, true,
+ params_list);
+ }
if (remote_conds)
{
***************
*** 1433,1440 **** deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
appendConditions(remote_conds, &context);
}
! deparseReturningList(buf, root, rtindex, rel, false,
! returningList, retrieved_attrs);
}
/*
--- 1555,1567 ----
appendConditions(remote_conds, &context);
}
! if (foreignrel->reloptkind == RELOPT_JOINREL)
! deparseJoinedReturningList(fdw_scan_tlist, rtindex, attrs_used,
! returningList, retrieved_attrs,
! result_attrs, result_attrnos, &context);
! else
! deparseReturningList(buf, root, rtindex, rel, false,
! returningList, retrieved_attrs);
}
/*
***************
*** 1474,1479 **** deparseReturningList(StringInfo buf, PlannerInfo *root,
--- 1601,1682 ----
}
/*
+ * Add a RETURNING clause, if needed, to an UPDATE/DELETE on a foreign join.
+ */
+ static void
+ deparseJoinedReturningList(List *fdw_scan_tlist,
+ Index rtindex,
+ Bitmapset *attrs_used,
+ List *returningList,
+ List **retrieved_attrs,
+ List **result_attrs,
+ List **result_attrnos,
+ deparse_expr_cxt *context)
+ {
+ StringInfo buf = context->buf;
+ List *returning_vars;
+ bool have_wholerow;
+ bool first;
+ ListCell *lc;
+ int i;
+
+ *retrieved_attrs = NIL;
+ *result_attrs = NIL;
+ *result_attrnos = NIL;
+
+ if (returningList == NIL)
+ return;
+
+ /* Pull out all the Vars mentioned in returningList */
+ returning_vars = pull_var_clause((Node *) returningList,
+ PVC_REJECT_AGGREGATES,
+ PVC_RECURSE_PLACEHOLDERS);
+
+ /*
+ * If there's a whole-row reference of the result relation, we'll need all
+ * the columns of the relation.
+ */
+ have_wholerow = bms_is_member(0 - FirstLowInvalidHeapAttributeNumber,
+ attrs_used);
+
+ first = true;
+ i = 1;
+ foreach(lc, fdw_scan_tlist)
+ {
+ TargetEntry *tle = (TargetEntry *) lfirst(lc);
+ Var *var = (Var *) tle->expr;
+
+ if (((var->varno != rtindex) &&
+ (var->varattno >= 0 ||
+ var->varattno == SelfItemPointerAttributeNumber) &&
+ list_member(returning_vars, var)) ||
+ ((var->varno == rtindex) &&
+ (have_wholerow ||
+ bms_is_member(var->varattno - FirstLowInvalidHeapAttributeNumber,
+ attrs_used))))
+ {
+ if (!first)
+ appendStringInfoString(buf, ", ");
+ else
+ appendStringInfoString(buf, " RETURNING ");
+ first = false;
+
+ deparseVar(var, context);
+
+ *retrieved_attrs = lappend_int(*retrieved_attrs, i);
+
+ if (var->varno == rtindex)
+ {
+ *result_attrs = lappend_int(*result_attrs, i);
+ *result_attrnos = lappend_int(*result_attrnos, var->varattno);
+ }
+ }
+
+ i++;
+ }
+ }
+
+ /*
* Construct SELECT statement to acquire size in blocks of given relation.
*
* Note: we use local definition of block size, not remote definition.
*** a/contrib/postgres_fdw/expected/postgres_fdw.out
--- b/contrib/postgres_fdw/expected/postgres_fdw.out
***************
*** 2387,2413 **** UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
! QUERY PLAN
! ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
! Remote SQL: UPDATE "S 1"."T 1" SET c2 = $2, c3 = $3, c7 = $4 WHERE ctid = $1
! -> Foreign Scan
! Output: ft2.c1, (ft2.c2 + 500), NULL::integer, (ft2.c3 || '_update9'::text), ft2.c4, ft2.c5, ft2.c6, 'ft2 '::character(10), ft2.c8, ft2.ctid, ft1.*
! Relations: (public.ft2) INNER JOIN (public.ft1)
! Remote SQL: SELECT r1."C 1", r1.c2, r1.c3, r1.c4, r1.c5, r1.c6, r1.c8, r1.ctid, ROW(r2."C 1", r2.c2, r2.c3, r2.c4, r2.c5, r2.c6, r2.c7, r2.c8) FROM ("S 1"."T 1" r1 INNER JOIN "S 1"."T 1" r2 ON (TRUE)) WHERE ((r1.c2 = r2."C 1")) AND (((r2."C 1" % 10) = 9)) FOR UPDATE OF r1
! -> Hash Join
! Output: ft2.c1, ft2.c2, ft2.c3, ft2.c4, ft2.c5, ft2.c6, ft2.c8, ft2.ctid, ft1.*
! Hash Cond: (ft2.c2 = ft1.c1)
! -> Foreign Scan on public.ft2
! Output: ft2.c1, ft2.c2, ft2.c3, ft2.c4, ft2.c5, ft2.c6, ft2.c8, ft2.ctid
! Remote SQL: SELECT "C 1", c2, c3, c4, c5, c6, c8, ctid FROM "S 1"."T 1" FOR UPDATE
! -> Hash
! Output: ft1.*, ft1.c1
! -> Foreign Scan on public.ft1
! Output: ft1.*, ft1.c1
! Remote SQL: SELECT "C 1", c2, c3, c4, c5, c6, c7, c8 FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 9))
! (17 rows)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
--- 2387,2399 ----
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can be pushed down
! QUERY PLAN
! -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
! -> Foreign Update
! Remote SQL: UPDATE "S 1"."T 1" r1 SET c2 = (r1.c2 + 500), c3 = (r1.c3 || '_update9'::text), c7 = 'ft2 '::character(10) FROM "S 1"."T 1" r2 WHERE ((r1.c2 = r2."C 1")) AND (((r2."C 1" % 10) = 9))
! (3 rows)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
***************
*** 2530,2556 **** DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
! QUERY PLAN
! --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1
! -> Foreign Scan
! Output: ft2.ctid, ft1.*
! Relations: (public.ft2) INNER JOIN (public.ft1)
! Remote SQL: SELECT r1.ctid, ROW(r2."C 1", r2.c2, r2.c3, r2.c4, r2.c5, r2.c6, r2.c7, r2.c8) FROM ("S 1"."T 1" r1 INNER JOIN "S 1"."T 1" r2 ON (TRUE)) WHERE ((r1.c2 = r2."C 1")) AND (((r2."C 1" % 10) = 2)) FOR UPDATE OF r1
! -> Hash Join
! Output: ft2.ctid, ft1.*
! Hash Cond: (ft2.c2 = ft1.c1)
! -> Foreign Scan on public.ft2
! Output: ft2.ctid, ft2.c2
! Remote SQL: SELECT c2, ctid FROM "S 1"."T 1" FOR UPDATE
! -> Hash
! Output: ft1.*, ft1.c1
! -> Foreign Scan on public.ft1
! Output: ft1.*, ft1.c1
! Remote SQL: SELECT "C 1", c2, c3, c4, c5, c6, c7, c8 FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 2))
! (17 rows)
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
--- 2516,2528 ----
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can be pushed down
! QUERY PLAN
! ----------------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
! -> Foreign Delete
! Remote SQL: DELETE FROM "S 1"."T 1" r1 USING "S 1"."T 1" r2 WHERE ((r1.c2 = r2."C 1")) AND (((r2."C 1" % 10) = 2))
! (3 rows)
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
***************
*** 108,113 **** enum FdwModifyPrivateIndex
--- 108,115 ----
* 2) Boolean flag showing if the remote query has a RETURNING clause
* 3) Integer list of attribute numbers retrieved by RETURNING, if any
* 4) Boolean flag showing if we set the command es_processed
+ * 5) Integer list of the attribute numbers of result rel's columns
+ * 6) Integer list of result rel's attribute numbers of the columns
*/
enum FdwDmlPushdownPrivateIndex
{
***************
*** 118,124 **** enum FdwDmlPushdownPrivateIndex
/* Integer list of attribute numbers retrieved by RETURNING */
FdwDmlPushdownPrivateRetrievedAttrs,
/* set-processed flag (as an integer Value node) */
! FdwDmlPushdownPrivateSetProcessed
};
/*
--- 120,130 ----
/* Integer list of attribute numbers retrieved by RETURNING */
FdwDmlPushdownPrivateRetrievedAttrs,
/* set-processed flag (as an integer Value node) */
! FdwDmlPushdownPrivateSetProcessed,
! /* Integer list of the attribute numbers of result rel's columns */
! FdwDmlPushdownPrivateResultAttrs,
! /* Integer list of result rel's attribute numbers of the columns */
! FdwDmlPushdownPrivateResultAttrnos
};
/*
***************
*** 201,206 **** typedef struct PgFdwDmlPushdownState
--- 207,214 ----
bool has_returning; /* is there a RETURNING clause? */
List *retrieved_attrs; /* attr numbers retrieved by RETURNING */
bool set_processed; /* do we set the command es_processed? */
+ List *result_attrs;
+ List *result_attrnos;
/* for remote query execution */
PGconn *conn; /* connection for the update */
***************
*** 213,218 **** typedef struct PgFdwDmlPushdownState
--- 221,230 ----
PGresult *result; /* result for query */
int num_tuples; /* # of result tuples */
int next_tuple; /* index of next one to return */
+ Relation resultRel; /* relcache entry for the target table */
+ TupleTableSlot *resultSlot; /* slot for updated/deleted tuples */
+ AttrNumber *attnoMap;
+ AttrNumber ctidAttno;
/* working memory context */
MemoryContext temp_cxt; /* context for per-tuple temporary data */
***************
*** 378,385 **** static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
--- 390,404 ----
TupleTableSlot *slot);
static void store_returning_result(PgFdwModifyState *fmstate,
TupleTableSlot *slot, PGresult *res);
+ static void expand_scan_tlist(Index resultRelation,
+ Relation rel,
+ Bitmapset *attrs_used,
+ List **fdw_scan_tlist);
static void execute_dml_stmt(ForeignScanState *node);
static TupleTableSlot *get_returning_data(ForeignScanState *node);
+ static void init_returning_filter(PgFdwDmlPushdownState *dpstate);
+ static TupleTableSlot *execute_returning_filter(PgFdwDmlPushdownState *dpstate,
+ TupleTableSlot *slot);
static void prepare_query_params(PlanState *node,
List *fdw_exprs,
int numParams,
***************
*** 2069,2079 **** postgresPlanDMLPushdown(PlannerInfo *root,
--- 2088,2102 ----
Relation rel;
StringInfoData sql;
ForeignScan *fscan;
+ RelOptInfo *foreignrel;
+ Bitmapset *attrs_used = NULL;
List *targetAttrs = NIL;
List *remote_conds;
List *params_list = NIL;
List *returningList = NIL;
List *retrieved_attrs = NIL;
+ List *result_attrs = NIL;
+ List *result_attrnos = NIL;
/*
* Decide whether the table modification is pushdown-safe.
***************
*** 2100,2113 **** postgresPlanDMLPushdown(PlannerInfo *root,
return false;
/*
! * 4. We can't push down an UPDATE or DELETE on a foreign join for now.
! */
! fscan = (ForeignScan *) subplan;
! if (fscan->scan.scanrelid == 0)
! return false;
!
! /*
! * 5. We can't push down an UPDATE, if any expressions to assign to the
* target columns are unsafe to evaluate on the remote end.
*/
if (operation == CMD_UPDATE)
--- 2123,2129 ----
return false;
/*
! * 4. We can't push down an UPDATE, if any expressions to assign to the
* target columns are unsafe to evaluate on the remote end.
*/
if (operation == CMD_UPDATE)
***************
*** 2141,2146 **** postgresPlanDMLPushdown(PlannerInfo *root,
--- 2157,2164 ----
/*
* Ok, modify subplan so as to push down the command to the remote server.
*/
+ fscan = (ForeignScan *) subplan;
+
initStringInfo(&sql);
/*
***************
*** 2150,2155 **** postgresPlanDMLPushdown(PlannerInfo *root,
--- 2168,2185 ----
rel = heap_open(rte->relid, NoLock);
/*
+ * Get a rel for this foreign table or join.
+ */
+ if (fscan->scan.scanrelid == 0)
+ {
+ /* We should have a rel for this foreign join. */
+ foreignrel = find_join_rel(root, fscan->fs_relids);
+ Assert(foreignrel);
+ }
+ else
+ foreignrel = find_base_rel(root, resultRelation);
+
+ /*
* Extract the baserestrictinfo clauses that can be evaluated remotely.
*/
remote_conds = (List *) list_nth(fscan->fdw_private,
***************
*** 2159,2166 **** postgresPlanDMLPushdown(PlannerInfo *root,
--- 2189,2219 ----
* Extract the relevant RETURNING list if any.
*/
if (plan->returningLists)
+ {
returningList = (List *) list_nth(plan->returningLists, subplan_index);
+ if (foreignrel->reloptkind == RELOPT_JOINREL)
+ {
+ /*
+ * We need the attrs, non-system and system, mentioned in the local
+ * query's RETURNING list.
+ */
+ pull_varattnos((Node *) returningList, resultRelation,
+ &attrs_used);
+
+ /*
+ * For UPDATE queries, fdw_scan_tlist contains tlist entries for
+ * all attributes of the result relation through the work of the
+ * rewriter and planner, but for DELETE queries, it doesn't contain
+ * any such entriers except for ctid. So, add to it tlist entries
+ * needed for the local query's RETURNING calculation.
+ */
+ if (operation == CMD_DELETE)
+ expand_scan_tlist(resultRelation, rel, attrs_used,
+ &fscan->fdw_scan_tlist);
+ }
+ }
+
/*
* Construct the SQL command string.
*/
***************
*** 2168,2182 **** postgresPlanDMLPushdown(PlannerInfo *root,
{
case CMD_UPDATE:
deparsePushedDownUpdateSql(&sql, root, resultRelation, rel,
((Plan *) fscan)->targetlist,
targetAttrs,
remote_conds, ¶ms_list,
! returningList, &retrieved_attrs);
break;
case CMD_DELETE:
deparsePushedDownDeleteSql(&sql, root, resultRelation, rel,
remote_conds, ¶ms_list,
! returningList, &retrieved_attrs);
break;
default:
elog(ERROR, "unexpected operation: %d", (int) operation);
--- 2221,2241 ----
{
case CMD_UPDATE:
deparsePushedDownUpdateSql(&sql, root, resultRelation, rel,
+ foreignrel, fscan->fdw_scan_tlist,
+ attrs_used,
((Plan *) fscan)->targetlist,
targetAttrs,
remote_conds, ¶ms_list,
! returningList, &retrieved_attrs,
! &result_attrs, &result_attrnos);
break;
case CMD_DELETE:
deparsePushedDownDeleteSql(&sql, root, resultRelation, rel,
+ foreignrel, fscan->fdw_scan_tlist,
+ attrs_used,
remote_conds, ¶ms_list,
! returningList, &retrieved_attrs,
! &result_attrs, &result_attrnos);
break;
default:
elog(ERROR, "unexpected operation: %d", (int) operation);
***************
*** 2202,2207 **** postgresPlanDMLPushdown(PlannerInfo *root,
--- 2261,2272 ----
retrieved_attrs,
makeInteger(plan->canSetTag));
+ fscan->fdw_private = lappend(fscan->fdw_private, result_attrs);
+ fscan->fdw_private = lappend(fscan->fdw_private, result_attrnos);
+
+ if (fscan->scan.scanrelid == 0)
+ fscan->scan.plan.lefttree = NULL;
+
heap_close(rel, NoLock);
return true;
}
***************
*** 2216,2221 **** postgresBeginDMLPushdown(ForeignScanState *node, int eflags)
--- 2281,2287 ----
ForeignScan *fsplan = (ForeignScan *) node->ss.ps.plan;
EState *estate = node->ss.ps.state;
PgFdwDmlPushdownState *dpstate;
+ Index rtindex;
RangeTblEntry *rte;
Oid userid;
ForeignTable *table;
***************
*** 2238,2248 **** postgresBeginDMLPushdown(ForeignScanState *node, int eflags)
* Identify which user to do the remote access as. This should match what
* ExecCheckRTEPerms() does.
*/
! rte = rt_fetch(fsplan->scan.scanrelid, estate->es_range_table);
userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
/* Get info about foreign table. */
! dpstate->rel = node->ss.ss_currentRelation;
table = GetForeignTable(RelationGetRelid(dpstate->rel));
user = GetUserMapping(userid, table->serverid);
--- 2304,2323 ----
* Identify which user to do the remote access as. This should match what
* ExecCheckRTEPerms() does.
*/
! if (fsplan->scan.scanrelid == 0)
! rtindex = estate->es_result_relation_info->ri_RangeTableIndex;
! else
! rtindex = fsplan->scan.scanrelid;
!
! rte = rt_fetch(rtindex, estate->es_range_table);
userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
/* Get info about foreign table. */
! if (fsplan->scan.scanrelid == 0)
! dpstate->rel = ExecOpenScanRelation(estate, rtindex, eflags);
! else
! dpstate->rel = node->ss.ss_currentRelation;
!
table = GetForeignTable(RelationGetRelid(dpstate->rel));
user = GetUserMapping(userid, table->serverid);
***************
*** 2252,2257 **** postgresBeginDMLPushdown(ForeignScanState *node, int eflags)
--- 2327,2341 ----
*/
dpstate->conn = GetConnection(user, false);
+ if (fsplan->scan.scanrelid == 0)
+ {
+ /* Save info about target table. */
+ dpstate->resultRel = dpstate->rel;
+
+ /* rel should be NULL if foreign join. */
+ dpstate->rel = NULL;
+ }
+
/* Initialize state variable */
dpstate->num_tuples = -1; /* -1 means not set yet */
***************
*** 2264,2269 **** postgresBeginDMLPushdown(ForeignScanState *node, int eflags)
--- 2348,2357 ----
FdwDmlPushdownPrivateRetrievedAttrs);
dpstate->set_processed = intVal(list_nth(fsplan->fdw_private,
FdwDmlPushdownPrivateSetProcessed));
+ dpstate->result_attrs = (List *) list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateResultAttrs);
+ dpstate->result_attrnos = (List *) list_nth(fsplan->fdw_private,
+ FdwDmlPushdownPrivateResultAttrnos);
/* Create context for per-tuple temp workspace. */
dpstate->temp_cxt = AllocSetContextCreate(estate->es_query_cxt,
***************
*** 2274,2280 **** postgresBeginDMLPushdown(ForeignScanState *node, int eflags)
/* Prepare for input conversion of RETURNING results. */
if (dpstate->has_returning)
! dpstate->attinmeta = TupleDescGetAttInMetadata(RelationGetDescr(dpstate->rel));
/*
* Prepare for processing of parameters used in remote query, if any.
--- 2362,2386 ----
/* Prepare for input conversion of RETURNING results. */
if (dpstate->has_returning)
! {
! TupleDesc tupdesc;
!
! if (fsplan->scan.scanrelid == 0)
! tupdesc = node->ss.ss_ScanTupleSlot->tts_tupleDescriptor;
! else
! tupdesc = RelationGetDescr(dpstate->rel);
!
! dpstate->attinmeta = TupleDescGetAttInMetadata(tupdesc);
!
! if (fsplan->scan.scanrelid == 0)
! {
! /* Make a new slot for storing an output tuple. */
! dpstate->resultSlot = ExecInitExtraTupleSlot(estate);
!
! /* Initialize a filter to extract an updated/deleted tuple. */
! init_returning_filter(dpstate);
! }
! }
/*
* Prepare for processing of parameters used in remote query, if any.
***************
*** 2355,2360 **** postgresEndDMLPushdown(ForeignScanState *node)
--- 2461,2470 ----
ReleaseConnection(dpstate->conn);
dpstate->conn = NULL;
+ /* close the result relation. */
+ if (dpstate->resultRel)
+ ExecCloseScanRelation(dpstate->resultRel);
+
/* MemoryContext will be deleted automatically. */
}
***************
*** 3116,3121 **** store_returning_result(PgFdwModifyState *fmstate,
--- 3226,3282 ----
}
/*
+ * Add to *fdw_scan_tlist tlist entries for the columns specified in attrs_used.
+ */
+ static void
+ expand_scan_tlist(Index resultRelation,
+ Relation rel,
+ Bitmapset *attrs_used,
+ List **fdw_scan_tlist)
+ {
+ TupleDesc tupdesc = RelationGetDescr(rel);
+ bool have_wholerow;
+ int next_resno;
+ int i;
+
+ /* If there's a whole-row reference, we'll need all the columns. */
+ have_wholerow = bms_is_member(0 - FirstLowInvalidHeapAttributeNumber,
+ attrs_used);
+
+ next_resno = list_length(*fdw_scan_tlist) + 1;
+ for (i = 1; i <= tupdesc->natts; i++)
+ {
+ Form_pg_attribute attr = tupdesc->attrs[i - 1];
+
+ /* Ignore dropped attributes. */
+ if (attr->attisdropped)
+ continue;
+
+ if (have_wholerow ||
+ bms_is_member(i - FirstLowInvalidHeapAttributeNumber,
+ attrs_used))
+ {
+ Node *expr;
+ TargetEntry *tle;
+
+ expr = (Node *) makeVar(resultRelation,
+ i,
+ attr->atttypid,
+ attr->atttypmod,
+ attr->attcollation,
+ 0);
+
+ tle = makeTargetEntry((Expr *) expr,
+ next_resno++,
+ NULL,
+ false);
+
+ *fdw_scan_tlist = lappend(*fdw_scan_tlist, tle);
+ }
+ }
+ }
+
+ /*
* Execute a pushed-down UPDATE/DELETE statement.
*/
static void
***************
*** 3169,3174 **** get_returning_data(ForeignScanState *node)
--- 3330,3336 ----
EState *estate = node->ss.ps.state;
ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+ TupleTableSlot *resultSlot;
Assert(resultRelInfo->ri_projectReturning);
***************
*** 3186,3192 **** get_returning_data(ForeignScanState *node)
--- 3348,3357 ----
* UPDATE/DELETE .. RETURNING 1 for example.)
*/
if (!dpstate->has_returning)
+ {
ExecStoreAllNullTuple(slot);
+ resultSlot = slot;
+ }
else
{
/*
***************
*** 3202,3208 **** get_returning_data(ForeignScanState *node)
dpstate->rel,
dpstate->attinmeta,
dpstate->retrieved_attrs,
! NULL,
dpstate->temp_cxt);
ExecStoreTuple(newtup, slot, InvalidBuffer, false);
}
--- 3367,3373 ----
dpstate->rel,
dpstate->attinmeta,
dpstate->retrieved_attrs,
! node,
dpstate->temp_cxt);
ExecStoreTuple(newtup, slot, InvalidBuffer, false);
}
***************
*** 3213,3228 **** get_returning_data(ForeignScanState *node)
PG_RE_THROW();
}
PG_END_TRY();
}
dpstate->next_tuple++;
! /* Make slot available for evaluation of the local query RETURNING list. */
! resultRelInfo->ri_projectReturning->pi_exprContext->ecxt_scantuple = slot;
return slot;
}
/*
* Prepare for processing of parameters used in remote query.
*/
static void
--- 3378,3521 ----
PG_RE_THROW();
}
PG_END_TRY();
+
+ /* Get the updated/deleted tuple. */
+ if (dpstate->rel)
+ resultSlot = slot;
+ else
+ resultSlot = execute_returning_filter(dpstate, slot);
}
dpstate->next_tuple++;
! /*
! * Make resultSlot available to the ModifyTable node for evaluation of
! * the local query's RETURNING list.
! */
! resultRelInfo->ri_projectReturning->pi_exprContext->ecxt_scantuple = resultSlot;
return slot;
}
/*
+ * Initialize a filter to extract an updated/deleted tuple from a given
+ * fdw_scan_tlist tuple.
+ */
+ static void
+ init_returning_filter(PgFdwDmlPushdownState *dpstate)
+ {
+ TupleDesc resultTupType = RelationGetDescr(dpstate->resultRel);
+ ListCell *lc;
+ ListCell *lc2;
+
+ ExecSetSlotDescriptor(dpstate->resultSlot, resultTupType);
+
+ /*
+ * Calculate the mapping between the fdw_scan_tlist tuple's attributes and
+ * the updated/deleted tuple's attributes.
+ *
+ * The "map" is an array of the resultRel attribute numbers, i.e. one
+ * entry for every attribute of the updated/deleted tuple. The value of
+ * this entry is the attribute number of the corresponding attribute of
+ * the fdw_scan_tlist tuple. We store zero for any attributes that won't
+ * be retrieved from the remote server, marking that a NULL is needed in
+ * the output tuple. We also get the attribute number of the ctid of the
+ * updated/deleted tuple, if any.
+ */
+ dpstate->attnoMap = (AttrNumber *) palloc0(resultTupType->natts * sizeof(AttrNumber));
+
+ dpstate->ctidAttno = 0;
+
+ forboth(lc, dpstate->result_attrs, lc2, dpstate->result_attrnos)
+ {
+ int attr = lfirst_int(lc);
+ int attrno = lfirst_int(lc2);
+
+ if (attrno == SelfItemPointerAttributeNumber)
+ dpstate->ctidAttno = attr;
+ else
+ {
+ Assert(attrno > 0);
+ dpstate->attnoMap[attrno - 1] = attr;
+ }
+ }
+ }
+
+ /*
+ * Extract and return an updated/deleted tuple from a given fdw_scan_tlist
+ * tuple.
+ */
+ static TupleTableSlot *
+ execute_returning_filter(PgFdwDmlPushdownState *dpstate,
+ TupleTableSlot *slot)
+ {
+ TupleTableSlot *resultSlot = dpstate->resultSlot;
+ TupleDesc resultTupType = RelationGetDescr(dpstate->resultRel);
+ int i;
+ Datum *values;
+ bool *isnull;
+ Datum *old_values;
+ bool *old_isnull;
+
+ /*
+ * Extract all the values of the old tuple.
+ */
+ slot_getallattrs(slot);
+ old_values = slot->tts_values;
+ old_isnull = slot->tts_isnull;
+
+ /*
+ * Prepare to build a result tuple.
+ */
+ ExecClearTuple(resultSlot);
+ values = resultSlot->tts_values;
+ isnull = resultSlot->tts_isnull;
+
+ /*
+ * Transpose data into proper fields of the new tuple.
+ */
+ for (i = 0; i < resultTupType->natts; i++)
+ {
+ int j = dpstate->attnoMap[i];
+
+ if (j == 0)
+ {
+ values[i] = (Datum) 0;
+ isnull[i] = true;
+ }
+ else
+ {
+ values[i] = old_values[j - 1];
+ isnull[i] = old_isnull[j - 1];
+ }
+ }
+
+ /*
+ * Build the virtual result tuple.
+ */
+ ExecStoreVirtualTuple(resultSlot);
+
+ /*
+ * If we have a CTID to return, install it in t_self.
+ */
+ if (dpstate->ctidAttno)
+ {
+ ItemPointer ctid = NULL;
+ HeapTuple resultTup;
+
+ ctid = (ItemPointer) DatumGetPointer(old_values[dpstate->ctidAttno - 1]);
+
+ resultTup = ExecMaterializeSlot(resultSlot);
+
+ resultTup->t_self = *ctid;
+ }
+
+ /*
+ * And return the result tuple.
+ */
+ return resultSlot;
+ }
+
+ /*
* Prepare for processing of parameters used in remote query.
*/
static void
***************
*** 4258,4268 **** make_tuple_from_result_row(PGresult *res,
tupdesc = RelationGetDescr(rel);
else
{
- PgFdwScanState *fdw_sstate;
-
Assert(fsstate);
! fdw_sstate = (PgFdwScanState *) fsstate->fdw_state;
! tupdesc = fdw_sstate->tupdesc;
}
values = (Datum *) palloc0(tupdesc->natts * sizeof(Datum));
--- 4551,4558 ----
tupdesc = RelationGetDescr(rel);
else
{
Assert(fsstate);
! tupdesc = fsstate->ss.ss_ScanTupleSlot->tts_tupleDescriptor;
}
values = (Datum *) palloc0(tupdesc->natts * sizeof(Datum));
*** a/contrib/postgres_fdw/postgres_fdw.h
--- b/contrib/postgres_fdw/postgres_fdw.h
***************
*** 132,153 **** extern void deparseUpdateSql(StringInfo buf, PlannerInfo *root,
List **retrieved_attrs);
extern void deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *targetlist,
List *targetAttrs,
List *remote_conds,
List **params_list,
List *returningList,
! List **retrieved_attrs);
extern void deparseDeleteSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *returningList,
List **retrieved_attrs);
extern void deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *remote_conds,
List **params_list,
List *returningList,
! List **retrieved_attrs);
extern void deparseAnalyzeSizeSql(StringInfo buf, Relation rel);
extern void deparseAnalyzeSql(StringInfo buf, Relation rel,
List **retrieved_attrs);
--- 132,163 ----
List **retrieved_attrs);
extern void deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
+ RelOptInfo *foreignrel,
+ List *fdw_scan_tlist,
+ Bitmapset *attrs_used,
List *targetlist,
List *targetAttrs,
List *remote_conds,
List **params_list,
List *returningList,
! List **retrieved_attrs,
! List **result_attrs,
! List **result_attrnos);
extern void deparseDeleteSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *returningList,
List **retrieved_attrs);
extern void deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
+ RelOptInfo *foreignrel,
+ List *fdw_scan_tlist,
+ Bitmapset *attrs_used,
List *remote_conds,
List **params_list,
List *returningList,
! List **retrieved_attrs,
! List **result_attrs,
! List **result_attrnos);
extern void deparseAnalyzeSizeSql(StringInfo buf, Relation rel);
extern void deparseAnalyzeSql(StringInfo buf, Relation rel,
List **retrieved_attrs);
*** a/contrib/postgres_fdw/sql/postgres_fdw.sql
--- b/contrib/postgres_fdw/sql/postgres_fdw.sql
***************
*** 607,620 **** UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
EXPLAIN (verbose, costs off)
--- 607,620 ----
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can be pushed down
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can be pushed down
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
EXPLAIN (verbose, costs off)
On Tue, Feb 23, 2016 at 1:18 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
Thanks again for updating the patch and fixing the issues!
Some comments on the latest version. I haven't reviewed the
postgres_fdw changes in detail here, so this is just about the core
changes.
I see that show_plan_tlist checks whether the operation is any of
CMD_INSERT, CMD_UPDATE, or CMD_DELETE. But practically every place
else where a similar test is needed instead tests whether the
operation is *not* CMD_SELECT. I think this place should do it that
way, too.
+ resultRelInfo = mtstate->resultRelInfo;
for (i = 0; i < nplans; i++)
{
ExecAuxRowMark *aerm;
+ /*
+ * ignore subplan if the FDW pushes down the
command to the remote
+ * server; the ModifyTable won't have anything
to do except for
+ * evaluation of RETURNING expressions
+ */
+ if (resultRelInfo->ri_FdwPushdown)
+ {
+ resultRelInfo++;
+ continue;
+ }
+
subplan = mtstate->mt_plans[i]->plan;
aerm = ExecBuildAuxRowMark(erm, subplan->targetlist);
mtstate->mt_arowmarks[i] =
lappend(mtstate->mt_arowmarks[i], aerm);
+ resultRelInfo++;
}
This kind of thing creates a hazard for future people maintaining this
code. If somebody adds some code to this loop that needs to execute
even when resultRelInfo->ri_FdwPushdown is true, they have to add two
copies of it. It's much better to move the three lines of logic that
execute only in the non-pushdown case inside of if
(!resultRelInfo->ri_FdwPushdown).
This issue crops up elsewhere as well. The changes to
ExecModifyTable() have the same problem -- in that case, it might be
wise to move the code that's going to have to be indented yet another
level into a separate function. That code also says this:
+ /* No need to provide scan tuple to
ExecProcessReturning. */
+ slot = ExecProcessReturning(resultRelInfo,
NULL, planSlot);
...but, uh, why not? The comment says what the code does, but what it
should do is explain why it does it.
On a broader level, I'm not very happy with the naming this patch
uses. Here's an example:
+ <para>
+ If an FDW supports optimizing foreign table updates, it still needs to
+ provide <function>PlanDMLPushdown</>, <function>BeginDMLPushdown</>,
+ <function>IterateDMLPushdown</> and <function>EndDMLPushdown</>
+ described below.
+ </para>
"Optimizing foreign table updates" is both inaccurate (since it
doesn't only optimize updates) and so vague as to be meaningless
unless you already know what it means. The actual patch uses
terminology like "fdwPushdowns" which is just as bad. We might push a
lot of things to the foreign side -- sorts, joins, aggregates, limits
-- and this is just one of them. Worse, "pushdown" is itself
something of a term of art - will people who haven't been following
all of the mammoth, multi-hundred-email threads on this topic know
what that means? I think we need some better terminology here.
The best thing that I can come up with offhand is "bulk modify". So
we'd have PlanBulkModify, BeginBulkModify, IterateBulkModify,
EndBulkModify, ExplainBulkModify. Other suggestions welcome. The
ResultRelInfo flag could be ri_usesFDWBulkModify. The documentation
could say something like this:
Some inserts, updates, and deletes to foreign tables can be optimized
by implementing an alternate set of interfaces. The ordinary
interfaces for inserts, updates, and deletes fetch rows from the
remote server and then modify those rows one at a time. In some
cases, this row-by-row approach is necessary, but it can be
inefficient. If it is possible for the foreign server to determine
which rows should be modified without actually retrieving them, and if
there are no local triggers which would affect the operation, then it
is possible to arrange things so that the entire operation is
performed on the remote server. The interfaces described below make
this possible.
+ Begin executing a foreign table update directly on the remote server.
I think this should say "Prepare to execute a bulk modification
directly on the remote server". It shouldn't actually begin the
execution phase.
+ End the table update and release resources. It is normally not important
And I think this one should say "Clean up following a bulk
modification on the remote server". It's not actually ending the
update; the iterate method already did that.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/03/05 5:45, Robert Haas wrote:
Some comments on the latest version. I haven't reviewed the
postgres_fdw changes in detail here, so this is just about the core
changes.
Thank you for taking the time to review the patch!
I see that show_plan_tlist checks whether the operation is any of
CMD_INSERT, CMD_UPDATE, or CMD_DELETE. But practically every place
else where a similar test is needed instead tests whether the
operation is *not* CMD_SELECT. I think this place should do it that
way, too.+ resultRelInfo = mtstate->resultRelInfo;
for (i = 0; i < nplans; i++)
{
ExecAuxRowMark *aerm;+ /* + * ignore subplan if the FDW pushes down the command to the remote + * server; the ModifyTable won't have anything to do except for + * evaluation of RETURNING expressions + */ + if (resultRelInfo->ri_FdwPushdown) + { + resultRelInfo++; + continue; + } + subplan = mtstate->mt_plans[i]->plan; aerm = ExecBuildAuxRowMark(erm, subplan->targetlist); mtstate->mt_arowmarks[i] = lappend(mtstate->mt_arowmarks[i], aerm); + resultRelInfo++; }This kind of thing creates a hazard for future people maintaining this
code. If somebody adds some code to this loop that needs to execute
even when resultRelInfo->ri_FdwPushdown is true, they have to add two
copies of it. It's much better to move the three lines of logic that
execute only in the non-pushdown case inside of if
(!resultRelInfo->ri_FdwPushdown).
Another option to avoid such a hazard would be to remove the two changes
from ExecInitModifyTable and create ExecAuxRowMarks and junk filters
even in the pushdown case. I made the changes because we won't use
ExecAuxRowMarks in that case since we don't need to do EvalPlanQual
rechecks and because we won't use junk filters in that case since we do
UPDATE/DELETE in the subplan. But the creating cost is enough small, so
simply removing the changes seems like a good idea.
This issue crops up elsewhere as well. The changes to
ExecModifyTable() have the same problem -- in that case, it might be
wise to move the code that's going to have to be indented yet another
level into a separate function. That code also says this:+ /* No need to provide scan tuple to ExecProcessReturning. */ + slot = ExecProcessReturning(resultRelInfo, NULL, planSlot);...but, uh, why not? The comment says what the code does, but what it
should do is explain why it does it.
As documented in IterateDMLPushdown in fdwhandler.sgml, the reason for
that is that in the pushdown case it's the IterateDMLPushdown's
responsiblity to get actually inserted/updated/deleted tuples and make
those tuples available to the ExecProcessReturning. I'll add comments.
On a broader level, I'm not very happy with the naming this patch
uses. Here's an example:+ <para> + If an FDW supports optimizing foreign table updates, it still needs to + provide <function>PlanDMLPushdown</>, <function>BeginDMLPushdown</>, + <function>IterateDMLPushdown</> and <function>EndDMLPushdown</> + described below. + </para>"Optimizing foreign table updates" is both inaccurate (since it
doesn't only optimize updates) and so vague as to be meaningless
unless you already know what it means. The actual patch uses
terminology like "fdwPushdowns" which is just as bad. We might push a
lot of things to the foreign side -- sorts, joins, aggregates, limits
-- and this is just one of them. Worse, "pushdown" is itself
something of a term of art - will people who haven't been following
all of the mammoth, multi-hundred-email threads on this topic know
what that means? I think we need some better terminology here.The best thing that I can come up with offhand is "bulk modify". So
we'd have PlanBulkModify, BeginBulkModify, IterateBulkModify,
EndBulkModify, ExplainBulkModify. Other suggestions welcome. The
ResultRelInfo flag could be ri_usesFDWBulkModify.
I'm not sure that "bulk modify" is best. Yeah, this would improve the
performance especially in the bulk-modification case, but would improve
the performance even in the case where an UPDATE/DELETE modifies just a
single row. Let me explain using an example. Without the patch, we
have the following plan for an UPDATE on a foreign table that updates a
single row:
postgres=# explain verbose update foo set a = a + 1 where a = 1;
QUERY PLAN
----------------------------------------------------------------------------------
Update on public.foo (cost=100.00..101.05 rows=1 width=14)
Remote SQL: UPDATE public.foo SET a = $2 WHERE ctid = $1
-> Foreign Scan on public.foo (cost=100.00..101.05 rows=1 width=14)
Output: (a + 1), b, ctid
Remote SQL: SELECT a, b, ctid FROM public.foo WHERE ((a = 1))
FOR UPDATE
(5 rows)
The plan requires two queries, SELECT and UPDATE, to do the update.
(Actually, the plan have additional overheads in creating a cursor for
the SELECT and establishing a prepared statement for the UPDATE.) But
with the patch, we have:
postgres=# explain verbose update foo set a = a + 1 where a = 1;
QUERY PLAN
---------------------------------------------------------------------------
Update on public.foo (cost=100.00..101.05 rows=1 width=14)
-> Foreign Update on public.foo (cost=100.00..101.05 rows=1 width=14)
Remote SQL: UPDATE public.foo SET a = (a + 1) WHERE ((a = 1))
(3 rows)
The optimized plan requires just a single UPDATE query to do that! So,
even in the single-row-modification case the patch could improve the
performance.
How about "Direct Modify"; PlanDirectModify, BeginDirectModify,
IterateDirectModify, EndDirectModify, ExplainDirectModify, and
ri_usesFDWDirectModify.
The documentation
could say something like this:Some inserts, updates, and deletes to foreign tables can be optimized
by implementing an alternate set of interfaces. The ordinary
interfaces for inserts, updates, and deletes fetch rows from the
remote server and then modify those rows one at a time. In some
cases, this row-by-row approach is necessary, but it can be
inefficient. If it is possible for the foreign server to determine
which rows should be modified without actually retrieving them, and if
there are no local triggers which would affect the operation, then it
is possible to arrange things so that the entire operation is
performed on the remote server. The interfaces described below make
this possible.
Will update as proposed.
+ Begin executing a foreign table update directly on the remote server.
I think this should say "Prepare to execute a bulk modification
directly on the remote server". It shouldn't actually begin the
execution phase.+ End the table update and release resources. It is normally not important
And I think this one should say "Clean up following a bulk
modification on the remote server". It's not actually ending the
update; the iterate method already did that.
OK, will fix.
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Mar 7, 2016 at 7:53 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
Another option to avoid such a hazard would be to remove the two changes
from ExecInitModifyTable and create ExecAuxRowMarks and junk filters even in
the pushdown case. I made the changes because we won't use ExecAuxRowMarks
in that case since we don't need to do EvalPlanQual rechecks and because we
won't use junk filters in that case since we do UPDATE/DELETE in the
subplan. But the creating cost is enough small, so simply removing the
changes seems like a good idea.
Sure, that works.
This issue crops up elsewhere as well. The changes to
ExecModifyTable() have the same problem -- in that case, it might be
wise to move the code that's going to have to be indented yet another
level into a separate function. That code also says this:+ /* No need to provide scan tuple to ExecProcessReturning. */ + slot = ExecProcessReturning(resultRelInfo, NULL, planSlot);...but, uh, why not? The comment says what the code does, but what it
should do is explain why it does it.As documented in IterateDMLPushdown in fdwhandler.sgml, the reason for that
is that in the pushdown case it's the IterateDMLPushdown's responsiblity to
get actually inserted/updated/deleted tuples and make those tuples available
to the ExecProcessReturning. I'll add comments.
Comments are good things to have. :-)
On a broader level, I'm not very happy with the naming this patch
uses. Here's an example:+ <para> + If an FDW supports optimizing foreign table updates, it still needs to + provide <function>PlanDMLPushdown</>, <function>BeginDMLPushdown</>, + <function>IterateDMLPushdown</> and <function>EndDMLPushdown</> + described below. + </para>"Optimizing foreign table updates" is both inaccurate (since it
doesn't only optimize updates) and so vague as to be meaningless
unless you already know what it means. The actual patch uses
terminology like "fdwPushdowns" which is just as bad. We might push a
lot of things to the foreign side -- sorts, joins, aggregates, limits
-- and this is just one of them. Worse, "pushdown" is itself
something of a term of art - will people who haven't been following
all of the mammoth, multi-hundred-email threads on this topic know
what that means? I think we need some better terminology here.The best thing that I can come up with offhand is "bulk modify". So
we'd have PlanBulkModify, BeginBulkModify, IterateBulkModify,
EndBulkModify, ExplainBulkModify. Other suggestions welcome. The
ResultRelInfo flag could be ri_usesFDWBulkModify.I'm not sure that "bulk modify" is best. Yeah, this would improve the
performance especially in the bulk-modification case, but would improve the
performance even in the case where an UPDATE/DELETE modifies just a single
row. Let me explain using an example. Without the patch, we have the
following plan for an UPDATE on a foreign table that updates a single row:postgres=# explain verbose update foo set a = a + 1 where a = 1;
QUERY PLAN
----------------------------------------------------------------------------------
Update on public.foo (cost=100.00..101.05 rows=1 width=14)
Remote SQL: UPDATE public.foo SET a = $2 WHERE ctid = $1
-> Foreign Scan on public.foo (cost=100.00..101.05 rows=1 width=14)
Output: (a + 1), b, ctid
Remote SQL: SELECT a, b, ctid FROM public.foo WHERE ((a = 1)) FOR
UPDATE
(5 rows)The plan requires two queries, SELECT and UPDATE, to do the update.
(Actually, the plan have additional overheads in creating a cursor for the
SELECT and establishing a prepared statement for the UPDATE.) But with the
patch, we have:postgres=# explain verbose update foo set a = a + 1 where a = 1;
QUERY PLAN
---------------------------------------------------------------------------
Update on public.foo (cost=100.00..101.05 rows=1 width=14)
-> Foreign Update on public.foo (cost=100.00..101.05 rows=1 width=14)
Remote SQL: UPDATE public.foo SET a = (a + 1) WHERE ((a = 1))
(3 rows)The optimized plan requires just a single UPDATE query to do that! So, even
in the single-row-modification case the patch could improve the performance.How about "Direct Modify"; PlanDirectModify, BeginDirectModify,
IterateDirectModify, EndDirectModify, ExplainDirectModify, and
ri_usesFDWDirectModify.
Works for me!
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/03/08 2:35, Robert Haas wrote:
On Mon, Mar 7, 2016 at 7:53 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:Another option to avoid such a hazard would be to remove the two changes
from ExecInitModifyTable and create ExecAuxRowMarks and junk filters even in
the pushdown case. I made the changes because we won't use ExecAuxRowMarks
in that case since we don't need to do EvalPlanQual rechecks and because we
won't use junk filters in that case since we do UPDATE/DELETE in the
subplan. But the creating cost is enough small, so simply removing the
changes seems like a good idea.
Sure, that works.
OK, I removed the changes.
This issue crops up elsewhere as well. The changes to
ExecModifyTable() have the same problem -- in that case, it might be
wise to move the code that's going to have to be indented yet another
level into a separate function. That code also says this:+ /* No need to provide scan tuple to ExecProcessReturning. */ + slot = ExecProcessReturning(resultRelInfo, NULL, planSlot);...but, uh, why not? The comment says what the code does, but what it
should do is explain why it does it.As documented in IterateDMLPushdown in fdwhandler.sgml, the reason for that
is that in the pushdown case it's the IterateDMLPushdown's responsiblity to
get actually inserted/updated/deleted tuples and make those tuples available
to the ExecProcessReturning. I'll add comments.
Comments are good things to have. :-)
Yeah, I added comments.
On a broader level, I'm not very happy with the naming this patch
uses. Here's an example:+ <para> + If an FDW supports optimizing foreign table updates, it still needs to + provide <function>PlanDMLPushdown</>, <function>BeginDMLPushdown</>, + <function>IterateDMLPushdown</> and <function>EndDMLPushdown</> + described below. + </para>"Optimizing foreign table updates" is both inaccurate (since it
doesn't only optimize updates) and so vague as to be meaningless
unless you already know what it means. The actual patch uses
terminology like "fdwPushdowns" which is just as bad. We might push a
lot of things to the foreign side -- sorts, joins, aggregates, limits
-- and this is just one of them. Worse, "pushdown" is itself
something of a term of art - will people who haven't been following
all of the mammoth, multi-hundred-email threads on this topic know
what that means? I think we need some better terminology here.The best thing that I can come up with offhand is "bulk modify". So
we'd have PlanBulkModify, BeginBulkModify, IterateBulkModify,
EndBulkModify, ExplainBulkModify. Other suggestions welcome. The
ResultRelInfo flag could be ri_usesFDWBulkModify.I'm not sure that "bulk modify" is best. Yeah, this would improve the
performance especially in the bulk-modification case, but would improve the
performance even in the case where an UPDATE/DELETE modifies just a single
row. Let me explain using an example. Without the patch, we have the
following plan for an UPDATE on a foreign table that updates a single row:postgres=# explain verbose update foo set a = a + 1 where a = 1;
QUERY PLAN
----------------------------------------------------------------------------------
Update on public.foo (cost=100.00..101.05 rows=1 width=14)
Remote SQL: UPDATE public.foo SET a = $2 WHERE ctid = $1
-> Foreign Scan on public.foo (cost=100.00..101.05 rows=1 width=14)
Output: (a + 1), b, ctid
Remote SQL: SELECT a, b, ctid FROM public.foo WHERE ((a = 1)) FOR
UPDATE
(5 rows)The plan requires two queries, SELECT and UPDATE, to do the update.
(Actually, the plan have additional overheads in creating a cursor for the
SELECT and establishing a prepared statement for the UPDATE.) But with the
patch, we have:postgres=# explain verbose update foo set a = a + 1 where a = 1;
QUERY PLAN
---------------------------------------------------------------------------
Update on public.foo (cost=100.00..101.05 rows=1 width=14)
-> Foreign Update on public.foo (cost=100.00..101.05 rows=1 width=14)
Remote SQL: UPDATE public.foo SET a = (a + 1) WHERE ((a = 1))
(3 rows)The optimized plan requires just a single UPDATE query to do that! So, even
in the single-row-modification case the patch could improve the performance.How about "Direct Modify"; PlanDirectModify, BeginDirectModify,
IterateDirectModify, EndDirectModify, ExplainDirectModify, and
ri_usesFDWDirectModify.
Works for me!
Great! I changed the naming. I also updated docs as proposed by you in
a previous email, and rebased the patch to the latest HEAD. Please find
attached an updated version of the patch.
Best regards,
Etsuro Fujita
Attachments:
fdw-dml-pushdown-v10.patchbinary/octet-stream; name=fdw-dml-pushdown-v10.patchDownload
*** a/contrib/postgres_fdw/deparse.c
--- b/contrib/postgres_fdw/deparse.c
***************
*** 1316,1321 **** deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 1316,1384 ----
}
/*
+ * deparse remote UPDATE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ deparse_expr_cxt context;
+ int nestlevel;
+ bool first;
+ ListCell *lc;
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = params_list;
+
+ appendStringInfoString(buf, "UPDATE ");
+ deparseRelation(buf, rel);
+ appendStringInfoString(buf, " SET ");
+
+ /* Make sure any constants in the exprs are printed portably */
+ nestlevel = set_transmission_modes();
+
+ first = true;
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(targetlist, attnum);
+
+ if (!first)
+ appendStringInfoString(buf, ", ");
+ first = false;
+
+ deparseColumnRef(buf, rtindex, attnum, root, false);
+ appendStringInfoString(buf, " = ");
+ deparseExpr((Expr *) tle->expr, &context);
+ }
+
+ reset_transmission_modes(nestlevel);
+
+ if (remote_conds)
+ {
+ appendStringInfo(buf, " WHERE ");
+ appendConditions(remote_conds, &context);
+ }
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* deparse remote DELETE statement
*
* The statement text is appended to buf, and we also create an integer List
***************
*** 1338,1343 **** deparseDeleteSql(StringInfo buf, PlannerInfo *root,
--- 1401,1443 ----
}
/*
+ * deparse remote DELETE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ deparse_expr_cxt context;
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = params_list;
+
+ appendStringInfoString(buf, "DELETE FROM ");
+ deparseRelation(buf, rel);
+
+ if (remote_conds)
+ {
+ appendStringInfo(buf, " WHERE ");
+ appendConditions(remote_conds, &context);
+ }
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* Add a RETURNING clause, if needed, to an INSERT/UPDATE/DELETE.
*/
static void
*** a/contrib/postgres_fdw/expected/postgres_fdw.out
--- b/contrib/postgres_fdw/expected/postgres_fdw.out
***************
*** 2258,2264 **** INSERT INTO ft2 (c1,c2,c3)
--- 2258,2283 ----
(3 rows)
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 300), c3 = (c3 || '_update3'::text) WHERE ((("C 1" % 10) = 3))
+ (3 rows)
+
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Output: c1, c2, c3, c4, c5, c6, c7, c8
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 400), c3 = (c3 || '_update7'::text) WHERE ((("C 1" % 10) = 7)) RETURNING "C 1", c2, c3, c4, c5, c6, c7, c8
+ (4 rows)
+
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
c1 | c2 | c3 | c4 | c5 | c6 | c7 | c8
------+-----+--------------------+------------------------------+--------------------------+----+------------+-----
***************
*** 2368,2374 **** UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
--- 2387,2393 ----
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
***************
*** 2393,2408 **** UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
! QUERY PLAN
! ----------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1 RETURNING "C 1", c4
! -> Foreign Scan on public.ft2
! Output: ctid
! Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) FOR UPDATE
! (6 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
--- 2412,2425 ----
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
! QUERY PLAN
! --------------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
! -> Foreign Delete on public.ft2
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) RETURNING "C 1", c4
! (4 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
***************
*** 2513,2519 **** DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
--- 2530,2536 ----
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
***************
*** 3378,3393 **** INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
(1 row)
EXPLAIN (verbose, costs off)
! UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
! QUERY PLAN
! -------------------------------------------------------------------------------------------------------------------
Update on public.ft2
Output: (tableoid)::regclass
! Remote SQL: UPDATE "S 1"."T 1" SET c3 = $2 WHERE ctid = $1
! -> Foreign Scan on public.ft2
! Output: c1, c2, NULL::integer, 'bar'::text, c4, c5, c6, c7, c8, ctid
! Remote SQL: SELECT "C 1", c2, c4, c5, c6, c7, c8, ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
! (6 rows)
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
--- 3395,3408 ----
(1 row)
EXPLAIN (verbose, costs off)
! UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
! QUERY PLAN
! ------------------------------------------------------------------------------------
Update on public.ft2
Output: (tableoid)::regclass
! -> Foreign Update on public.ft2
! Remote SQL: UPDATE "S 1"."T 1" SET c3 = 'bar'::text WHERE (("C 1" = 9999))
! (4 rows)
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
***************
*** 3396,3411 **** UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
(1 row)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
! QUERY PLAN
! ------------------------------------------------------------------------------------
Delete on public.ft2
Output: (tableoid)::regclass
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1
! -> Foreign Scan on public.ft2
! Output: ctid
! Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
! (6 rows)
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
--- 3411,3424 ----
(1 row)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
! QUERY PLAN
! --------------------------------------------------------------------
Delete on public.ft2
Output: (tableoid)::regclass
! -> Foreign Delete on public.ft2
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE (("C 1" = 9999))
! (4 rows)
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
***************
*** 3559,3565 **** CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 3572,3578 ----
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 3718,3724 **** savepoint s3;
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 3731,3737 ----
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (-2) WHERE ((c2 = 42)) AND (("C 1" = 10))
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 3938,3944 **** CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2positive;
-- But inconsistent check constraints provide inconsistent results
ALTER FOREIGN TABLE ft1 ADD CONSTRAINT ft1_c2negative CHECK (c2 < 0);
--- 3951,3957 ----
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2positive;
-- But inconsistent check constraints provide inconsistent results
ALTER FOREIGN TABLE ft1 ADD CONSTRAINT ft1_c2negative CHECK (c2 < 0);
***************
*** 4331,4336 **** NOTICE: NEW: (13,"test triggered !")
--- 4344,4542 ----
(0,27)
(1 row)
+ -- cleanup
+ DROP TRIGGER trig_row_before ON rem1;
+ DROP TRIGGER trig_row_after ON rem1;
+ DROP TRIGGER trig_local_before ON loc1;
+ -- Test direct foreign table modification functionality
+ -- Test with statement-level triggers
+ CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_stmt_before ON rem1;
+ CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_stmt_after ON rem1;
+ -- Test with row-level ON INSERT triggers
+ CREATE TRIGGER trig_row_before_insert
+ BEFORE INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_before_insert ON rem1;
+ CREATE TRIGGER trig_row_after_insert
+ AFTER INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_after_insert ON rem1;
+ -- Test with row-level ON UPDATE triggers
+ CREATE TRIGGER trig_row_before_update
+ BEFORE UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_before_update ON rem1;
+ CREATE TRIGGER trig_row_after_update
+ AFTER UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+ -------------------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_after_update ON rem1;
+ -- Test with row-level ON DELETE triggers
+ CREATE TRIGGER trig_row_before_delete
+ BEFORE DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ DROP TRIGGER trig_row_before_delete ON rem1;
+ CREATE TRIGGER trig_row_after_delete
+ AFTER DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+ ------------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ DROP TRIGGER trig_row_after_delete ON rem1;
-- ===================================================================
-- test inheritance features
-- ===================================================================
***************
*** 4800,4805 **** fetch from c;
--- 5006,5061 ----
update bar set f2 = null where current of c;
ERROR: WHERE CURRENT OF is not supported for this table type
rollback;
+ explain (verbose, costs off)
+ delete from foo where f1 < 5 returning *;
+ QUERY PLAN
+ --------------------------------------------------------------------------------
+ Delete on public.foo
+ Output: foo.f1, foo.f2
+ Delete on public.foo
+ Foreign Delete on public.foo2
+ -> Index Scan using i_foo_f1 on public.foo
+ Output: foo.ctid
+ Index Cond: (foo.f1 < 5)
+ -> Foreign Delete on public.foo2
+ Remote SQL: DELETE FROM public.loct1 WHERE ((f1 < 5)) RETURNING f1, f2
+ (9 rows)
+
+ delete from foo where f1 < 5 returning *;
+ f1 | f2
+ ----+----
+ 1 | 1
+ 3 | 3
+ 0 | 0
+ 2 | 2
+ 4 | 4
+ (5 rows)
+
+ explain (verbose, costs off)
+ update bar set f2 = f2 + 100 returning *;
+ QUERY PLAN
+ ------------------------------------------------------------------------------
+ Update on public.bar
+ Output: bar.f1, bar.f2
+ Update on public.bar
+ Foreign Update on public.bar2
+ -> Seq Scan on public.bar
+ Output: bar.f1, (bar.f2 + 100), bar.ctid
+ -> Foreign Update on public.bar2
+ Remote SQL: UPDATE public.loct2 SET f2 = (f2 + 100) RETURNING f1, f2
+ (8 rows)
+
+ update bar set f2 = f2 + 100 returning *;
+ f1 | f2
+ ----+-----
+ 1 | 311
+ 2 | 322
+ 6 | 266
+ 3 | 333
+ 4 | 344
+ 7 | 277
+ (6 rows)
+
drop table foo cascade;
NOTICE: drop cascades to foreign table foo2
drop table bar cascade;
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
***************
*** 61,66 **** enum FdwScanPrivateIndex
--- 61,68 ----
{
/* SQL statement to execute remotely (as a String node) */
FdwScanPrivateSelectSql,
+ /* List of restriction clauses that can be executed remotely */
+ FdwScanPrivateRemoteConds,
/* Integer list of attribute numbers retrieved by the SELECT */
FdwScanPrivateRetrievedAttrs,
/* Integer representing the desired fetch_size */
***************
*** 98,103 **** enum FdwModifyPrivateIndex
--- 100,126 ----
};
/*
+ * Similarly, this enum describes what's kept in the fdw_private list for
+ * a ForeignScan node that modifies a foreign table directly. We store:
+ *
+ * 1) UPDATE/DELETE statement text to be sent to the remote server
+ * 2) Boolean flag showing if the remote query has a RETURNING clause
+ * 3) Integer list of attribute numbers retrieved by RETURNING, if any
+ * 4) Boolean flag showing if we set the command es_processed
+ */
+ enum FdwDirectModifyPrivateIndex
+ {
+ /* SQL statement to execute remotely (as a String node) */
+ FdwDirectModifyPrivateUpdateSql,
+ /* has-returning flag (as an integer Value node) */
+ FdwDirectModifyPrivateHasReturning,
+ /* Integer list of attribute numbers retrieved by RETURNING */
+ FdwDirectModifyPrivateRetrievedAttrs,
+ /* set-processed flag (as an integer Value node) */
+ FdwDirectModifyPrivateSetProcessed
+ };
+
+ /*
* Execution state of a foreign scan using postgres_fdw.
*/
typedef struct PgFdwScanState
***************
*** 164,169 **** typedef struct PgFdwModifyState
--- 187,222 ----
} PgFdwModifyState;
/*
+ * Execution state of a foreign scan that modifies a foreign table directly.
+ */
+ typedef struct PgFdwDirectModifyState
+ {
+ Relation rel; /* relcache entry for the foreign table */
+ AttInMetadata *attinmeta; /* attribute datatype conversion metadata */
+
+ /* extracted fdw_private data */
+ char *query; /* text of UPDATE/DELETE command */
+ bool has_returning; /* is there a RETURNING clause? */
+ List *retrieved_attrs; /* attr numbers retrieved by RETURNING */
+ bool set_processed; /* do we set the command es_processed? */
+
+ /* for remote query execution */
+ PGconn *conn; /* connection for the update */
+ int numParams; /* number of parameters passed to query */
+ FmgrInfo *param_flinfo; /* output conversion functions for them */
+ List *param_exprs; /* executable expressions for param values */
+ const char **param_values; /* textual values of query parameters */
+
+ /* for storing result tuples */
+ PGresult *result; /* result for query */
+ int num_tuples; /* # of result tuples */
+ int next_tuple; /* index of next one to return */
+
+ /* working memory context */
+ MemoryContext temp_cxt; /* context for per-tuple temporary data */
+ } PgFdwDirectModifyState;
+
+ /*
* Workspace for analyzing a foreign table.
*/
typedef struct PgFdwAnalyzeState
***************
*** 263,268 **** static TupleTableSlot *postgresExecForeignDelete(EState *estate,
--- 316,328 ----
static void postgresEndForeignModify(EState *estate,
ResultRelInfo *resultRelInfo);
static int postgresIsForeignRelUpdatable(Relation rel);
+ static bool postgresPlanDirectModify(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+ static void postgresBeginDirectModify(ForeignScanState *node, int eflags);
+ static TupleTableSlot *postgresIterateDirectModify(ForeignScanState *node);
+ static void postgresEndDirectModify(ForeignScanState *node);
static void postgresExplainForeignScan(ForeignScanState *node,
ExplainState *es);
static void postgresExplainForeignModify(ModifyTableState *mtstate,
***************
*** 270,275 **** static void postgresExplainForeignModify(ModifyTableState *mtstate,
--- 330,337 ----
List *fdw_private,
int subplan_index,
ExplainState *es);
+ static void postgresExplainDirectModify(ForeignScanState *node,
+ ExplainState *es);
static bool postgresAnalyzeForeignTable(Relation relation,
AcquireSampleRowsFunc *func,
BlockNumber *totalpages);
***************
*** 314,319 **** static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
--- 376,393 ----
TupleTableSlot *slot);
static void store_returning_result(PgFdwModifyState *fmstate,
TupleTableSlot *slot, PGresult *res);
+ static void execute_dml_stmt(ForeignScanState *node);
+ static TupleTableSlot *get_returning_data(ForeignScanState *node);
+ static void prepare_query_params(PlanState *node,
+ List *fdw_exprs,
+ int numParams,
+ FmgrInfo **param_flinfo,
+ List **param_exprs,
+ const char ***param_values);
+ static void process_query_params(ExprContext *econtext,
+ FmgrInfo *param_flinfo,
+ List *param_exprs,
+ const char **param_values);
static int postgresAcquireSampleRowsFunc(Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
***************
*** 360,371 **** postgres_fdw_handler(PG_FUNCTION_ARGS)
--- 434,450 ----
routine->ExecForeignDelete = postgresExecForeignDelete;
routine->EndForeignModify = postgresEndForeignModify;
routine->IsForeignRelUpdatable = postgresIsForeignRelUpdatable;
+ routine->PlanDirectModify = postgresPlanDirectModify;
+ routine->BeginDirectModify = postgresBeginDirectModify;
+ routine->IterateDirectModify = postgresIterateDirectModify;
+ routine->EndDirectModify = postgresEndDirectModify;
/* Function for EvalPlanQual rechecks */
routine->RecheckForeignScan = postgresRecheckForeignScan;
/* Support functions for EXPLAIN */
routine->ExplainForeignScan = postgresExplainForeignScan;
routine->ExplainForeignModify = postgresExplainForeignModify;
+ routine->ExplainDirectModify = postgresExplainDirectModify;
/* Support functions for ANALYZE */
routine->AnalyzeForeignTable = postgresAnalyzeForeignTable;
***************
*** 1133,1139 **** postgresGetForeignPlan(PlannerInfo *root,
* Build the fdw_private list that will be available to the executor.
* Items in the list must match order in enum FdwScanPrivateIndex.
*/
! fdw_private = list_make4(makeString(sql.data),
retrieved_attrs,
makeInteger(fpinfo->fetch_size),
makeInteger(foreignrel->umid));
--- 1212,1219 ----
* Build the fdw_private list that will be available to the executor.
* Items in the list must match order in enum FdwScanPrivateIndex.
*/
! fdw_private = list_make5(makeString(sql.data),
! remote_conds,
retrieved_attrs,
makeInteger(fpinfo->fetch_size),
makeInteger(foreignrel->umid));
***************
*** 1170,1177 **** postgresBeginForeignScan(ForeignScanState *node, int eflags)
PgFdwScanState *fsstate;
UserMapping *user;
int numParams;
- int i;
- ListCell *lc;
/*
* Do nothing in EXPLAIN (no ANALYZE) case. node->fdw_state stays NULL.
--- 1250,1255 ----
***************
*** 1258,1299 **** postgresBeginForeignScan(ForeignScanState *node, int eflags)
fsstate->attinmeta = TupleDescGetAttInMetadata(fsstate->tupdesc);
- /* Prepare for output conversion of parameters used in remote query. */
- numParams = list_length(fsplan->fdw_exprs);
- fsstate->numParams = numParams;
- fsstate->param_flinfo = (FmgrInfo *) palloc0(sizeof(FmgrInfo) * numParams);
-
- i = 0;
- foreach(lc, fsplan->fdw_exprs)
- {
- Node *param_expr = (Node *) lfirst(lc);
- Oid typefnoid;
- bool isvarlena;
-
- getTypeOutputInfo(exprType(param_expr), &typefnoid, &isvarlena);
- fmgr_info(typefnoid, &fsstate->param_flinfo[i]);
- i++;
- }
-
/*
! * Prepare remote-parameter expressions for evaluation. (Note: in
! * practice, we expect that all these expressions will be just Params, so
! * we could possibly do something more efficient than using the full
! * expression-eval machinery for this. But probably there would be little
! * benefit, and it'd require postgres_fdw to know more than is desirable
! * about Param evaluation.)
! */
! fsstate->param_exprs = (List *)
! ExecInitExpr((Expr *) fsplan->fdw_exprs,
! (PlanState *) node);
!
! /*
! * Allocate buffer for text form of query parameters, if any.
*/
if (numParams > 0)
! fsstate->param_values = (const char **) palloc0(numParams * sizeof(char *));
! else
! fsstate->param_values = NULL;
}
/*
--- 1336,1353 ----
fsstate->attinmeta = TupleDescGetAttInMetadata(fsstate->tupdesc);
/*
! * Prepare for processing of parameters used in remote query, if any.
*/
+ numParams = list_length(fsplan->fdw_exprs);
+ fsstate->numParams = numParams;
if (numParams > 0)
! prepare_query_params((PlanState *) node,
! fsplan->fdw_exprs,
! numParams,
! &fsstate->param_flinfo,
! &fsstate->param_exprs,
! &fsstate->param_values);
}
/*
***************
*** 1458,1470 **** postgresAddForeignUpdateTargets(Query *parsetree,
/*
* postgresPlanForeignModify
* Plan an insert/update/delete operation on a foreign table
- *
- * Note: currently, the plan tree generated for UPDATE/DELETE will always
- * include a ForeignScan that retrieves ctids (using SELECT FOR UPDATE)
- * and then the ModifyTable node will have to execute individual remote
- * UPDATE/DELETE commands. If there are no local conditions or joins
- * needed, it'd be better to let the scan node do UPDATE/DELETE RETURNING
- * and then do nothing at ModifyTable. Room for future optimization ...
*/
static List *
postgresPlanForeignModify(PlannerInfo *root,
--- 1512,1517 ----
***************
*** 2003,2008 **** postgresRecheckForeignScan(ForeignScanState *node, TupleTableSlot *slot)
--- 2050,2363 ----
}
/*
+ * postgresPlanDirectModify
+ * Consider a direct foreign table modification
+ *
+ * Decide whether it is safe to modify a foreign table directly, and if so,
+ * rewrite subplan accordingly.
+ */
+ static bool
+ postgresPlanDirectModify(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index)
+ {
+ CmdType operation = plan->operation;
+ Plan *subplan = (Plan *) list_nth(plan->plans, subplan_index);
+ RangeTblEntry *rte = planner_rt_fetch(resultRelation, root);
+ Relation rel;
+ StringInfoData sql;
+ ForeignScan *fscan;
+ List *targetAttrs = NIL;
+ List *remote_conds;
+ List *params_list = NIL;
+ List *returningList = NIL;
+ List *retrieved_attrs = NIL;
+
+ /*
+ * Decide whether it is safe to modify a foreign table directly.
+ */
+
+ /*
+ * 1. The table modification must be an UPDATE or DELETE.
+ */
+ if (operation != CMD_UPDATE && operation != CMD_DELETE)
+ return false;
+
+ /*
+ * 2. It's unsafe to modify a foreign table directly if there are any
+ * local joins needed.
+ */
+ if (!IsA(subplan, ForeignScan))
+ return false;
+
+ /*
+ * 3. It's unsafe to modify a foreign table directly if there are any
+ * quals that should be evaluated locally.
+ */
+ if (subplan->qual != NIL)
+ return false;
+
+ /*
+ * 4. We can't handle an UPDATE or DELETE on a foreign join for now.
+ */
+ fscan = (ForeignScan *) subplan;
+ if (fscan->scan.scanrelid == 0)
+ return false;
+
+ /*
+ * 5. It's unsafe to update a foreign table directly, if any expressions
+ * to assign to the target columns are unsafe to evaluate remotely.
+ */
+ if (operation == CMD_UPDATE)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[resultRelation];
+ int col;
+
+ /*
+ * We transmit only columns that were explicitly targets of the UPDATE,
+ * so as to avoid unnecessary data transmission.
+ */
+ col = -1;
+ while ((col = bms_next_member(rte->updatedCols, col)) >= 0)
+ {
+ /* bit numbers are offset by FirstLowInvalidHeapAttributeNumber */
+ AttrNumber attno = col + FirstLowInvalidHeapAttributeNumber;
+ TargetEntry *tle;
+
+ if (attno <= InvalidAttrNumber) /* shouldn't happen */
+ elog(ERROR, "system-column update is not supported");
+
+ tle = get_tle_by_resno(subplan->targetlist, attno);
+
+ if (!is_foreign_expr(root, baserel, (Expr *) tle->expr))
+ return false;
+
+ targetAttrs = lappend_int(targetAttrs, attno);
+ }
+ }
+
+ /*
+ * Ok, rewrite subplan so as to modify the foreign table directly.
+ */
+ initStringInfo(&sql);
+
+ /*
+ * Core code already has some lock on each rel being planned, so we can
+ * use NoLock here.
+ */
+ rel = heap_open(rte->relid, NoLock);
+
+ /*
+ * Extract the baserestrictinfo clauses that can be evaluated remotely.
+ */
+ remote_conds = (List *) list_nth(fscan->fdw_private,
+ FdwScanPrivateRemoteConds);
+
+ /*
+ * Extract the relevant RETURNING list if any.
+ */
+ if (plan->returningLists)
+ returningList = (List *) list_nth(plan->returningLists, subplan_index);
+
+ /*
+ * Construct the SQL command string.
+ */
+ switch (operation)
+ {
+ case CMD_UPDATE:
+ deparsePushedDownUpdateSql(&sql, root, resultRelation, rel,
+ ((Plan *) fscan)->targetlist,
+ targetAttrs,
+ remote_conds, ¶ms_list,
+ returningList, &retrieved_attrs);
+ break;
+ case CMD_DELETE:
+ deparsePushedDownDeleteSql(&sql, root, resultRelation, rel,
+ remote_conds, ¶ms_list,
+ returningList, &retrieved_attrs);
+ break;
+ default:
+ elog(ERROR, "unexpected operation: %d", (int) operation);
+ break;
+ }
+
+ /*
+ * Update the operation info.
+ */
+ fscan->operation = operation;
+
+ /*
+ * Update the fdw_exprs list that will be available to the executor.
+ */
+ fscan->fdw_exprs = params_list;
+
+ /*
+ * Update the fdw_private list that will be available to the executor.
+ * Items in the list must match enum FdwDirectModifyPrivateIndex, above.
+ */
+ fscan->fdw_private = list_make4(makeString(sql.data),
+ makeInteger((retrieved_attrs != NIL)),
+ retrieved_attrs,
+ makeInteger(plan->canSetTag));
+
+ heap_close(rel, NoLock);
+ return true;
+ }
+
+ /*
+ * postgresBeginDirectModify
+ * Prepare a direct foreign table modification
+ */
+ static void
+ postgresBeginDirectModify(ForeignScanState *node, int eflags)
+ {
+ ForeignScan *fsplan = (ForeignScan *) node->ss.ps.plan;
+ EState *estate = node->ss.ps.state;
+ PgFdwDirectModifyState *dmstate;
+ RangeTblEntry *rte;
+ Oid userid;
+ ForeignTable *table;
+ UserMapping *user;
+ int numParams;
+
+ /*
+ * Do nothing in EXPLAIN (no ANALYZE) case. node->fdw_state stays NULL.
+ */
+ if (eflags & EXEC_FLAG_EXPLAIN_ONLY)
+ return;
+
+ /*
+ * We'll save private state in node->fdw_state.
+ */
+ dmstate = (PgFdwDirectModifyState *) palloc0(sizeof(PgFdwDirectModifyState));
+ node->fdw_state = (void *) dmstate;
+
+ /*
+ * Identify which user to do the remote access as. This should match what
+ * ExecCheckRTEPerms() does.
+ */
+ rte = rt_fetch(fsplan->scan.scanrelid, estate->es_range_table);
+ userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
+
+ /* Get info about foreign table. */
+ dmstate->rel = node->ss.ss_currentRelation;
+ table = GetForeignTable(RelationGetRelid(dmstate->rel));
+ user = GetUserMapping(userid, table->serverid);
+
+ /*
+ * Get connection to the foreign server. Connection manager will
+ * establish new connection if necessary.
+ */
+ dmstate->conn = GetConnection(user, false);
+
+ /* Initialize state variable */
+ dmstate->num_tuples = -1; /* -1 means not set yet */
+
+ /* Get private info created by planner functions. */
+ dmstate->query = strVal(list_nth(fsplan->fdw_private,
+ FdwDirectModifyPrivateUpdateSql));
+ dmstate->has_returning = intVal(list_nth(fsplan->fdw_private,
+ FdwDirectModifyPrivateHasReturning));
+ dmstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
+ FdwDirectModifyPrivateRetrievedAttrs);
+ dmstate->set_processed = intVal(list_nth(fsplan->fdw_private,
+ FdwDirectModifyPrivateSetProcessed));
+
+ /* Create context for per-tuple temp workspace. */
+ dmstate->temp_cxt = AllocSetContextCreate(estate->es_query_cxt,
+ "postgres_fdw temporary data",
+ ALLOCSET_SMALL_MINSIZE,
+ ALLOCSET_SMALL_INITSIZE,
+ ALLOCSET_SMALL_MAXSIZE);
+
+ /* Prepare for input conversion of RETURNING results. */
+ if (dmstate->has_returning)
+ dmstate->attinmeta = TupleDescGetAttInMetadata(RelationGetDescr(dmstate->rel));
+
+ /*
+ * Prepare for processing of parameters used in remote query, if any.
+ */
+ numParams = list_length(fsplan->fdw_exprs);
+ dmstate->numParams = numParams;
+ if (numParams > 0)
+ prepare_query_params((PlanState *) node,
+ fsplan->fdw_exprs,
+ numParams,
+ &dmstate->param_flinfo,
+ &dmstate->param_exprs,
+ &dmstate->param_values);
+ }
+
+ /*
+ * postgresIterateDirectModify
+ * Execute a direct foreign table modification
+ */
+ static TupleTableSlot *
+ postgresIterateDirectModify(ForeignScanState *node)
+ {
+ PgFdwDirectModifyState *dmstate = (PgFdwDirectModifyState *) node->fdw_state;
+ EState *estate = node->ss.ps.state;
+ ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
+
+ /*
+ * If this is the first call after Begin, execute the statement.
+ */
+ if (dmstate->num_tuples == -1)
+ execute_dml_stmt(node);
+
+ /*
+ * If the local query doesn't specify RETURNING, just clear tuple slot.
+ */
+ if (!resultRelInfo->ri_projectReturning)
+ {
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+ Instrumentation *instr = node->ss.ps.instrument;
+
+ Assert(!dmstate->has_returning);
+
+ /* Increment the command es_processed count if necessary. */
+ if (dmstate->set_processed)
+ estate->es_processed += dmstate->num_tuples;
+
+ /* Increment the tuple count for EXPLAIN ANALYZE if necessary. */
+ if (instr)
+ instr->tuplecount += dmstate->num_tuples;
+
+ return ExecClearTuple(slot);
+ }
+
+ /*
+ * Get the next RETURNING tuple.
+ */
+ return get_returning_data(node);
+ }
+
+ /*
+ * postgresEndDirectModify
+ * Finish a direct foreign table modification
+ */
+ static void
+ postgresEndDirectModify(ForeignScanState *node)
+ {
+ PgFdwDirectModifyState *dmstate = (PgFdwDirectModifyState *) node->fdw_state;
+
+ /* if dmstate is NULL, we are in EXPLAIN; nothing to do */
+ if (dmstate == NULL)
+ return;
+
+ /* Release PGresult */
+ if (dmstate->result)
+ PQclear(dmstate->result);
+
+ /* Release remote connection */
+ ReleaseConnection(dmstate->conn);
+ dmstate->conn = NULL;
+
+ /* MemoryContext will be deleted automatically. */
+ }
+
+ /*
* postgresExplainForeignScan
* Produce extra output for EXPLAIN of a ForeignScan on a foreign table
*/
***************
*** 2055,2060 **** postgresExplainForeignModify(ModifyTableState *mtstate,
--- 2410,2434 ----
}
}
+ /*
+ * postgresExplainDirectModify
+ * Produce extra output for EXPLAIN of a ForeignScan that modifies a
+ * foreign table directly
+ */
+ static void
+ postgresExplainDirectModify(ForeignScanState *node, ExplainState *es)
+ {
+ List *fdw_private;
+ char *sql;
+
+ if (es->verbose)
+ {
+ fdw_private = ((ForeignScan *) node->ss.ps.plan)->fdw_private;
+ sql = strVal(list_nth(fdw_private, FdwDirectModifyPrivateUpdateSql));
+ ExplainPropertyText("Remote SQL", sql, es);
+ }
+ }
+
/*
* estimate_path_cost_size
***************
*** 2413,2450 **** create_cursor(ForeignScanState *node)
*/
if (numParams > 0)
{
- int nestlevel;
MemoryContext oldcontext;
- int i;
- ListCell *lc;
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! nestlevel = set_transmission_modes();
!
! i = 0;
! foreach(lc, fsstate->param_exprs)
! {
! ExprState *expr_state = (ExprState *) lfirst(lc);
! Datum expr_value;
! bool isNull;
!
! /* Evaluate the parameter expression */
! expr_value = ExecEvalExpr(expr_state, econtext, &isNull, NULL);
!
! /*
! * Get string representation of each parameter value by invoking
! * type-specific output function, unless the value is null.
! */
! if (isNull)
! values[i] = NULL;
! else
! values[i] = OutputFunctionCall(&fsstate->param_flinfo[i],
! expr_value);
! i++;
! }
!
! reset_transmission_modes(nestlevel);
MemoryContextSwitchTo(oldcontext);
}
--- 2787,2800 ----
*/
if (numParams > 0)
{
MemoryContext oldcontext;
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! process_query_params(econtext,
! fsstate->param_flinfo,
! fsstate->param_exprs,
! values);
MemoryContextSwitchTo(oldcontext);
}
***************
*** 2765,2770 **** store_returning_result(PgFdwModifyState *fmstate,
--- 3115,3311 ----
}
/*
+ * Execute a pushed-down UPDATE/DELETE statement.
+ */
+ static void
+ execute_dml_stmt(ForeignScanState *node)
+ {
+ PgFdwDirectModifyState *dmstate = (PgFdwDirectModifyState *) node->fdw_state;
+ ExprContext *econtext = node->ss.ps.ps_ExprContext;
+ int numParams = dmstate->numParams;
+ const char **values = dmstate->param_values;
+
+ /*
+ * Construct array of query parameter values in text format.
+ */
+ if (numParams > 0)
+ process_query_params(econtext,
+ dmstate->param_flinfo,
+ dmstate->param_exprs,
+ values);
+
+ /*
+ * Notice that we pass NULL for paramTypes, thus forcing the remote server
+ * to infer types for all parameters. Since we explicitly cast every
+ * parameter (see deparse.c), the "inference" is trivial and will produce
+ * the desired result. This allows us to avoid assuming that the remote
+ * server has the same OIDs we do for the parameters' types.
+ *
+ * We don't use a PG_TRY block here, so be careful not to throw error
+ * without releasing the PGresult.
+ */
+ dmstate->result = PQexecParams(dmstate->conn, dmstate->query,
+ numParams, NULL, values, NULL, NULL, 0);
+ if (PQresultStatus(dmstate->result) !=
+ (dmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
+ pgfdw_report_error(ERROR, dmstate->result, dmstate->conn, true,
+ dmstate->query);
+
+ /* Get the number of rows affected. */
+ if (dmstate->has_returning)
+ dmstate->num_tuples = PQntuples(dmstate->result);
+ else
+ dmstate->num_tuples = atoi(PQcmdTuples(dmstate->result));
+ }
+
+ /*
+ * Get the result of a RETURNING clause.
+ */
+ static TupleTableSlot *
+ get_returning_data(ForeignScanState *node)
+ {
+ PgFdwDirectModifyState *dmstate = (PgFdwDirectModifyState *) node->fdw_state;
+ EState *estate = node->ss.ps.state;
+ ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+
+ Assert(resultRelInfo->ri_projectReturning);
+
+ /* If we didn't get any tuples, must be end of data. */
+ if (dmstate->next_tuple >= dmstate->num_tuples)
+ return ExecClearTuple(slot);
+
+ /* Increment the command es_processed count if necessary. */
+ if (dmstate->set_processed)
+ estate->es_processed += 1;
+
+ /*
+ * Store a RETURNING tuple. If has_returning is false, just emit a dummy
+ * tuple. (We have has_returning=false if the local query is of the form
+ * UPDATE/DELETE .. RETURNING 1 for example.)
+ */
+ if (!dmstate->has_returning)
+ ExecStoreAllNullTuple(slot);
+ else
+ {
+ /*
+ * On error, be sure to release the PGresult on the way out. Callers
+ * do not have PG_TRY blocks to ensure this happens.
+ */
+ PG_TRY();
+ {
+ HeapTuple newtup;
+
+ newtup = make_tuple_from_result_row(dmstate->result,
+ dmstate->next_tuple,
+ dmstate->rel,
+ dmstate->attinmeta,
+ dmstate->retrieved_attrs,
+ NULL,
+ dmstate->temp_cxt);
+ ExecStoreTuple(newtup, slot, InvalidBuffer, false);
+ }
+ PG_CATCH();
+ {
+ if (dmstate->result)
+ PQclear(dmstate->result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ }
+ dmstate->next_tuple++;
+
+ /* Make slot available for evaluation of the local query RETURNING list. */
+ resultRelInfo->ri_projectReturning->pi_exprContext->ecxt_scantuple = slot;
+
+ return slot;
+ }
+
+ /*
+ * Prepare for processing of parameters used in remote query.
+ */
+ static void
+ prepare_query_params(PlanState *node,
+ List *fdw_exprs,
+ int numParams,
+ FmgrInfo **param_flinfo,
+ List **param_exprs,
+ const char ***param_values)
+ {
+ int i;
+ ListCell *lc;
+
+ Assert(numParams > 0);
+
+ /* Prepare for output conversion of parameters used in remote query. */
+ *param_flinfo = (FmgrInfo *) palloc0(sizeof(FmgrInfo) * numParams);
+
+ i = 0;
+ foreach(lc, fdw_exprs)
+ {
+ Node *param_expr = (Node *) lfirst(lc);
+ Oid typefnoid;
+ bool isvarlena;
+
+ getTypeOutputInfo(exprType(param_expr), &typefnoid, &isvarlena);
+ fmgr_info(typefnoid, &(*param_flinfo)[i]);
+ i++;
+ }
+
+ /*
+ * Prepare remote-parameter expressions for evaluation. (Note: in
+ * practice, we expect that all these expressions will be just Params, so
+ * we could possibly do something more efficient than using the full
+ * expression-eval machinery for this. But probably there would be little
+ * benefit, and it'd require postgres_fdw to know more than is desirable
+ * about Param evaluation.)
+ */
+ *param_exprs = (List *) ExecInitExpr((Expr *) fdw_exprs, node);
+
+ /* Allocate buffer for text form of query parameters. */
+ *param_values = (const char **) palloc0(numParams * sizeof(char *));
+ }
+
+ /*
+ * Construct array of query parameter values in text format.
+ */
+ static void
+ process_query_params(ExprContext *econtext,
+ FmgrInfo *param_flinfo,
+ List *param_exprs,
+ const char **param_values)
+ {
+ int nestlevel;
+ int i;
+ ListCell *lc;
+
+ nestlevel = set_transmission_modes();
+
+ i = 0;
+ foreach(lc, param_exprs)
+ {
+ ExprState *expr_state = (ExprState *) lfirst(lc);
+ Datum expr_value;
+ bool isNull;
+
+ /* Evaluate the parameter expression */
+ expr_value = ExecEvalExpr(expr_state, econtext, &isNull, NULL);
+
+ /*
+ * Get string representation of each parameter value by invoking
+ * type-specific output function, unless the value is null.
+ */
+ if (isNull)
+ param_values[i] = NULL;
+ else
+ param_values[i] = OutputFunctionCall(¶m_flinfo[i], expr_value);
+ i++;
+ }
+
+ reset_transmission_modes(nestlevel);
+ }
+
+ /*
* postgresAnalyzeForeignTable
* Test whether analyzing this foreign table is supported
*/
*** a/contrib/postgres_fdw/postgres_fdw.h
--- b/contrib/postgres_fdw/postgres_fdw.h
***************
*** 130,139 **** extern void deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 130,153 ----
Index rtindex, Relation rel,
List *targetAttrs, List *returningList,
List **retrieved_attrs);
+ extern void deparsePushedDownUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseDeleteSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *returningList,
List **retrieved_attrs);
+ extern void deparsePushedDownDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseAnalyzeSizeSql(StringInfo buf, Relation rel);
extern void deparseAnalyzeSql(StringInfo buf, Relation rel,
List **retrieved_attrs);
*** a/contrib/postgres_fdw/sql/postgres_fdw.sql
--- b/contrib/postgres_fdw/sql/postgres_fdw.sql
***************
*** 599,626 **** INSERT INTO ft2 (c1,c2,c3) SELECT c1+1000,c2+100, c3 || c3 FROM ft2 LIMIT 20;
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
EXPLAIN (verbose, costs off)
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
! UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
-- Test that trigger on remote table works as expected
--- 599,630 ----
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
EXPLAIN (verbose, costs off)
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
! UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
-- Test that trigger on remote table works as expected
***************
*** 949,954 **** UPDATE rem1 SET f2 = 'testo';
--- 953,1042 ----
-- Test returning a system attribute
INSERT INTO rem1(f2) VALUES ('test') RETURNING ctid;
+ -- cleanup
+ DROP TRIGGER trig_row_before ON rem1;
+ DROP TRIGGER trig_row_after ON rem1;
+ DROP TRIGGER trig_local_before ON loc1;
+
+
+ -- Test direct foreign table modification functionality
+
+ -- Test with statement-level triggers
+ CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_stmt_before ON rem1;
+
+ CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_stmt_after ON rem1;
+
+ -- Test with row-level ON INSERT triggers
+ CREATE TRIGGER trig_row_before_insert
+ BEFORE INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_before_insert ON rem1;
+
+ CREATE TRIGGER trig_row_after_insert
+ AFTER INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_after_insert ON rem1;
+
+ -- Test with row-level ON UPDATE triggers
+ CREATE TRIGGER trig_row_before_update
+ BEFORE UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_before_update ON rem1;
+
+ CREATE TRIGGER trig_row_after_update
+ AFTER UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_after_update ON rem1;
+
+ -- Test with row-level ON DELETE triggers
+ CREATE TRIGGER trig_row_before_delete
+ BEFORE DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ DROP TRIGGER trig_row_before_delete ON rem1;
+
+ CREATE TRIGGER trig_row_after_delete
+ AFTER DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ DROP TRIGGER trig_row_after_delete ON rem1;
+
-- ===================================================================
-- test inheritance features
-- ===================================================================
***************
*** 1080,1085 **** fetch from c;
--- 1168,1180 ----
update bar set f2 = null where current of c;
rollback;
+ explain (verbose, costs off)
+ delete from foo where f1 < 5 returning *;
+ delete from foo where f1 < 5 returning *;
+ explain (verbose, costs off)
+ update bar set f2 = f2 + 100 returning *;
+ update bar set f2 = f2 + 100 returning *;
+
drop table foo cascade;
drop table bar cascade;
drop table loct1;
*** a/doc/src/sgml/fdwhandler.sgml
--- b/doc/src/sgml/fdwhandler.sgml
***************
*** 665,670 **** IsForeignRelUpdatable (Relation rel);
--- 665,822 ----
updatability for display in the <literal>information_schema</> views.)
</para>
+ <para>
+ Some inserts, updates, and deletes to foreign tables can be optimized
+ by implementing an alternative set of interfaces. The ordinary
+ interfaces for inserts, updates, and deletes fetch rows from the remote
+ server and then modify those rows one at a time. In some cases, this
+ row-by-row approach is necessary, but it can be inefficient. If it is
+ possible for the foreign server to determine which rows should be
+ modified without actually retrieving them, and if there are no local
+ triggers which would affect the operation, then it is possible to
+ arrange things so that the entire operation is performed on the remote
+ server. The interfaces described below make this possible.
+ </para>
+
+ <para>
+ <programlisting>
+ bool
+ PlanDirectModify (PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+ </programlisting>
+
+ Decide whether it is safe to execute a direct modification
+ on the remote server. If so, return <literal>true</> after performing
+ planning actions needed for that. Otherwise, return <literal>false</>.
+ This optional function is called during query planning.
+ If this function succeeds, <function>BeginDirectModify</>,
+ <function>IterateDirectModify</> and <function>EndDirectModify</> will
+ be called at the execution stage, instead. Otherwise, the table
+ modification will be executed using the table-updating functions
+ described above.
+ The parameters are the same as for <function>PlanForeignModify</>.
+ </para>
+
+ <para>
+ To execute the direct modification on the remote server, this function
+ must rewrite the target subplan with a <structname>ForeignScan</> plan
+ node that executes the direct modification on the remote server. The
+ <structfield>operation</> field of the <structname>ForeignScan</> must
+ be set to the <literal>CmdType</> enumeration appropriately; that is,
+ <literal>CMD_UPDATE</> for <command>UPDATE</>,
+ <literal>CMD_INSERT</> for <command>INSERT</>, and
+ <literal>CMD_DELETE</> for <command>DELETE</>.
+ </para>
+
+ <para>
+ See <xref linkend="fdw-planning"> for additional information.
+ </para>
+
+ <para>
+ If the <function>PlanDirectModify</> pointer is set to
+ <literal>NULL</>, no attempts to execute a direct modification on the
+ remote server are taken.
+ </para>
+
+ <para>
+ <programlisting>
+ void
+ BeginDirectModify (ForeignScanState *node,
+ int eflags);
+ </programlisting>
+
+ Prepare to execute a direct modification on the remote server.
+ This is called during executor startup. It should perform any
+ initialization needed prior to the direct modification (that should be
+ done upon the first call to <function>IterateDirectModify</>).
+ The <structname>ForeignScanState</> node has already been created, but
+ its <structfield>fdw_state</> field is still NULL. Information about
+ the table to modify is accessible through the
+ <structname>ForeignScanState</> node (in particular, from the underlying
+ <structname>ForeignScan</> plan node, which contains any FDW-private
+ information provided by <function>PlanDirectModify</>).
+ <literal>eflags</> contains flag bits describing the executor's
+ operating mode for this plan node.
+ </para>
+
+ <para>
+ Note that when <literal>(eflags & EXEC_FLAG_EXPLAIN_ONLY)</> is
+ true, this function should not perform any externally-visible actions;
+ it should only do the minimum required to make the node state valid
+ for <function>ExplainDirectModify</> and <function>EndDirectModify</>.
+ </para>
+
+ <para>
+ If the <function>BeginDirectModify</> pointer is set to
+ <literal>NULL</>, no attempts to execute a direct modification on the
+ remote server are taken.
+ </para>
+
+ <para>
+ <programlisting>
+ TupleTableSlot *
+ IterateDirectModify (ForeignScanState *node);
+ </programlisting>
+
+ When the <command>INSERT</>, <command>UPDATE</> or <command>DELETE</>
+ query doesn't have a <literal>RETURNING</> clause, just return NULL
+ after a direct modification on the remote server.
+ When the query has the clause, fetch one result containing the data
+ needed for the <literal>RETURNING</> calculation, returning it in a
+ tuple table slot (the node's <structfield>ScanTupleSlot</> should be
+ used for this purpose). The data that was actually inserted, updated
+ or deleted must be stored in the
+ <literal>es_result_relation_info->ri_projectReturning->pi_exprContext->ecxt_scantuple</>
+ of the node's <structname>EState</>.
+ Return NULL if no more rows are available.
+ Note that this is called in a short-lived memory context that will be
+ reset between invocations. Create a memory context in
+ <function>BeginDirectModify</> if you need longer-lived storage, or use
+ the <structfield>es_query_cxt</> of the node's <structname>EState</>.
+ </para>
+
+ <para>
+ The rows returned must match the <structfield>fdw_scan_tlist</> target
+ list if one was supplied, otherwise they must match the row type of the
+ foreign table being updated. If you choose to optimize away fetching
+ columns that are not needed for the <literal>RETURNING</> calculation,
+ you should insert nulls in those column positions, or else generate a
+ <structfield>fdw_scan_tlist</> list with those columns omitted.
+ </para>
+
+ <para>
+ Whether the query has the clause or not, the query's reported row count
+ must be incremented by the FDW itself. When the query doesn't have the
+ clause, the FDW must also increment the row count for the
+ <structname>ForeignScanState</> node in the <command>EXPLAIN ANALYZE</>
+ case.
+ </para>
+
+ <para>
+ If the <function>IterateDirectModify</> pointer is set to
+ <literal>NULL</>, no attempts to execute a direct modification on the
+ remote server are taken.
+ </para>
+
+ <para>
+ <programlisting>
+ void
+ EndDirectModify (ForeignScanState *node);
+ </programlisting>
+
+ Clean up following a direc modification on the remote server. It is
+ normally not important to release palloc'd memory, but for example open
+ files and connections to the remote server should be cleaned up.
+ </para>
+
+ <para>
+ If the <function>EndDirectModify</> pointer is set to
+ <literal>NULL</>, no attempts to execute a direct modification on the
+ remote server are taken.
+ </para>
+
</sect2>
<sect2 id="fdw-callbacks-row-locking">
***************
*** 856,861 **** ExplainForeignModify (ModifyTableState *mtstate,
--- 1008,1036 ----
<command>EXPLAIN</>.
</para>
+ <para>
+ <programlisting>
+ void
+ ExplainDirectModify (ForeignScanState *node,
+ ExplainState *es);
+ </programlisting>
+
+ Print additional <command>EXPLAIN</> output for a direct modification
+ on the remote server.
+ This function can call <function>ExplainPropertyText</> and
+ related functions to add fields to the <command>EXPLAIN</> output.
+ The flag fields in <literal>es</> can be used to determine what to
+ print, and the state of the <structname>ForeignScanState</> node
+ can be inspected to provide run-time statistics in the <command>EXPLAIN
+ ANALYZE</> case.
+ </para>
+
+ <para>
+ If the <function>ExplainDirectModify</> pointer is set to
+ <literal>NULL</>, no additional information is printed during
+ <command>EXPLAIN</>.
+ </para>
+
</sect2>
<sect2 id="fdw-callbacks-analyze">
***************
*** 1160,1166 **** GetForeignServerByName(const char *name, bool missing_ok);
<para>
The FDW callback functions <function>GetForeignRelSize</>,
<function>GetForeignPaths</>, <function>GetForeignPlan</>,
! <function>PlanForeignModify</>, and <function>GetForeignJoinPaths</>
must fit into the workings of the <productname>PostgreSQL</> planner.
Here are some notes about what they must do.
</para>
--- 1335,1342 ----
<para>
The FDW callback functions <function>GetForeignRelSize</>,
<function>GetForeignPaths</>, <function>GetForeignPlan</>,
! <function>PlanForeignModify</>, <function>GetForeignJoinPaths</>, and
! <function>PlanDirectModify</>
must fit into the workings of the <productname>PostgreSQL</> planner.
Here are some notes about what they must do.
</para>
***************
*** 1354,1360 **** GetForeignServerByName(const char *name, bool missing_ok);
<para>
When planning an <command>UPDATE</> or <command>DELETE</>,
! <function>PlanForeignModify</> can look up the <structname>RelOptInfo</>
struct for the foreign table and make use of the
<literal>baserel->fdw_private</> data previously created by the
scan-planning functions. However, in <command>INSERT</> the target
--- 1530,1537 ----
<para>
When planning an <command>UPDATE</> or <command>DELETE</>,
! <function>PlanForeignModify</> and <function>PlanDirectModify</>
! can look up the <structname>RelOptInfo</>
struct for the foreign table and make use of the
<literal>baserel->fdw_private</> data previously created by the
scan-planning functions. However, in <command>INSERT</> the target
*** a/doc/src/sgml/postgres-fdw.sgml
--- b/doc/src/sgml/postgres-fdw.sgml
***************
*** 484,489 ****
--- 484,498 ----
extension that's listed in the foreign server's <literal>extensions</>
option. Operators and functions in such clauses must
be <literal>IMMUTABLE</> as well.
+ For an <command>UPDATE</> or <command>DELETE</> query,
+ <filename>postgres_fdw</> attempts to optimize the query execution by
+ sending the whole query to the remote server if there are no query
+ <literal>WHERE</> clauses that cannot be sent to the remote server,
+ no local joins for the query, and no row-level local <literal>BEFORE</> or
+ <literal>AFTER</> triggers on the target table. In <command>UPDATE</>,
+ expressions to assign to target columns must use only built-in data types,
+ <literal>IMMUTABLE</> operators, or <literal>IMMUTABLE</> functions,
+ to reduce the risk of misexecution of the query.
</para>
<para>
*** a/src/backend/commands/explain.c
--- b/src/backend/commands/explain.c
***************
*** 900,906 **** ExplainNode(PlanState *planstate, List *ancestors,
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
! pname = sname = "Foreign Scan";
break;
case T_CustomScan:
sname = "Custom Scan";
--- 900,928 ----
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
! sname = "Foreign Scan";
! switch (((ForeignScan *) plan)->operation)
! {
! case CMD_SELECT:
! pname = "Foreign Scan";
! operation = "Select";
! break;
! case CMD_INSERT:
! pname = "Foreign Insert";
! operation = "Insert";
! break;
! case CMD_UPDATE:
! pname = "Foreign Update";
! operation = "Update";
! break;
! case CMD_DELETE:
! pname = "Foreign Delete";
! operation = "Delete";
! break;
! default:
! pname = "???";
! break;
! }
break;
case T_CustomScan:
sname = "Custom Scan";
***************
*** 1648,1653 **** show_plan_tlist(PlanState *planstate, List *ancestors, ExplainState *es)
--- 1670,1690 ----
return;
if (IsA(plan, RecursiveUnion))
return;
+ /*
+ * Likewise for ForeignScan that has pushed down INSERT/UPDATE/DELETE
+ *
+ * Note: the tlist for an ForeignScan that has pushed down an INSERT/
+ * UPDATE might contain subplan output expressions that are confusing
+ * in this context. The tlist for an ForeignScan that has pushed down
+ * an UPDATE/DELETE always contains "junk" target columns to identify
+ * the exact row to update or delete, which would be confusing in this
+ * context. So, we suppress it in all the cases.
+ */
+ if (IsA(plan, ForeignScan) &&
+ (((ForeignScan *) plan)->operation == CMD_INSERT ||
+ ((ForeignScan *) plan)->operation == CMD_UPDATE ||
+ ((ForeignScan *) plan)->operation == CMD_DELETE))
+ return;
/* Set up deparsing context */
context = set_deparse_context_planstate(es->deparse_cxt,
***************
*** 2236,2243 **** show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es)
FdwRoutine *fdwroutine = fsstate->fdwroutine;
/* Let the FDW emit whatever fields it wants */
! if (fdwroutine->ExplainForeignScan != NULL)
! fdwroutine->ExplainForeignScan(fsstate, es);
}
/*
--- 2273,2290 ----
FdwRoutine *fdwroutine = fsstate->fdwroutine;
/* Let the FDW emit whatever fields it wants */
! if (((ForeignScan *) fsstate->ss.ps.plan)->operation == CMD_INSERT ||
! ((ForeignScan *) fsstate->ss.ps.plan)->operation == CMD_UPDATE ||
! ((ForeignScan *) fsstate->ss.ps.plan)->operation == CMD_DELETE)
! {
! if (fdwroutine->ExplainDirectModify != NULL)
! fdwroutine->ExplainDirectModify(fsstate, es);
! }
! else
! {
! if (fdwroutine->ExplainForeignScan != NULL)
! fdwroutine->ExplainForeignScan(fsstate, es);
! }
}
/*
***************
*** 2623,2630 **** show_modifytable_info(ModifyTableState *mtstate, List *ancestors,
}
}
! /* Give FDW a chance */
! if (fdwroutine && fdwroutine->ExplainForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, j);
--- 2670,2679 ----
}
}
! /* Give FDW a chance if needed */
! if (!resultRelInfo->ri_usesFdwDirectModify &&
! fdwroutine != NULL &&
! fdwroutine->ExplainForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, j);
*** a/src/backend/executor/execMain.c
--- b/src/backend/executor/execMain.c
***************
*** 1245,1250 **** InitResultRelInfo(ResultRelInfo *resultRelInfo,
--- 1245,1251 ----
else
resultRelInfo->ri_FdwRoutine = NULL;
resultRelInfo->ri_FdwState = NULL;
+ resultRelInfo->ri_usesFdwDirectModify = false;
resultRelInfo->ri_ConstraintExprs = NULL;
resultRelInfo->ri_junkFilter = NULL;
resultRelInfo->ri_projectReturning = NULL;
*** a/src/backend/executor/nodeForeignscan.c
--- b/src/backend/executor/nodeForeignscan.c
***************
*** 48,54 **** ForeignNext(ForeignScanState *node)
/* Call the Iterate function in short-lived context */
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! slot = node->fdwroutine->IterateForeignScan(node);
MemoryContextSwitchTo(oldcontext);
/*
--- 48,59 ----
/* Call the Iterate function in short-lived context */
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! if (plan->operation == CMD_INSERT ||
! plan->operation == CMD_UPDATE ||
! plan->operation == CMD_DELETE)
! slot = node->fdwroutine->IterateDirectModify(node);
! else
! slot = node->fdwroutine->IterateForeignScan(node);
MemoryContextSwitchTo(oldcontext);
/*
***************
*** 226,232 **** ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
/*
* Tell the FDW to initialize the scan.
*/
! fdwroutine->BeginForeignScan(scanstate, eflags);
return scanstate;
}
--- 231,242 ----
/*
* Tell the FDW to initialize the scan.
*/
! if (node->operation == CMD_INSERT ||
! node->operation == CMD_UPDATE ||
! node->operation == CMD_DELETE)
! fdwroutine->BeginDirectModify(scanstate, eflags);
! else
! fdwroutine->BeginForeignScan(scanstate, eflags);
return scanstate;
}
***************
*** 240,247 **** ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
void
ExecEndForeignScan(ForeignScanState *node)
{
/* Let the FDW shut down */
! node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
--- 250,264 ----
void
ExecEndForeignScan(ForeignScanState *node)
{
+ ForeignScan *plan = (ForeignScan *) node->ss.ps.plan;
+
/* Let the FDW shut down */
! if (plan->operation == CMD_INSERT ||
! plan->operation == CMD_UPDATE ||
! plan->operation == CMD_DELETE)
! node->fdwroutine->EndDirectModify(node);
! else
! node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
*** a/src/backend/executor/nodeModifyTable.c
--- b/src/backend/executor/nodeModifyTable.c
***************
*** 138,150 **** ExecCheckPlanOutput(Relation resultRel, List *targetList)
* tupleSlot: slot holding tuple actually inserted/updated/deleted
* planSlot: slot holding tuple returned by top subplan node
*
* Returns a slot holding the result tuple
*/
static TupleTableSlot *
! ExecProcessReturning(ProjectionInfo *projectReturning,
TupleTableSlot *tupleSlot,
TupleTableSlot *planSlot)
{
ExprContext *econtext = projectReturning->pi_exprContext;
/*
--- 138,154 ----
* tupleSlot: slot holding tuple actually inserted/updated/deleted
* planSlot: slot holding tuple returned by top subplan node
*
+ * Note: If tupleSlot is NULL, the FDW should have already provided econtext's
+ * scan tuple.
+ *
* Returns a slot holding the result tuple
*/
static TupleTableSlot *
! ExecProcessReturning(ResultRelInfo *resultRelInfo,
TupleTableSlot *tupleSlot,
TupleTableSlot *planSlot)
{
+ ProjectionInfo *projectReturning = resultRelInfo->ri_projectReturning;
ExprContext *econtext = projectReturning->pi_exprContext;
/*
***************
*** 154,160 **** ExecProcessReturning(ProjectionInfo *projectReturning,
ResetExprContext(econtext);
/* Make tuple and any needed join variables available to ExecProject */
! econtext->ecxt_scantuple = tupleSlot;
econtext->ecxt_outertuple = planSlot;
/* Compute the RETURNING expressions */
--- 158,177 ----
ResetExprContext(econtext);
/* Make tuple and any needed join variables available to ExecProject */
! if (tupleSlot)
! econtext->ecxt_scantuple = tupleSlot;
! else
! {
! HeapTuple tuple;
!
! /*
! * RETURNING expressions might reference the tableoid column, so
! * initialize t_tableOid before evaluating them.
! */
! Assert(!TupIsNull(econtext->ecxt_scantuple));
! tuple = ExecMaterializeSlot(econtext->ecxt_scantuple);
! tuple->t_tableOid = RelationGetRelid(resultRelInfo->ri_RelationDesc);
! }
econtext->ecxt_outertuple = planSlot;
/* Compute the RETURNING expressions */
***************
*** 496,503 **** ExecInsert(ModifyTableState *mtstate,
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo->ri_projectReturning,
! slot, planSlot);
return NULL;
}
--- 513,519 ----
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo, slot, planSlot);
return NULL;
}
***************
*** 738,745 **** ldelete:;
ExecStoreTuple(&deltuple, slot, InvalidBuffer, false);
}
! rslot = ExecProcessReturning(resultRelInfo->ri_projectReturning,
! slot, planSlot);
/*
* Before releasing the target tuple again, make sure rslot has a
--- 754,760 ----
ExecStoreTuple(&deltuple, slot, InvalidBuffer, false);
}
! rslot = ExecProcessReturning(resultRelInfo, slot, planSlot);
/*
* Before releasing the target tuple again, make sure rslot has a
***************
*** 1024,1031 **** lreplace:;
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo->ri_projectReturning,
! slot, planSlot);
return NULL;
}
--- 1039,1045 ----
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo, slot, planSlot);
return NULL;
}
***************
*** 1380,1385 **** ExecModifyTable(ModifyTableState *node)
--- 1394,1419 ----
break;
}
+ /*
+ * If resultRelInfo->ri_usesFdwDirectModify is true, all we need to do
+ * here is compute the RETURNING expressions.
+ */
+ if (resultRelInfo->ri_usesFdwDirectModify)
+ {
+ Assert(resultRelInfo->ri_projectReturning);
+
+ /*
+ * A scan slot containing the data that was actually inserted,
+ * updated or deleted has already been made available to
+ * ExecProcessReturning by IterateDirectModify, so no need to
+ * provide it here.
+ */
+ slot = ExecProcessReturning(resultRelInfo, NULL, planSlot);
+
+ estate->es_result_relation_info = saved_resultRelInfo;
+ return slot;
+ }
+
EvalPlanQualSetSlot(&node->mt_epqstate, planSlot);
slot = planSlot;
***************
*** 1559,1564 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
--- 1593,1602 ----
{
subplan = (Plan *) lfirst(l);
+ /* Initialize the usesFdwDirectModify flag */
+ resultRelInfo->ri_usesFdwDirectModify = bms_is_member(i,
+ node->fdwDirectModifyPlans);
+
/*
* Verify result relation is a valid target for the current operation
*/
***************
*** 1583,1589 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
/* Also let FDWs init themselves for foreign-table result rels */
! if (resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, i);
--- 1621,1628 ----
mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
/* Also let FDWs init themselves for foreign-table result rels */
! if (!resultRelInfo->ri_usesFdwDirectModify &&
! resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, i);
***************
*** 1910,1916 **** ExecEndModifyTable(ModifyTableState *node)
{
ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
! if (resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->EndForeignModify != NULL)
resultRelInfo->ri_FdwRoutine->EndForeignModify(node->ps.state,
resultRelInfo);
--- 1949,1956 ----
{
ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
! if (!resultRelInfo->ri_usesFdwDirectModify &&
! resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->EndForeignModify != NULL)
resultRelInfo->ri_FdwRoutine->EndForeignModify(node->ps.state,
resultRelInfo);
*** a/src/backend/nodes/copyfuncs.c
--- b/src/backend/nodes/copyfuncs.c
***************
*** 188,193 **** _copyModifyTable(const ModifyTable *from)
--- 188,194 ----
COPY_NODE_FIELD(withCheckOptionLists);
COPY_NODE_FIELD(returningLists);
COPY_NODE_FIELD(fdwPrivLists);
+ COPY_BITMAPSET_FIELD(fdwDirectModifyPlans);
COPY_NODE_FIELD(rowMarks);
COPY_SCALAR_FIELD(epqParam);
COPY_SCALAR_FIELD(onConflictAction);
***************
*** 648,653 **** _copyForeignScan(const ForeignScan *from)
--- 649,655 ----
/*
* copy remainder of node
*/
+ COPY_SCALAR_FIELD(operation);
COPY_SCALAR_FIELD(fs_server);
COPY_NODE_FIELD(fdw_exprs);
COPY_NODE_FIELD(fdw_private);
*** a/src/backend/nodes/outfuncs.c
--- b/src/backend/nodes/outfuncs.c
***************
*** 356,361 **** _outModifyTable(StringInfo str, const ModifyTable *node)
--- 356,362 ----
WRITE_NODE_FIELD(withCheckOptionLists);
WRITE_NODE_FIELD(returningLists);
WRITE_NODE_FIELD(fdwPrivLists);
+ WRITE_BITMAPSET_FIELD(fdwDirectModifyPlans);
WRITE_NODE_FIELD(rowMarks);
WRITE_INT_FIELD(epqParam);
WRITE_ENUM_FIELD(onConflictAction, OnConflictAction);
***************
*** 608,613 **** _outForeignScan(StringInfo str, const ForeignScan *node)
--- 609,615 ----
_outScanInfo(str, (const Scan *) node);
+ WRITE_ENUM_FIELD(operation, CmdType);
WRITE_OID_FIELD(fs_server);
WRITE_NODE_FIELD(fdw_exprs);
WRITE_NODE_FIELD(fdw_private);
*** a/src/backend/nodes/readfuncs.c
--- b/src/backend/nodes/readfuncs.c
***************
*** 1481,1486 **** _readModifyTable(void)
--- 1481,1487 ----
READ_NODE_FIELD(withCheckOptionLists);
READ_NODE_FIELD(returningLists);
READ_NODE_FIELD(fdwPrivLists);
+ READ_BITMAPSET_FIELD(fdwDirectModifyPlans);
READ_NODE_FIELD(rowMarks);
READ_INT_FIELD(epqParam);
READ_ENUM_FIELD(onConflictAction, OnConflictAction);
*** a/src/backend/optimizer/plan/createplan.c
--- b/src/backend/optimizer/plan/createplan.c
***************
*** 4904,4909 **** make_foreignscan(List *qptlist,
--- 4904,4910 ----
plan->lefttree = outer_plan;
plan->righttree = NULL;
node->scan.scanrelid = scanrelid;
+ node->operation = CMD_SELECT;
/* fs_server will be filled in by create_foreignscan_plan */
node->fs_server = InvalidOid;
node->fdw_exprs = fdw_exprs;
***************
*** 6017,6022 **** make_modifytable(PlannerInfo *root,
--- 6018,6024 ----
{
ModifyTable *node = makeNode(ModifyTable);
List *fdw_private_list;
+ Bitmapset *direct_modify_plans;
ListCell *lc;
int i;
***************
*** 6074,6085 **** make_modifytable(PlannerInfo *root,
--- 6076,6089 ----
* construct private plan data, and accumulate it all into a list.
*/
fdw_private_list = NIL;
+ direct_modify_plans = NULL;
i = 0;
foreach(lc, resultRelations)
{
Index rti = lfirst_int(lc);
FdwRoutine *fdwroutine;
List *fdw_private;
+ bool direct_modify;
/*
* If possible, we want to get the FdwRoutine from our RelOptInfo for
***************
*** 6106,6112 **** make_modifytable(PlannerInfo *root,
--- 6110,6132 ----
fdwroutine = NULL;
}
+ /*
+ * If the target foreign table has any row-level triggers, we can't
+ * modify the foreign table directly.
+ */
+ direct_modify = false;
if (fdwroutine != NULL &&
+ fdwroutine->PlanDirectModify != NULL &&
+ fdwroutine->BeginDirectModify != NULL &&
+ fdwroutine->IterateDirectModify != NULL &&
+ fdwroutine->EndDirectModify != NULL &&
+ !has_row_triggers(root, rti, operation))
+ direct_modify = fdwroutine->PlanDirectModify(root, node, rti, i);
+ if (direct_modify)
+ direct_modify_plans = bms_add_member(direct_modify_plans, i);
+
+ if (!direct_modify &&
+ fdwroutine != NULL &&
fdwroutine->PlanForeignModify != NULL)
fdw_private = fdwroutine->PlanForeignModify(root, node, rti, i);
else
***************
*** 6115,6120 **** make_modifytable(PlannerInfo *root,
--- 6135,6141 ----
i++;
}
node->fdwPrivLists = fdw_private_list;
+ node->fdwDirectModifyPlans = direct_modify_plans;
return node;
}
*** a/src/backend/optimizer/util/plancat.c
--- b/src/backend/optimizer/util/plancat.c
***************
*** 1540,1542 **** has_unique_index(RelOptInfo *rel, AttrNumber attno)
--- 1540,1589 ----
}
return false;
}
+
+
+ /*
+ * has_row_triggers
+ *
+ * Detect whether the specified relation has any row-level triggers for event.
+ */
+ bool
+ has_row_triggers(PlannerInfo *root, Index rti, CmdType event)
+ {
+ RangeTblEntry *rte = planner_rt_fetch(rti, root);
+ Relation relation;
+ TriggerDesc *trigDesc;
+ bool result = false;
+
+ /* Assume we already have adequate lock */
+ relation = heap_open(rte->relid, NoLock);
+
+ trigDesc = relation->trigdesc;
+ switch (event)
+ {
+ case CMD_INSERT:
+ if (trigDesc &&
+ (trigDesc->trig_insert_after_row ||
+ trigDesc->trig_insert_before_row))
+ result = true;
+ break;
+ case CMD_UPDATE:
+ if (trigDesc &&
+ (trigDesc->trig_update_after_row ||
+ trigDesc->trig_update_before_row))
+ result = true;
+ break;
+ case CMD_DELETE:
+ if (trigDesc &&
+ (trigDesc->trig_delete_after_row ||
+ trigDesc->trig_delete_before_row))
+ result = true;
+ break;
+ default:
+ elog(ERROR, "unrecognized CmdType: %d", (int) event);
+ break;
+ }
+
+ heap_close(relation, NoLock);
+ return result;
+ }
*** a/src/include/foreign/fdwapi.h
--- b/src/include/foreign/fdwapi.h
***************
*** 94,99 **** typedef void (*EndForeignModify_function) (EState *estate,
--- 94,111 ----
typedef int (*IsForeignRelUpdatable_function) (Relation rel);
+ typedef bool (*PlanDirectModify_function) (PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+
+ typedef void (*BeginDirectModify_function) (ForeignScanState *node,
+ int eflags);
+
+ typedef TupleTableSlot *(*IterateDirectModify_function) (ForeignScanState *node);
+
+ typedef void (*EndDirectModify_function) (ForeignScanState *node);
+
typedef RowMarkType (*GetForeignRowMarkType_function) (RangeTblEntry *rte,
LockClauseStrength strength);
***************
*** 111,116 **** typedef void (*ExplainForeignModify_function) (ModifyTableState *mtstate,
--- 123,131 ----
int subplan_index,
struct ExplainState *es);
+ typedef void (*ExplainDirectModify_function) (ForeignScanState *node,
+ struct ExplainState *es);
+
typedef int (*AcquireSampleRowsFunc) (Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
***************
*** 175,180 **** typedef struct FdwRoutine
--- 190,199 ----
ExecForeignDelete_function ExecForeignDelete;
EndForeignModify_function EndForeignModify;
IsForeignRelUpdatable_function IsForeignRelUpdatable;
+ PlanDirectModify_function PlanDirectModify;
+ BeginDirectModify_function BeginDirectModify;
+ IterateDirectModify_function IterateDirectModify;
+ EndDirectModify_function EndDirectModify;
/* Functions for SELECT FOR UPDATE/SHARE row locking */
GetForeignRowMarkType_function GetForeignRowMarkType;
***************
*** 184,189 **** typedef struct FdwRoutine
--- 203,209 ----
/* Support functions for EXPLAIN */
ExplainForeignScan_function ExplainForeignScan;
ExplainForeignModify_function ExplainForeignModify;
+ ExplainDirectModify_function ExplainDirectModify;
/* Support functions for ANALYZE */
AnalyzeForeignTable_function AnalyzeForeignTable;
*** a/src/include/nodes/execnodes.h
--- b/src/include/nodes/execnodes.h
***************
*** 311,316 **** typedef struct JunkFilter
--- 311,317 ----
* TrigInstrument optional runtime measurements for triggers
* FdwRoutine FDW callback functions, if foreign table
* FdwState available to save private state of FDW
+ * usesFdwDirectModify true when modifying foreign table directly
* WithCheckOptions list of WithCheckOption's to be checked
* WithCheckOptionExprs list of WithCheckOption expr states
* ConstraintExprs array of constraint-checking expr states
***************
*** 334,339 **** typedef struct ResultRelInfo
--- 335,341 ----
Instrumentation *ri_TrigInstrument;
struct FdwRoutine *ri_FdwRoutine;
void *ri_FdwState;
+ bool ri_usesFdwDirectModify;
List *ri_WithCheckOptions;
List *ri_WithCheckOptionExprs;
List **ri_ConstraintExprs;
*** a/src/include/nodes/pg_list.h
--- b/src/include/nodes/pg_list.h
***************
*** 134,149 **** list_length(const List *l)
--- 134,152 ----
#define list_make2(x1,x2) lcons(x1, list_make1(x2))
#define list_make3(x1,x2,x3) lcons(x1, list_make2(x2, x3))
#define list_make4(x1,x2,x3,x4) lcons(x1, list_make3(x2, x3, x4))
+ #define list_make5(x1,x2,x3,x4,x5) lcons(x1, list_make4(x2, x3, x4, x5))
#define list_make1_int(x1) lcons_int(x1, NIL)
#define list_make2_int(x1,x2) lcons_int(x1, list_make1_int(x2))
#define list_make3_int(x1,x2,x3) lcons_int(x1, list_make2_int(x2, x3))
#define list_make4_int(x1,x2,x3,x4) lcons_int(x1, list_make3_int(x2, x3, x4))
+ #define list_make5_int(x1,x2,x3,x4,x5) lcons_int(x1, list_make4_int(x2, x3, x4, x5))
#define list_make1_oid(x1) lcons_oid(x1, NIL)
#define list_make2_oid(x1,x2) lcons_oid(x1, list_make1_oid(x2))
#define list_make3_oid(x1,x2,x3) lcons_oid(x1, list_make2_oid(x2, x3))
#define list_make4_oid(x1,x2,x3,x4) lcons_oid(x1, list_make3_oid(x2, x3, x4))
+ #define list_make5_oid(x1,x2,x3,x4,x5) lcons_oid(x1, list_make4_oid(x2, x3, x4, x5))
/*
* foreach -
*** a/src/include/nodes/plannodes.h
--- b/src/include/nodes/plannodes.h
***************
*** 189,194 **** typedef struct ModifyTable
--- 189,195 ----
List *withCheckOptionLists; /* per-target-table WCO lists */
List *returningLists; /* per-target-table RETURNING tlists */
List *fdwPrivLists; /* per-target-table FDW private data lists */
+ Bitmapset *fdwDirectModifyPlans; /* indices of FDW DM plans */
List *rowMarks; /* PlanRowMarks (non-locking only) */
int epqParam; /* ID of Param for EvalPlanQual re-eval */
OnConflictAction onConflictAction; /* ON CONFLICT action */
***************
*** 531,536 **** typedef struct WorkTableScan
--- 532,538 ----
typedef struct ForeignScan
{
Scan scan;
+ CmdType operation; /* SELECT/INSERT/UPDATE/DELETE */
Oid fs_server; /* OID of foreign server */
List *fdw_exprs; /* expressions that FDW may evaluate */
List *fdw_private; /* private data for FDW */
*** a/src/include/optimizer/plancat.h
--- b/src/include/optimizer/plancat.h
***************
*** 55,58 **** extern Selectivity join_selectivity(PlannerInfo *root,
--- 55,60 ----
JoinType jointype,
SpecialJoinInfo *sjinfo);
+ extern bool has_row_triggers(PlannerInfo *root, Index rti, CmdType event);
+
#endif /* PLANCAT_H */
On Wed, Mar 9, 2016 at 3:30 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
Great! I changed the naming. I also updated docs as proposed by you in a
previous email, and rebased the patch to the latest HEAD. Please find
attached an updated version of the patch.
Thanks. The new naming looks much better (and better also than what I
suggested).
I see that you went and changed all of the places that tested for !=
CMD_SELECT and made them test for == CMD_INSERT || == CMD_UPDATE || ==
CMD_DELETE instead. I think that's the wrong direction. I think that
we should use the != CMD_SELECT version of the test everywhere.
That's a single test instead of three, so marginally faster, and maybe
marginally more future-proof.
I think deparsePushedDownUpdateSql should be renamed to use the new
"direct modify" naming, like deparseDirectUpdateSql, maybe.
I would suggest not numbering the tests in postgresPlanDirectModify.
That just becomes a nuisance to keep up to date as things change.
Overall, I think this is looking pretty good.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Robert Haas <robertmhaas@gmail.com> writes:
Overall, I think this is looking pretty good.
I hadn't been paying any attention to this thread, but I wonder whether
this entire approach isn't considerably inferior to what we can do now
with the planner pathification patch. To quote from the new docs:
PlanForeignModify and the other callbacks described in Section 54.2.3
are designed around the assumption that the foreign relation will be
scanned in the usual way and then individual row updates will be driven
by a local ModifyTable plan node. This approach is necessary for the
general case where an update requires reading local tables as well as
foreign tables. However, if the operation could be executed entirely by
the foreign server, the FDW could generate a path representing that and
insert it into the UPPERREL_FINAL upper relation, where it would
compete against the ModifyTable approach. This approach could also be
used to implement remote SELECT FOR UPDATE, rather than using the row
locking callbacks described in Section 54.2.4. Keep in mind that a path
inserted into UPPERREL_FINAL is responsible for implementing all
behavior of the query.
I don't really see anything that this patch does that wouldn't be better
done that way. And I'd kind of like to get a working example of that type
of path insertion into 9.6, so that we can find out if we need any hooks
or callbacks that aren't there today.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Mar 9, 2016 at 1:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Robert Haas <robertmhaas@gmail.com> writes:
Overall, I think this is looking pretty good.
I hadn't been paying any attention to this thread, but I wonder whether
this entire approach isn't considerably inferior to what we can do now
with the planner pathification patch. To quote from the new docs:PlanForeignModify and the other callbacks described in Section 54.2.3
are designed around the assumption that the foreign relation will be
scanned in the usual way and then individual row updates will be driven
by a local ModifyTable plan node. This approach is necessary for the
general case where an update requires reading local tables as well as
foreign tables. However, if the operation could be executed entirely by
the foreign server, the FDW could generate a path representing that and
insert it into the UPPERREL_FINAL upper relation, where it would
compete against the ModifyTable approach. This approach could also be
used to implement remote SELECT FOR UPDATE, rather than using the row
locking callbacks described in Section 54.2.4. Keep in mind that a path
inserted into UPPERREL_FINAL is responsible for implementing all
behavior of the query.I don't really see anything that this patch does that wouldn't be better
done that way. And I'd kind of like to get a working example of that type
of path insertion into 9.6, so that we can find out if we need any hooks
or callbacks that aren't there today.
Well, I guess I'm not quite seeing it. What do you have in mind?
Just taking a guess here, you might be thinking that instead of
something like this...
Update on public.ft2
-> Foreign Update on public.ft2
We could boil it all the way down to this:
Foreign Update on public.ft2
But can you, really? What if the UPDATE is targeting an inheritance
hierarchy containing some local tables and some remote tables?
Apologies if I've completely misunderstood what you have in mind here,
but you haven't really explained what you have in mind here.
IMHO, if you want to do something really cool with the new
pathification stuff, the thing to do would be pushing down aggregates
to foreign servers. A lot of people would be really happy to see
SELECT count(*) FROM ft ship the count operation to the remote side!
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Robert Haas <robertmhaas@gmail.com> writes:
On Wed, Mar 9, 2016 at 1:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
I hadn't been paying any attention to this thread, but I wonder whether
this entire approach isn't considerably inferior to what we can do now
with the planner pathification patch. To quote from the new docs:
Well, I guess I'm not quite seeing it. What do you have in mind?
Just taking a guess here, you might be thinking that instead of
something like this...
Update on public.ft2
-> Foreign Update on public.ft2
We could boil it all the way down to this:
Foreign Update on public.ft2
Exactly. I'm not claiming that that would be particularly faster, but
it would eliminate a whole bunch of seriously ugly stuff that's in
this patch.
But can you, really? What if the UPDATE is targeting an inheritance
hierarchy containing some local tables and some remote tables?
I don't really see why that couldn't be made to work. You'd end up
with ForeignUpdates on the remote tables and a ModifyTable handling
the rest.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Mar 9, 2016 at 3:30 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Robert Haas <robertmhaas@gmail.com> writes:
On Wed, Mar 9, 2016 at 1:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
I hadn't been paying any attention to this thread, but I wonder whether
this entire approach isn't considerably inferior to what we can do now
with the planner pathification patch. To quote from the new docs:Well, I guess I'm not quite seeing it. What do you have in mind?
Just taking a guess here, you might be thinking that instead of
something like this...Update on public.ft2
-> Foreign Update on public.ft2We could boil it all the way down to this:
Foreign Update on public.ft2
Exactly. I'm not claiming that that would be particularly faster, but
it would eliminate a whole bunch of seriously ugly stuff that's in
this patch.
Like what?
But can you, really? What if the UPDATE is targeting an inheritance
hierarchy containing some local tables and some remote tables?I don't really see why that couldn't be made to work. You'd end up
with ForeignUpdates on the remote tables and a ModifyTable handling
the rest.
I don't get it. I mean, what's the parent node going to be? If it's
the ModifyTable, then the plan tree looks the same as what this
already does. If not, then what?
Just to recap the history, this patch was rewritten, criticized by
Stephen and you and rewritten to match your feedback. Then, both of
you ignored it for a long time while I and others but a lot of work
into it. Now, we're up against the deadline for this release and you
don't like it again. Well, OK. If you want to rewrite it into some
form you think is better, I'm cool with that. But it would be really
unfair if this slipped out of this release after so much work has been
put into making it match the design ideas that *you* put forward just
because, at the eleventh hour, you now have new ones. Personally, I
think we should just commit the darned thing and you can rewrite it
later if you want. If you want to rewrite it now instead, I can live
with that, too. But let's figure out how we're going to move this
forward.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/03/10 19:51, Robert Haas wrote:
On Wed, Mar 9, 2016 at 3:30 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Robert Haas <robertmhaas@gmail.com> writes:
Just taking a guess here, you might be thinking that instead of
something like this...Update on public.ft2
-> Foreign Update on public.ft2We could boil it all the way down to this:
Foreign Update on public.ft2
Exactly. I'm not claiming that that would be particularly faster, but
it would eliminate a whole bunch of seriously ugly stuff that's in
this patch.
But can you, really? What if the UPDATE is targeting an inheritance
hierarchy containing some local tables and some remote tables?I don't really see why that couldn't be made to work. You'd end up
with ForeignUpdates on the remote tables and a ModifyTable handling
the rest.I don't get it. I mean, what's the parent node going to be? If it's
the ModifyTable, then the plan tree looks the same as what this
already does. If not, then what?
I don't get it, either. If the ForeignUpdates would be executed
separately from the ModifyTable, how would the query's reported row
count (ie, estate->es_processed) be handled?
Just to recap the history, this patch was rewritten, criticized by
Stephen and you and rewritten to match your feedback. Then, both of
you ignored it for a long time while I and others but a lot of work
into it. Now, we're up against the deadline for this release and you
don't like it again. Well, OK. If you want to rewrite it into some
form you think is better, I'm cool with that. But it would be really
unfair if this slipped out of this release after so much work has been
put into making it match the design ideas that *you* put forward just
because, at the eleventh hour, you now have new ones. Personally, I
think we should just commit the darned thing and you can rewrite it
later if you want. If you want to rewrite it now instead, I can live
with that, too. But let's figure out how we're going to move this
forward.
I agree with Robert.
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/03/10 2:56, Robert Haas wrote:
I see that you went and changed all of the places that tested for !=
CMD_SELECT and made them test for == CMD_INSERT || == CMD_UPDATE || ==
CMD_DELETE instead. I think that's the wrong direction. I think that
we should use the != CMD_SELECT version of the test everywhere.
That's a single test instead of three, so marginally faster, and maybe
marginally more future-proof.I think deparsePushedDownUpdateSql should be renamed to use the new
"direct modify" naming, like deparseDirectUpdateSql, maybe.I would suggest not numbering the tests in postgresPlanDirectModify.
That just becomes a nuisance to keep up to date as things change.
Agreed. I updated the patch to address these comments. Attached is the
updated version of the patch.
Best regards,
Etsuro Fujita
Attachments:
fdw-dml-pushdown-v11.patchapplication/x-patch; name=fdw-dml-pushdown-v11.patchDownload
*** a/contrib/postgres_fdw/deparse.c
--- b/contrib/postgres_fdw/deparse.c
***************
*** 1315,1320 **** deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 1315,1383 ----
}
/*
+ * deparse remote UPDATE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparseDirectUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ deparse_expr_cxt context;
+ int nestlevel;
+ bool first;
+ ListCell *lc;
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = params_list;
+
+ appendStringInfoString(buf, "UPDATE ");
+ deparseRelation(buf, rel);
+ appendStringInfoString(buf, " SET ");
+
+ /* Make sure any constants in the exprs are printed portably */
+ nestlevel = set_transmission_modes();
+
+ first = true;
+ foreach(lc, targetAttrs)
+ {
+ int attnum = lfirst_int(lc);
+ TargetEntry *tle = get_tle_by_resno(targetlist, attnum);
+
+ if (!first)
+ appendStringInfoString(buf, ", ");
+ first = false;
+
+ deparseColumnRef(buf, rtindex, attnum, root, false);
+ appendStringInfoString(buf, " = ");
+ deparseExpr((Expr *) tle->expr, &context);
+ }
+
+ reset_transmission_modes(nestlevel);
+
+ if (remote_conds)
+ {
+ appendStringInfo(buf, " WHERE ");
+ appendConditions(remote_conds, &context);
+ }
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* deparse remote DELETE statement
*
* The statement text is appended to buf, and we also create an integer List
***************
*** 1337,1342 **** deparseDeleteSql(StringInfo buf, PlannerInfo *root,
--- 1400,1442 ----
}
/*
+ * deparse remote DELETE statement
+ *
+ * The statement text is appended to buf, and we also create an integer List
+ * of the columns being retrieved by RETURNING (if any), which is returned
+ * to *retrieved_attrs.
+ */
+ void
+ deparseDirectDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[rtindex];
+ deparse_expr_cxt context;
+
+ /* Set up context struct for recursion */
+ context.root = root;
+ context.foreignrel = baserel;
+ context.buf = buf;
+ context.params_list = params_list;
+
+ appendStringInfoString(buf, "DELETE FROM ");
+ deparseRelation(buf, rel);
+
+ if (remote_conds)
+ {
+ appendStringInfo(buf, " WHERE ");
+ appendConditions(remote_conds, &context);
+ }
+
+ deparseReturningList(buf, root, rtindex, rel, false,
+ returningList, retrieved_attrs);
+ }
+
+ /*
* Add a RETURNING clause, if needed, to an INSERT/UPDATE/DELETE.
*/
static void
*** a/contrib/postgres_fdw/expected/postgres_fdw.out
--- b/contrib/postgres_fdw/expected/postgres_fdw.out
***************
*** 2259,2265 **** INSERT INTO ft2 (c1,c2,c3)
--- 2259,2284 ----
(3 rows)
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 300), c3 = (c3 || '_update3'::text) WHERE ((("C 1" % 10) = 3))
+ (3 rows)
+
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
+ QUERY PLAN
+ ------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ Update on public.ft2
+ Output: c1, c2, c3, c4, c5, c6, c7, c8
+ -> Foreign Update on public.ft2
+ Remote SQL: UPDATE "S 1"."T 1" SET c2 = (c2 + 400), c3 = (c3 || '_update7'::text) WHERE ((("C 1" % 10) = 7)) RETURNING "C 1", c2, c3, c4, c5, c6, c7, c8
+ (4 rows)
+
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
c1 | c2 | c3 | c4 | c5 | c6 | c7 | c8
------+-----+--------------------+------------------------------+--------------------------+----+------------+-----
***************
*** 2369,2375 **** UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
--- 2388,2394 ----
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update on public.ft2
***************
*** 2394,2409 **** UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
! QUERY PLAN
! ----------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1 RETURNING "C 1", c4
! -> Foreign Scan on public.ft2
! Output: ctid
! Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) FOR UPDATE
! (6 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
--- 2413,2426 ----
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
! QUERY PLAN
! --------------------------------------------------------------------------------------------
Delete on public.ft2
Output: c1, c4
! -> Foreign Delete on public.ft2
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ((("C 1" % 10) = 5)) RETURNING "C 1", c4
! (4 rows)
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
c1 | c4
***************
*** 2514,2520 **** DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
--- 2531,2537 ----
(103 rows)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Delete on public.ft2
***************
*** 3379,3394 **** INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
(1 row)
EXPLAIN (verbose, costs off)
! UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
! QUERY PLAN
! -------------------------------------------------------------------------------------------------------------------
Update on public.ft2
Output: (tableoid)::regclass
! Remote SQL: UPDATE "S 1"."T 1" SET c3 = $2 WHERE ctid = $1
! -> Foreign Scan on public.ft2
! Output: c1, c2, NULL::integer, 'bar'::text, c4, c5, c6, c7, c8, ctid
! Remote SQL: SELECT "C 1", c2, c4, c5, c6, c7, c8, ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
! (6 rows)
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
--- 3396,3409 ----
(1 row)
EXPLAIN (verbose, costs off)
! UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
! QUERY PLAN
! ------------------------------------------------------------------------------------
Update on public.ft2
Output: (tableoid)::regclass
! -> Foreign Update on public.ft2
! Remote SQL: UPDATE "S 1"."T 1" SET c3 = 'bar'::text WHERE (("C 1" = 9999))
! (4 rows)
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
***************
*** 3397,3412 **** UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
(1 row)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
! QUERY PLAN
! ------------------------------------------------------------------------------------
Delete on public.ft2
Output: (tableoid)::regclass
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE ctid = $1
! -> Foreign Scan on public.ft2
! Output: ctid
! Remote SQL: SELECT ctid FROM "S 1"."T 1" WHERE (("C 1" = 9999)) FOR UPDATE
! (6 rows)
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
--- 3412,3425 ----
(1 row)
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
! QUERY PLAN
! --------------------------------------------------------------------
Delete on public.ft2
Output: (tableoid)::regclass
! -> Foreign Delete on public.ft2
! Remote SQL: DELETE FROM "S 1"."T 1" WHERE (("C 1" = 9999))
! (4 rows)
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
tableoid
***************
*** 3560,3566 **** CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 3573,3579 ----
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
-- Test savepoint/rollback behavior
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 3719,3725 **** savepoint s3;
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
--- 3732,3738 ----
update ft2 set c2 = -2 where c2 = 42 and c1 = 10; -- fail on remote side
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (10, -2, 00010_trig_update_trig_update, 1970-01-11 08:00:00+00, 1970-01-11 00:00:00, 0, 0 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (-2) WHERE ((c2 = 42)) AND (("C 1" = 10))
rollback to savepoint s3;
select c2, count(*) from ft2 where c2 < 500 group by 1 order by 1;
c2 | count
***************
*** 3939,3945 **** CONTEXT: Remote SQL command: INSERT INTO "S 1"."T 1"("C 1", c2, c3, c4, c5, c6,
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = $2 WHERE ctid = $1
ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2positive;
-- But inconsistent check constraints provide inconsistent results
ALTER FOREIGN TABLE ft1 ADD CONSTRAINT ft1_c2negative CHECK (c2 < 0);
--- 3952,3958 ----
UPDATE ft1 SET c2 = -c2 WHERE c1 = 1; -- c2positive
ERROR: new row for relation "T 1" violates check constraint "c2positive"
DETAIL: Failing row contains (1, -1, 00001_trig_update, 1970-01-02 08:00:00+00, 1970-01-02 00:00:00, 1, 1 , foo).
! CONTEXT: Remote SQL command: UPDATE "S 1"."T 1" SET c2 = (- c2) WHERE (("C 1" = 1))
ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2positive;
-- But inconsistent check constraints provide inconsistent results
ALTER FOREIGN TABLE ft1 ADD CONSTRAINT ft1_c2negative CHECK (c2 < 0);
***************
*** 4332,4337 **** NOTICE: NEW: (13,"test triggered !")
--- 4345,4543 ----
(0,27)
(1 row)
+ -- cleanup
+ DROP TRIGGER trig_row_before ON rem1;
+ DROP TRIGGER trig_row_after ON rem1;
+ DROP TRIGGER trig_local_before ON loc1;
+ -- Test direct foreign table modification functionality
+ -- Test with statement-level triggers
+ CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_stmt_before ON rem1;
+ CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_stmt_after ON rem1;
+ -- Test with row-level ON INSERT triggers
+ CREATE TRIGGER trig_row_before_insert
+ BEFORE INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_before_insert ON rem1;
+ CREATE TRIGGER trig_row_after_insert
+ AFTER INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_after_insert ON rem1;
+ -- Test with row-level ON UPDATE triggers
+ CREATE TRIGGER trig_row_before_update
+ BEFORE UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_before_update ON rem1;
+ CREATE TRIGGER trig_row_after_update
+ AFTER UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ QUERY PLAN
+ -------------------------------------------------------------------------------
+ Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = $2 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: f1, ''::text, ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ QUERY PLAN
+ ---------------------------------------------
+ Delete on public.rem1
+ -> Foreign Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1
+ (3 rows)
+
+ DROP TRIGGER trig_row_after_update ON rem1;
+ -- Test with row-level ON DELETE triggers
+ CREATE TRIGGER trig_row_before_delete
+ BEFORE DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+ ---------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ DROP TRIGGER trig_row_before_delete ON rem1;
+ CREATE TRIGGER trig_row_after_delete
+ AFTER DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ QUERY PLAN
+ ----------------------------------------------------------
+ Update on public.rem1
+ -> Foreign Update on public.rem1
+ Remote SQL: UPDATE public.loc1 SET f2 = ''::text
+ (3 rows)
+
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ QUERY PLAN
+ ------------------------------------------------------------------------
+ Delete on public.rem1
+ Remote SQL: DELETE FROM public.loc1 WHERE ctid = $1 RETURNING f1, f2
+ -> Foreign Scan on public.rem1
+ Output: ctid, rem1.*
+ Remote SQL: SELECT f1, f2, ctid FROM public.loc1 FOR UPDATE
+ (5 rows)
+
+ DROP TRIGGER trig_row_after_delete ON rem1;
-- ===================================================================
-- test inheritance features
-- ===================================================================
***************
*** 4801,4806 **** fetch from c;
--- 5007,5062 ----
update bar set f2 = null where current of c;
ERROR: WHERE CURRENT OF is not supported for this table type
rollback;
+ explain (verbose, costs off)
+ delete from foo where f1 < 5 returning *;
+ QUERY PLAN
+ --------------------------------------------------------------------------------
+ Delete on public.foo
+ Output: foo.f1, foo.f2
+ Delete on public.foo
+ Foreign Delete on public.foo2
+ -> Index Scan using i_foo_f1 on public.foo
+ Output: foo.ctid
+ Index Cond: (foo.f1 < 5)
+ -> Foreign Delete on public.foo2
+ Remote SQL: DELETE FROM public.loct1 WHERE ((f1 < 5)) RETURNING f1, f2
+ (9 rows)
+
+ delete from foo where f1 < 5 returning *;
+ f1 | f2
+ ----+----
+ 1 | 1
+ 3 | 3
+ 0 | 0
+ 2 | 2
+ 4 | 4
+ (5 rows)
+
+ explain (verbose, costs off)
+ update bar set f2 = f2 + 100 returning *;
+ QUERY PLAN
+ ------------------------------------------------------------------------------
+ Update on public.bar
+ Output: bar.f1, bar.f2
+ Update on public.bar
+ Foreign Update on public.bar2
+ -> Seq Scan on public.bar
+ Output: bar.f1, (bar.f2 + 100), bar.ctid
+ -> Foreign Update on public.bar2
+ Remote SQL: UPDATE public.loct2 SET f2 = (f2 + 100) RETURNING f1, f2
+ (8 rows)
+
+ update bar set f2 = f2 + 100 returning *;
+ f1 | f2
+ ----+-----
+ 1 | 311
+ 2 | 322
+ 6 | 266
+ 3 | 333
+ 4 | 344
+ 7 | 277
+ (6 rows)
+
drop table foo cascade;
NOTICE: drop cascades to foreign table foo2
drop table bar cascade;
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
***************
*** 61,66 **** enum FdwScanPrivateIndex
--- 61,68 ----
{
/* SQL statement to execute remotely (as a String node) */
FdwScanPrivateSelectSql,
+ /* List of restriction clauses that can be executed remotely */
+ FdwScanPrivateRemoteConds,
/* Integer list of attribute numbers retrieved by the SELECT */
FdwScanPrivateRetrievedAttrs,
/* Integer representing the desired fetch_size */
***************
*** 98,103 **** enum FdwModifyPrivateIndex
--- 100,126 ----
};
/*
+ * Similarly, this enum describes what's kept in the fdw_private list for
+ * a ForeignScan node that modifies a foreign table directly. We store:
+ *
+ * 1) UPDATE/DELETE statement text to be sent to the remote server
+ * 2) Boolean flag showing if the remote query has a RETURNING clause
+ * 3) Integer list of attribute numbers retrieved by RETURNING, if any
+ * 4) Boolean flag showing if we set the command es_processed
+ */
+ enum FdwDirectModifyPrivateIndex
+ {
+ /* SQL statement to execute remotely (as a String node) */
+ FdwDirectModifyPrivateUpdateSql,
+ /* has-returning flag (as an integer Value node) */
+ FdwDirectModifyPrivateHasReturning,
+ /* Integer list of attribute numbers retrieved by RETURNING */
+ FdwDirectModifyPrivateRetrievedAttrs,
+ /* set-processed flag (as an integer Value node) */
+ FdwDirectModifyPrivateSetProcessed
+ };
+
+ /*
* Execution state of a foreign scan using postgres_fdw.
*/
typedef struct PgFdwScanState
***************
*** 164,169 **** typedef struct PgFdwModifyState
--- 187,222 ----
} PgFdwModifyState;
/*
+ * Execution state of a foreign scan that modifies a foreign table directly.
+ */
+ typedef struct PgFdwDirectModifyState
+ {
+ Relation rel; /* relcache entry for the foreign table */
+ AttInMetadata *attinmeta; /* attribute datatype conversion metadata */
+
+ /* extracted fdw_private data */
+ char *query; /* text of UPDATE/DELETE command */
+ bool has_returning; /* is there a RETURNING clause? */
+ List *retrieved_attrs; /* attr numbers retrieved by RETURNING */
+ bool set_processed; /* do we set the command es_processed? */
+
+ /* for remote query execution */
+ PGconn *conn; /* connection for the update */
+ int numParams; /* number of parameters passed to query */
+ FmgrInfo *param_flinfo; /* output conversion functions for them */
+ List *param_exprs; /* executable expressions for param values */
+ const char **param_values; /* textual values of query parameters */
+
+ /* for storing result tuples */
+ PGresult *result; /* result for query */
+ int num_tuples; /* # of result tuples */
+ int next_tuple; /* index of next one to return */
+
+ /* working memory context */
+ MemoryContext temp_cxt; /* context for per-tuple temporary data */
+ } PgFdwDirectModifyState;
+
+ /*
* Workspace for analyzing a foreign table.
*/
typedef struct PgFdwAnalyzeState
***************
*** 263,268 **** static TupleTableSlot *postgresExecForeignDelete(EState *estate,
--- 316,328 ----
static void postgresEndForeignModify(EState *estate,
ResultRelInfo *resultRelInfo);
static int postgresIsForeignRelUpdatable(Relation rel);
+ static bool postgresPlanDirectModify(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+ static void postgresBeginDirectModify(ForeignScanState *node, int eflags);
+ static TupleTableSlot *postgresIterateDirectModify(ForeignScanState *node);
+ static void postgresEndDirectModify(ForeignScanState *node);
static void postgresExplainForeignScan(ForeignScanState *node,
ExplainState *es);
static void postgresExplainForeignModify(ModifyTableState *mtstate,
***************
*** 270,275 **** static void postgresExplainForeignModify(ModifyTableState *mtstate,
--- 330,337 ----
List *fdw_private,
int subplan_index,
ExplainState *es);
+ static void postgresExplainDirectModify(ForeignScanState *node,
+ ExplainState *es);
static bool postgresAnalyzeForeignTable(Relation relation,
AcquireSampleRowsFunc *func,
BlockNumber *totalpages);
***************
*** 311,316 **** static const char **convert_prep_stmt_params(PgFdwModifyState *fmstate,
--- 373,390 ----
TupleTableSlot *slot);
static void store_returning_result(PgFdwModifyState *fmstate,
TupleTableSlot *slot, PGresult *res);
+ static void execute_dml_stmt(ForeignScanState *node);
+ static TupleTableSlot *get_returning_data(ForeignScanState *node);
+ static void prepare_query_params(PlanState *node,
+ List *fdw_exprs,
+ int numParams,
+ FmgrInfo **param_flinfo,
+ List **param_exprs,
+ const char ***param_values);
+ static void process_query_params(ExprContext *econtext,
+ FmgrInfo *param_flinfo,
+ List *param_exprs,
+ const char **param_values);
static int postgresAcquireSampleRowsFunc(Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
***************
*** 362,373 **** postgres_fdw_handler(PG_FUNCTION_ARGS)
--- 436,452 ----
routine->ExecForeignDelete = postgresExecForeignDelete;
routine->EndForeignModify = postgresEndForeignModify;
routine->IsForeignRelUpdatable = postgresIsForeignRelUpdatable;
+ routine->PlanDirectModify = postgresPlanDirectModify;
+ routine->BeginDirectModify = postgresBeginDirectModify;
+ routine->IterateDirectModify = postgresIterateDirectModify;
+ routine->EndDirectModify = postgresEndDirectModify;
/* Function for EvalPlanQual rechecks */
routine->RecheckForeignScan = postgresRecheckForeignScan;
/* Support functions for EXPLAIN */
routine->ExplainForeignScan = postgresExplainForeignScan;
routine->ExplainForeignModify = postgresExplainForeignModify;
+ routine->ExplainDirectModify = postgresExplainDirectModify;
/* Support functions for ANALYZE */
routine->AnalyzeForeignTable = postgresAnalyzeForeignTable;
***************
*** 1122,1128 **** postgresGetForeignPlan(PlannerInfo *root,
* Build the fdw_private list that will be available to the executor.
* Items in the list must match order in enum FdwScanPrivateIndex.
*/
! fdw_private = list_make4(makeString(sql.data),
retrieved_attrs,
makeInteger(fpinfo->fetch_size),
makeInteger(foreignrel->umid));
--- 1201,1208 ----
* Build the fdw_private list that will be available to the executor.
* Items in the list must match order in enum FdwScanPrivateIndex.
*/
! fdw_private = list_make5(makeString(sql.data),
! remote_conds,
retrieved_attrs,
makeInteger(fpinfo->fetch_size),
makeInteger(foreignrel->umid));
***************
*** 1159,1166 **** postgresBeginForeignScan(ForeignScanState *node, int eflags)
PgFdwScanState *fsstate;
UserMapping *user;
int numParams;
- int i;
- ListCell *lc;
/*
* Do nothing in EXPLAIN (no ANALYZE) case. node->fdw_state stays NULL.
--- 1239,1244 ----
***************
*** 1247,1288 **** postgresBeginForeignScan(ForeignScanState *node, int eflags)
fsstate->attinmeta = TupleDescGetAttInMetadata(fsstate->tupdesc);
- /* Prepare for output conversion of parameters used in remote query. */
- numParams = list_length(fsplan->fdw_exprs);
- fsstate->numParams = numParams;
- fsstate->param_flinfo = (FmgrInfo *) palloc0(sizeof(FmgrInfo) * numParams);
-
- i = 0;
- foreach(lc, fsplan->fdw_exprs)
- {
- Node *param_expr = (Node *) lfirst(lc);
- Oid typefnoid;
- bool isvarlena;
-
- getTypeOutputInfo(exprType(param_expr), &typefnoid, &isvarlena);
- fmgr_info(typefnoid, &fsstate->param_flinfo[i]);
- i++;
- }
-
/*
! * Prepare remote-parameter expressions for evaluation. (Note: in
! * practice, we expect that all these expressions will be just Params, so
! * we could possibly do something more efficient than using the full
! * expression-eval machinery for this. But probably there would be little
! * benefit, and it'd require postgres_fdw to know more than is desirable
! * about Param evaluation.)
! */
! fsstate->param_exprs = (List *)
! ExecInitExpr((Expr *) fsplan->fdw_exprs,
! (PlanState *) node);
!
! /*
! * Allocate buffer for text form of query parameters, if any.
*/
if (numParams > 0)
! fsstate->param_values = (const char **) palloc0(numParams * sizeof(char *));
! else
! fsstate->param_values = NULL;
}
/*
--- 1325,1342 ----
fsstate->attinmeta = TupleDescGetAttInMetadata(fsstate->tupdesc);
/*
! * Prepare for processing of parameters used in remote query, if any.
*/
+ numParams = list_length(fsplan->fdw_exprs);
+ fsstate->numParams = numParams;
if (numParams > 0)
! prepare_query_params((PlanState *) node,
! fsplan->fdw_exprs,
! numParams,
! &fsstate->param_flinfo,
! &fsstate->param_exprs,
! &fsstate->param_values);
}
/*
***************
*** 1447,1459 **** postgresAddForeignUpdateTargets(Query *parsetree,
/*
* postgresPlanForeignModify
* Plan an insert/update/delete operation on a foreign table
- *
- * Note: currently, the plan tree generated for UPDATE/DELETE will always
- * include a ForeignScan that retrieves ctids (using SELECT FOR UPDATE)
- * and then the ModifyTable node will have to execute individual remote
- * UPDATE/DELETE commands. If there are no local conditions or joins
- * needed, it'd be better to let the scan node do UPDATE/DELETE RETURNING
- * and then do nothing at ModifyTable. Room for future optimization ...
*/
static List *
postgresPlanForeignModify(PlannerInfo *root,
--- 1501,1506 ----
***************
*** 1992,1997 **** postgresRecheckForeignScan(ForeignScanState *node, TupleTableSlot *slot)
--- 2039,2352 ----
}
/*
+ * postgresPlanDirectModify
+ * Consider a direct foreign table modification
+ *
+ * Decide whether it is safe to modify a foreign table directly, and if so,
+ * rewrite subplan accordingly.
+ */
+ static bool
+ postgresPlanDirectModify(PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index)
+ {
+ CmdType operation = plan->operation;
+ Plan *subplan = (Plan *) list_nth(plan->plans, subplan_index);
+ RangeTblEntry *rte = planner_rt_fetch(resultRelation, root);
+ Relation rel;
+ StringInfoData sql;
+ ForeignScan *fscan;
+ List *targetAttrs = NIL;
+ List *remote_conds;
+ List *params_list = NIL;
+ List *returningList = NIL;
+ List *retrieved_attrs = NIL;
+
+ /*
+ * Decide whether it is safe to modify a foreign table directly.
+ */
+
+ /*
+ * The table modification must be an UPDATE or DELETE.
+ */
+ if (operation != CMD_UPDATE && operation != CMD_DELETE)
+ return false;
+
+ /*
+ * It's unsafe to modify a foreign table directly if there are any local
+ * joins needed.
+ */
+ if (!IsA(subplan, ForeignScan))
+ return false;
+
+ /*
+ * It's unsafe to modify a foreign table directly if there are any quals
+ * that should be evaluated locally.
+ */
+ if (subplan->qual != NIL)
+ return false;
+
+ /*
+ * We can't handle an UPDATE or DELETE on a foreign join for now.
+ */
+ fscan = (ForeignScan *) subplan;
+ if (fscan->scan.scanrelid == 0)
+ return false;
+
+ /*
+ * It's unsafe to update a foreign table directly, if any expressions to
+ * assign to the target columns are unsafe to evaluate remotely.
+ */
+ if (operation == CMD_UPDATE)
+ {
+ RelOptInfo *baserel = root->simple_rel_array[resultRelation];
+ int col;
+
+ /*
+ * We transmit only columns that were explicitly targets of the UPDATE,
+ * so as to avoid unnecessary data transmission.
+ */
+ col = -1;
+ while ((col = bms_next_member(rte->updatedCols, col)) >= 0)
+ {
+ /* bit numbers are offset by FirstLowInvalidHeapAttributeNumber */
+ AttrNumber attno = col + FirstLowInvalidHeapAttributeNumber;
+ TargetEntry *tle;
+
+ if (attno <= InvalidAttrNumber) /* shouldn't happen */
+ elog(ERROR, "system-column update is not supported");
+
+ tle = get_tle_by_resno(subplan->targetlist, attno);
+
+ if (!is_foreign_expr(root, baserel, (Expr *) tle->expr))
+ return false;
+
+ targetAttrs = lappend_int(targetAttrs, attno);
+ }
+ }
+
+ /*
+ * Ok, rewrite subplan so as to modify the foreign table directly.
+ */
+ initStringInfo(&sql);
+
+ /*
+ * Core code already has some lock on each rel being planned, so we can
+ * use NoLock here.
+ */
+ rel = heap_open(rte->relid, NoLock);
+
+ /*
+ * Extract the baserestrictinfo clauses that can be evaluated remotely.
+ */
+ remote_conds = (List *) list_nth(fscan->fdw_private,
+ FdwScanPrivateRemoteConds);
+
+ /*
+ * Extract the relevant RETURNING list if any.
+ */
+ if (plan->returningLists)
+ returningList = (List *) list_nth(plan->returningLists, subplan_index);
+
+ /*
+ * Construct the SQL command string.
+ */
+ switch (operation)
+ {
+ case CMD_UPDATE:
+ deparseDirectUpdateSql(&sql, root, resultRelation, rel,
+ ((Plan *) fscan)->targetlist,
+ targetAttrs,
+ remote_conds, ¶ms_list,
+ returningList, &retrieved_attrs);
+ break;
+ case CMD_DELETE:
+ deparseDirectDeleteSql(&sql, root, resultRelation, rel,
+ remote_conds, ¶ms_list,
+ returningList, &retrieved_attrs);
+ break;
+ default:
+ elog(ERROR, "unexpected operation: %d", (int) operation);
+ break;
+ }
+
+ /*
+ * Update the operation info.
+ */
+ fscan->operation = operation;
+
+ /*
+ * Update the fdw_exprs list that will be available to the executor.
+ */
+ fscan->fdw_exprs = params_list;
+
+ /*
+ * Update the fdw_private list that will be available to the executor.
+ * Items in the list must match enum FdwDirectModifyPrivateIndex, above.
+ */
+ fscan->fdw_private = list_make4(makeString(sql.data),
+ makeInteger((retrieved_attrs != NIL)),
+ retrieved_attrs,
+ makeInteger(plan->canSetTag));
+
+ heap_close(rel, NoLock);
+ return true;
+ }
+
+ /*
+ * postgresBeginDirectModify
+ * Prepare a direct foreign table modification
+ */
+ static void
+ postgresBeginDirectModify(ForeignScanState *node, int eflags)
+ {
+ ForeignScan *fsplan = (ForeignScan *) node->ss.ps.plan;
+ EState *estate = node->ss.ps.state;
+ PgFdwDirectModifyState *dmstate;
+ RangeTblEntry *rte;
+ Oid userid;
+ ForeignTable *table;
+ UserMapping *user;
+ int numParams;
+
+ /*
+ * Do nothing in EXPLAIN (no ANALYZE) case. node->fdw_state stays NULL.
+ */
+ if (eflags & EXEC_FLAG_EXPLAIN_ONLY)
+ return;
+
+ /*
+ * We'll save private state in node->fdw_state.
+ */
+ dmstate = (PgFdwDirectModifyState *) palloc0(sizeof(PgFdwDirectModifyState));
+ node->fdw_state = (void *) dmstate;
+
+ /*
+ * Identify which user to do the remote access as. This should match what
+ * ExecCheckRTEPerms() does.
+ */
+ rte = rt_fetch(fsplan->scan.scanrelid, estate->es_range_table);
+ userid = rte->checkAsUser ? rte->checkAsUser : GetUserId();
+
+ /* Get info about foreign table. */
+ dmstate->rel = node->ss.ss_currentRelation;
+ table = GetForeignTable(RelationGetRelid(dmstate->rel));
+ user = GetUserMapping(userid, table->serverid);
+
+ /*
+ * Get connection to the foreign server. Connection manager will
+ * establish new connection if necessary.
+ */
+ dmstate->conn = GetConnection(user, false);
+
+ /* Initialize state variable */
+ dmstate->num_tuples = -1; /* -1 means not set yet */
+
+ /* Get private info created by planner functions. */
+ dmstate->query = strVal(list_nth(fsplan->fdw_private,
+ FdwDirectModifyPrivateUpdateSql));
+ dmstate->has_returning = intVal(list_nth(fsplan->fdw_private,
+ FdwDirectModifyPrivateHasReturning));
+ dmstate->retrieved_attrs = (List *) list_nth(fsplan->fdw_private,
+ FdwDirectModifyPrivateRetrievedAttrs);
+ dmstate->set_processed = intVal(list_nth(fsplan->fdw_private,
+ FdwDirectModifyPrivateSetProcessed));
+
+ /* Create context for per-tuple temp workspace. */
+ dmstate->temp_cxt = AllocSetContextCreate(estate->es_query_cxt,
+ "postgres_fdw temporary data",
+ ALLOCSET_SMALL_MINSIZE,
+ ALLOCSET_SMALL_INITSIZE,
+ ALLOCSET_SMALL_MAXSIZE);
+
+ /* Prepare for input conversion of RETURNING results. */
+ if (dmstate->has_returning)
+ dmstate->attinmeta = TupleDescGetAttInMetadata(RelationGetDescr(dmstate->rel));
+
+ /*
+ * Prepare for processing of parameters used in remote query, if any.
+ */
+ numParams = list_length(fsplan->fdw_exprs);
+ dmstate->numParams = numParams;
+ if (numParams > 0)
+ prepare_query_params((PlanState *) node,
+ fsplan->fdw_exprs,
+ numParams,
+ &dmstate->param_flinfo,
+ &dmstate->param_exprs,
+ &dmstate->param_values);
+ }
+
+ /*
+ * postgresIterateDirectModify
+ * Execute a direct foreign table modification
+ */
+ static TupleTableSlot *
+ postgresIterateDirectModify(ForeignScanState *node)
+ {
+ PgFdwDirectModifyState *dmstate = (PgFdwDirectModifyState *) node->fdw_state;
+ EState *estate = node->ss.ps.state;
+ ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
+
+ /*
+ * If this is the first call after Begin, execute the statement.
+ */
+ if (dmstate->num_tuples == -1)
+ execute_dml_stmt(node);
+
+ /*
+ * If the local query doesn't specify RETURNING, just clear tuple slot.
+ */
+ if (!resultRelInfo->ri_projectReturning)
+ {
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+ Instrumentation *instr = node->ss.ps.instrument;
+
+ Assert(!dmstate->has_returning);
+
+ /* Increment the command es_processed count if necessary. */
+ if (dmstate->set_processed)
+ estate->es_processed += dmstate->num_tuples;
+
+ /* Increment the tuple count for EXPLAIN ANALYZE if necessary. */
+ if (instr)
+ instr->tuplecount += dmstate->num_tuples;
+
+ return ExecClearTuple(slot);
+ }
+
+ /*
+ * Get the next RETURNING tuple.
+ */
+ return get_returning_data(node);
+ }
+
+ /*
+ * postgresEndDirectModify
+ * Finish a direct foreign table modification
+ */
+ static void
+ postgresEndDirectModify(ForeignScanState *node)
+ {
+ PgFdwDirectModifyState *dmstate = (PgFdwDirectModifyState *) node->fdw_state;
+
+ /* if dmstate is NULL, we are in EXPLAIN; nothing to do */
+ if (dmstate == NULL)
+ return;
+
+ /* Release PGresult */
+ if (dmstate->result)
+ PQclear(dmstate->result);
+
+ /* Release remote connection */
+ ReleaseConnection(dmstate->conn);
+ dmstate->conn = NULL;
+
+ /* MemoryContext will be deleted automatically. */
+ }
+
+ /*
* postgresExplainForeignScan
* Produce extra output for EXPLAIN of a ForeignScan on a foreign table
*/
***************
*** 2044,2049 **** postgresExplainForeignModify(ModifyTableState *mtstate,
--- 2399,2423 ----
}
}
+ /*
+ * postgresExplainDirectModify
+ * Produce extra output for EXPLAIN of a ForeignScan that modifies a
+ * foreign table directly
+ */
+ static void
+ postgresExplainDirectModify(ForeignScanState *node, ExplainState *es)
+ {
+ List *fdw_private;
+ char *sql;
+
+ if (es->verbose)
+ {
+ fdw_private = ((ForeignScan *) node->ss.ps.plan)->fdw_private;
+ sql = strVal(list_nth(fdw_private, FdwDirectModifyPrivateUpdateSql));
+ ExplainPropertyText("Remote SQL", sql, es);
+ }
+ }
+
/*
* estimate_path_cost_size
***************
*** 2419,2456 **** create_cursor(ForeignScanState *node)
*/
if (numParams > 0)
{
- int nestlevel;
MemoryContext oldcontext;
- int i;
- ListCell *lc;
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! nestlevel = set_transmission_modes();
!
! i = 0;
! foreach(lc, fsstate->param_exprs)
! {
! ExprState *expr_state = (ExprState *) lfirst(lc);
! Datum expr_value;
! bool isNull;
!
! /* Evaluate the parameter expression */
! expr_value = ExecEvalExpr(expr_state, econtext, &isNull, NULL);
!
! /*
! * Get string representation of each parameter value by invoking
! * type-specific output function, unless the value is null.
! */
! if (isNull)
! values[i] = NULL;
! else
! values[i] = OutputFunctionCall(&fsstate->param_flinfo[i],
! expr_value);
! i++;
! }
!
! reset_transmission_modes(nestlevel);
MemoryContextSwitchTo(oldcontext);
}
--- 2793,2806 ----
*/
if (numParams > 0)
{
MemoryContext oldcontext;
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! process_query_params(econtext,
! fsstate->param_flinfo,
! fsstate->param_exprs,
! values);
MemoryContextSwitchTo(oldcontext);
}
***************
*** 2771,2776 **** store_returning_result(PgFdwModifyState *fmstate,
--- 3121,3317 ----
}
/*
+ * Execute a direct UPDATE/DELETE statement.
+ */
+ static void
+ execute_dml_stmt(ForeignScanState *node)
+ {
+ PgFdwDirectModifyState *dmstate = (PgFdwDirectModifyState *) node->fdw_state;
+ ExprContext *econtext = node->ss.ps.ps_ExprContext;
+ int numParams = dmstate->numParams;
+ const char **values = dmstate->param_values;
+
+ /*
+ * Construct array of query parameter values in text format.
+ */
+ if (numParams > 0)
+ process_query_params(econtext,
+ dmstate->param_flinfo,
+ dmstate->param_exprs,
+ values);
+
+ /*
+ * Notice that we pass NULL for paramTypes, thus forcing the remote server
+ * to infer types for all parameters. Since we explicitly cast every
+ * parameter (see deparse.c), the "inference" is trivial and will produce
+ * the desired result. This allows us to avoid assuming that the remote
+ * server has the same OIDs we do for the parameters' types.
+ *
+ * We don't use a PG_TRY block here, so be careful not to throw error
+ * without releasing the PGresult.
+ */
+ dmstate->result = PQexecParams(dmstate->conn, dmstate->query,
+ numParams, NULL, values, NULL, NULL, 0);
+ if (PQresultStatus(dmstate->result) !=
+ (dmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
+ pgfdw_report_error(ERROR, dmstate->result, dmstate->conn, true,
+ dmstate->query);
+
+ /* Get the number of rows affected. */
+ if (dmstate->has_returning)
+ dmstate->num_tuples = PQntuples(dmstate->result);
+ else
+ dmstate->num_tuples = atoi(PQcmdTuples(dmstate->result));
+ }
+
+ /*
+ * Get the result of a RETURNING clause.
+ */
+ static TupleTableSlot *
+ get_returning_data(ForeignScanState *node)
+ {
+ PgFdwDirectModifyState *dmstate = (PgFdwDirectModifyState *) node->fdw_state;
+ EState *estate = node->ss.ps.state;
+ ResultRelInfo *resultRelInfo = estate->es_result_relation_info;
+ TupleTableSlot *slot = node->ss.ss_ScanTupleSlot;
+
+ Assert(resultRelInfo->ri_projectReturning);
+
+ /* If we didn't get any tuples, must be end of data. */
+ if (dmstate->next_tuple >= dmstate->num_tuples)
+ return ExecClearTuple(slot);
+
+ /* Increment the command es_processed count if necessary. */
+ if (dmstate->set_processed)
+ estate->es_processed += 1;
+
+ /*
+ * Store a RETURNING tuple. If has_returning is false, just emit a dummy
+ * tuple. (has_returning is false when the local query is of the form
+ * "UPDATE/DELETE .. RETURNING 1" for example.)
+ */
+ if (!dmstate->has_returning)
+ ExecStoreAllNullTuple(slot);
+ else
+ {
+ /*
+ * On error, be sure to release the PGresult on the way out. Callers
+ * do not have PG_TRY blocks to ensure this happens.
+ */
+ PG_TRY();
+ {
+ HeapTuple newtup;
+
+ newtup = make_tuple_from_result_row(dmstate->result,
+ dmstate->next_tuple,
+ dmstate->rel,
+ dmstate->attinmeta,
+ dmstate->retrieved_attrs,
+ NULL,
+ dmstate->temp_cxt);
+ ExecStoreTuple(newtup, slot, InvalidBuffer, false);
+ }
+ PG_CATCH();
+ {
+ if (dmstate->result)
+ PQclear(dmstate->result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ }
+ dmstate->next_tuple++;
+
+ /* Make slot available for evaluation of the local query RETURNING list. */
+ resultRelInfo->ri_projectReturning->pi_exprContext->ecxt_scantuple = slot;
+
+ return slot;
+ }
+
+ /*
+ * Prepare for processing of parameters used in remote query.
+ */
+ static void
+ prepare_query_params(PlanState *node,
+ List *fdw_exprs,
+ int numParams,
+ FmgrInfo **param_flinfo,
+ List **param_exprs,
+ const char ***param_values)
+ {
+ int i;
+ ListCell *lc;
+
+ Assert(numParams > 0);
+
+ /* Prepare for output conversion of parameters used in remote query. */
+ *param_flinfo = (FmgrInfo *) palloc0(sizeof(FmgrInfo) * numParams);
+
+ i = 0;
+ foreach(lc, fdw_exprs)
+ {
+ Node *param_expr = (Node *) lfirst(lc);
+ Oid typefnoid;
+ bool isvarlena;
+
+ getTypeOutputInfo(exprType(param_expr), &typefnoid, &isvarlena);
+ fmgr_info(typefnoid, &(*param_flinfo)[i]);
+ i++;
+ }
+
+ /*
+ * Prepare remote-parameter expressions for evaluation. (Note: in
+ * practice, we expect that all these expressions will be just Params, so
+ * we could possibly do something more efficient than using the full
+ * expression-eval machinery for this. But probably there would be little
+ * benefit, and it'd require postgres_fdw to know more than is desirable
+ * about Param evaluation.)
+ */
+ *param_exprs = (List *) ExecInitExpr((Expr *) fdw_exprs, node);
+
+ /* Allocate buffer for text form of query parameters. */
+ *param_values = (const char **) palloc0(numParams * sizeof(char *));
+ }
+
+ /*
+ * Construct array of query parameter values in text format.
+ */
+ static void
+ process_query_params(ExprContext *econtext,
+ FmgrInfo *param_flinfo,
+ List *param_exprs,
+ const char **param_values)
+ {
+ int nestlevel;
+ int i;
+ ListCell *lc;
+
+ nestlevel = set_transmission_modes();
+
+ i = 0;
+ foreach(lc, param_exprs)
+ {
+ ExprState *expr_state = (ExprState *) lfirst(lc);
+ Datum expr_value;
+ bool isNull;
+
+ /* Evaluate the parameter expression */
+ expr_value = ExecEvalExpr(expr_state, econtext, &isNull, NULL);
+
+ /*
+ * Get string representation of each parameter value by invoking
+ * type-specific output function, unless the value is null.
+ */
+ if (isNull)
+ param_values[i] = NULL;
+ else
+ param_values[i] = OutputFunctionCall(¶m_flinfo[i], expr_value);
+ i++;
+ }
+
+ reset_transmission_modes(nestlevel);
+ }
+
+ /*
* postgresAnalyzeForeignTable
* Test whether analyzing this foreign table is supported
*/
*** a/contrib/postgres_fdw/postgres_fdw.h
--- b/contrib/postgres_fdw/postgres_fdw.h
***************
*** 130,139 **** extern void deparseUpdateSql(StringInfo buf, PlannerInfo *root,
--- 130,153 ----
Index rtindex, Relation rel,
List *targetAttrs, List *returningList,
List **retrieved_attrs);
+ extern void deparseDirectUpdateSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *targetlist,
+ List *targetAttrs,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseDeleteSql(StringInfo buf, PlannerInfo *root,
Index rtindex, Relation rel,
List *returningList,
List **retrieved_attrs);
+ extern void deparseDirectDeleteSql(StringInfo buf, PlannerInfo *root,
+ Index rtindex, Relation rel,
+ List *remote_conds,
+ List **params_list,
+ List *returningList,
+ List **retrieved_attrs);
extern void deparseAnalyzeSizeSql(StringInfo buf, Relation rel);
extern void deparseAnalyzeSql(StringInfo buf, Relation rel,
List **retrieved_attrs);
*** a/contrib/postgres_fdw/sql/postgres_fdw.sql
--- b/contrib/postgres_fdw/sql/postgres_fdw.sql
***************
*** 604,631 **** INSERT INTO ft2 (c1,c2,c3) SELECT c1+1000,c2+100, c3 || c3 FROM ft2 LIMIT 20;
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
EXPLAIN (verbose, costs off)
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
! UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
-- Test that trigger on remote table works as expected
--- 604,635 ----
INSERT INTO ft2 (c1,c2,c3)
VALUES (1101,201,'aaa'), (1102,202,'bbb'), (1103,203,'ccc') RETURNING *;
INSERT INTO ft2 (c1,c2,c3) VALUES (1104,204,'ddd'), (1105,205,'eee');
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 300, c3 = c3 || '_update3' WHERE c1 % 10 = 3;
+ EXPLAIN (verbose, costs off)
+ UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *; -- can be pushed down
UPDATE ft2 SET c2 = c2 + 400, c3 = c3 || '_update7' WHERE c1 % 10 = 7 RETURNING *;
EXPLAIN (verbose, costs off)
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
! FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9; -- can't be pushed down
UPDATE ft2 SET c2 = ft2.c2 + 500, c3 = ft2.c3 || '_update9', c7 = DEFAULT
FROM ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 9;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4; -- can be pushed down
DELETE FROM ft2 WHERE c1 % 10 = 5 RETURNING c1, c4;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2; -- can't be pushed down
DELETE FROM ft2 USING ft1 WHERE ft1.c1 = ft2.c2 AND ft1.c1 % 10 = 2;
SELECT c1,c2,c3,c4 FROM ft2 ORDER BY c1;
EXPLAIN (verbose, costs off)
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
INSERT INTO ft2 (c1,c2,c3) VALUES (9999,999,'foo') RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
! UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
UPDATE ft2 SET c3 = 'bar' WHERE c1 = 9999 RETURNING tableoid::regclass;
EXPLAIN (verbose, costs off)
! DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass; -- can be pushed down
DELETE FROM ft2 WHERE c1 = 9999 RETURNING tableoid::regclass;
-- Test that trigger on remote table works as expected
***************
*** 954,959 **** UPDATE rem1 SET f2 = 'testo';
--- 958,1047 ----
-- Test returning a system attribute
INSERT INTO rem1(f2) VALUES ('test') RETURNING ctid;
+ -- cleanup
+ DROP TRIGGER trig_row_before ON rem1;
+ DROP TRIGGER trig_row_after ON rem1;
+ DROP TRIGGER trig_local_before ON loc1;
+
+
+ -- Test direct foreign table modification functionality
+
+ -- Test with statement-level triggers
+ CREATE TRIGGER trig_stmt_before
+ BEFORE DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_stmt_before ON rem1;
+
+ CREATE TRIGGER trig_stmt_after
+ AFTER DELETE OR INSERT OR UPDATE ON rem1
+ FOR EACH STATEMENT EXECUTE PROCEDURE trigger_func();
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_stmt_after ON rem1;
+
+ -- Test with row-level ON INSERT triggers
+ CREATE TRIGGER trig_row_before_insert
+ BEFORE INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_before_insert ON rem1;
+
+ CREATE TRIGGER trig_row_after_insert
+ AFTER INSERT ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_after_insert ON rem1;
+
+ -- Test with row-level ON UPDATE triggers
+ CREATE TRIGGER trig_row_before_update
+ BEFORE UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_before_update ON rem1;
+
+ CREATE TRIGGER trig_row_after_update
+ AFTER UPDATE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can't be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can be pushed down
+ DROP TRIGGER trig_row_after_update ON rem1;
+
+ -- Test with row-level ON DELETE triggers
+ CREATE TRIGGER trig_row_before_delete
+ BEFORE DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ DROP TRIGGER trig_row_before_delete ON rem1;
+
+ CREATE TRIGGER trig_row_after_delete
+ AFTER DELETE ON rem1
+ FOR EACH ROW EXECUTE PROCEDURE trigger_data(23,'skidoo');
+ EXPLAIN (verbose, costs off)
+ UPDATE rem1 set f2 = ''; -- can be pushed down
+ EXPLAIN (verbose, costs off)
+ DELETE FROM rem1; -- can't be pushed down
+ DROP TRIGGER trig_row_after_delete ON rem1;
+
-- ===================================================================
-- test inheritance features
-- ===================================================================
***************
*** 1085,1090 **** fetch from c;
--- 1173,1185 ----
update bar set f2 = null where current of c;
rollback;
+ explain (verbose, costs off)
+ delete from foo where f1 < 5 returning *;
+ delete from foo where f1 < 5 returning *;
+ explain (verbose, costs off)
+ update bar set f2 = f2 + 100 returning *;
+ update bar set f2 = f2 + 100 returning *;
+
drop table foo cascade;
drop table bar cascade;
drop table loct1;
*** a/doc/src/sgml/fdwhandler.sgml
--- b/doc/src/sgml/fdwhandler.sgml
***************
*** 698,703 **** IsForeignRelUpdatable (Relation rel);
--- 698,855 ----
updatability for display in the <literal>information_schema</> views.)
</para>
+ <para>
+ Some inserts, updates, and deletes to foreign tables can be optimized
+ by implementing an alternative set of interfaces. The ordinary
+ interfaces for inserts, updates, and deletes fetch rows from the remote
+ server and then modify those rows one at a time. In some cases, this
+ row-by-row approach is necessary, but it can be inefficient. If it is
+ possible for the foreign server to determine which rows should be
+ modified without actually retrieving them, and if there are no local
+ triggers which would affect the operation, then it is possible to
+ arrange things so that the entire operation is performed on the remote
+ server. The interfaces described below make this possible.
+ </para>
+
+ <para>
+ <programlisting>
+ bool
+ PlanDirectModify (PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+ </programlisting>
+
+ Decide whether it is safe to execute a direct modification
+ on the remote server. If so, return <literal>true</> after performing
+ planning actions needed for that. Otherwise, return <literal>false</>.
+ This optional function is called during query planning.
+ If this function succeeds, <function>BeginDirectModify</>,
+ <function>IterateDirectModify</> and <function>EndDirectModify</> will
+ be called at the execution stage, instead. Otherwise, the table
+ modification will be executed using the table-updating functions
+ described above.
+ The parameters are the same as for <function>PlanForeignModify</>.
+ </para>
+
+ <para>
+ To execute the direct modification on the remote server, this function
+ must rewrite the target subplan with a <structname>ForeignScan</> plan
+ node that executes the direct modification on the remote server. The
+ <structfield>operation</> field of the <structname>ForeignScan</> must
+ be set to the <literal>CmdType</> enumeration appropriately; that is,
+ <literal>CMD_UPDATE</> for <command>UPDATE</>,
+ <literal>CMD_INSERT</> for <command>INSERT</>, and
+ <literal>CMD_DELETE</> for <command>DELETE</>.
+ </para>
+
+ <para>
+ See <xref linkend="fdw-planning"> for additional information.
+ </para>
+
+ <para>
+ If the <function>PlanDirectModify</> pointer is set to
+ <literal>NULL</>, no attempts to execute a direct modification on the
+ remote server are taken.
+ </para>
+
+ <para>
+ <programlisting>
+ void
+ BeginDirectModify (ForeignScanState *node,
+ int eflags);
+ </programlisting>
+
+ Prepare to execute a direct modification on the remote server.
+ This is called during executor startup. It should perform any
+ initialization needed prior to the direct modification (that should be
+ done upon the first call to <function>IterateDirectModify</>).
+ The <structname>ForeignScanState</> node has already been created, but
+ its <structfield>fdw_state</> field is still NULL. Information about
+ the table to modify is accessible through the
+ <structname>ForeignScanState</> node (in particular, from the underlying
+ <structname>ForeignScan</> plan node, which contains any FDW-private
+ information provided by <function>PlanDirectModify</>).
+ <literal>eflags</> contains flag bits describing the executor's
+ operating mode for this plan node.
+ </para>
+
+ <para>
+ Note that when <literal>(eflags & EXEC_FLAG_EXPLAIN_ONLY)</> is
+ true, this function should not perform any externally-visible actions;
+ it should only do the minimum required to make the node state valid
+ for <function>ExplainDirectModify</> and <function>EndDirectModify</>.
+ </para>
+
+ <para>
+ If the <function>BeginDirectModify</> pointer is set to
+ <literal>NULL</>, no attempts to execute a direct modification on the
+ remote server are taken.
+ </para>
+
+ <para>
+ <programlisting>
+ TupleTableSlot *
+ IterateDirectModify (ForeignScanState *node);
+ </programlisting>
+
+ When the <command>INSERT</>, <command>UPDATE</> or <command>DELETE</>
+ query doesn't have a <literal>RETURNING</> clause, just return NULL
+ after a direct modification on the remote server.
+ When the query has the clause, fetch one result containing the data
+ needed for the <literal>RETURNING</> calculation, returning it in a
+ tuple table slot (the node's <structfield>ScanTupleSlot</> should be
+ used for this purpose). The data that was actually inserted, updated
+ or deleted must be stored in the
+ <literal>es_result_relation_info->ri_projectReturning->pi_exprContext->ecxt_scantuple</>
+ of the node's <structname>EState</>.
+ Return NULL if no more rows are available.
+ Note that this is called in a short-lived memory context that will be
+ reset between invocations. Create a memory context in
+ <function>BeginDirectModify</> if you need longer-lived storage, or use
+ the <structfield>es_query_cxt</> of the node's <structname>EState</>.
+ </para>
+
+ <para>
+ The rows returned must match the <structfield>fdw_scan_tlist</> target
+ list if one was supplied, otherwise they must match the row type of the
+ foreign table being updated. If you choose to optimize away fetching
+ columns that are not needed for the <literal>RETURNING</> calculation,
+ you should insert nulls in those column positions, or else generate a
+ <structfield>fdw_scan_tlist</> list with those columns omitted.
+ </para>
+
+ <para>
+ Whether the query has the clause or not, the query's reported row count
+ must be incremented by the FDW itself. When the query doesn't have the
+ clause, the FDW must also increment the row count for the
+ <structname>ForeignScanState</> node in the <command>EXPLAIN ANALYZE</>
+ case.
+ </para>
+
+ <para>
+ If the <function>IterateDirectModify</> pointer is set to
+ <literal>NULL</>, no attempts to execute a direct modification on the
+ remote server are taken.
+ </para>
+
+ <para>
+ <programlisting>
+ void
+ EndDirectModify (ForeignScanState *node);
+ </programlisting>
+
+ Clean up following a direc modification on the remote server. It is
+ normally not important to release palloc'd memory, but for example open
+ files and connections to the remote server should be cleaned up.
+ </para>
+
+ <para>
+ If the <function>EndDirectModify</> pointer is set to
+ <literal>NULL</>, no attempts to execute a direct modification on the
+ remote server are taken.
+ </para>
+
</sect2>
<sect2 id="fdw-callbacks-row-locking">
***************
*** 889,894 **** ExplainForeignModify (ModifyTableState *mtstate,
--- 1041,1069 ----
<command>EXPLAIN</>.
</para>
+ <para>
+ <programlisting>
+ void
+ ExplainDirectModify (ForeignScanState *node,
+ ExplainState *es);
+ </programlisting>
+
+ Print additional <command>EXPLAIN</> output for a direct modification
+ on the remote server.
+ This function can call <function>ExplainPropertyText</> and
+ related functions to add fields to the <command>EXPLAIN</> output.
+ The flag fields in <literal>es</> can be used to determine what to
+ print, and the state of the <structname>ForeignScanState</> node
+ can be inspected to provide run-time statistics in the <command>EXPLAIN
+ ANALYZE</> case.
+ </para>
+
+ <para>
+ If the <function>ExplainDirectModify</> pointer is set to
+ <literal>NULL</>, no additional information is printed during
+ <command>EXPLAIN</>.
+ </para>
+
</sect2>
<sect2 id="fdw-callbacks-analyze">
***************
*** 1194,1200 **** GetForeignServerByName(const char *name, bool missing_ok);
The FDW callback functions <function>GetForeignRelSize</>,
<function>GetForeignPaths</>, <function>GetForeignPlan</>,
<function>PlanForeignModify</>, <function>GetForeignJoinPaths</>,
! and <function>GetForeignUpperPaths</>
must fit into the workings of the <productname>PostgreSQL</> planner.
Here are some notes about what they must do.
</para>
--- 1369,1375 ----
The FDW callback functions <function>GetForeignRelSize</>,
<function>GetForeignPaths</>, <function>GetForeignPlan</>,
<function>PlanForeignModify</>, <function>GetForeignJoinPaths</>,
! <function>GetForeignUpperPaths</>, and <function>PlanDirectModify</>
must fit into the workings of the <productname>PostgreSQL</> planner.
Here are some notes about what they must do.
</para>
***************
*** 1391,1397 **** GetForeignServerByName(const char *name, bool missing_ok);
<para>
When planning an <command>UPDATE</> or <command>DELETE</>,
! <function>PlanForeignModify</> can look up the <structname>RelOptInfo</>
struct for the foreign table and make use of the
<literal>baserel->fdw_private</> data previously created by the
scan-planning functions. However, in <command>INSERT</> the target
--- 1566,1573 ----
<para>
When planning an <command>UPDATE</> or <command>DELETE</>,
! <function>PlanForeignModify</> and <function>PlanDirectModify</>
! can look up the <structname>RelOptInfo</>
struct for the foreign table and make use of the
<literal>baserel->fdw_private</> data previously created by the
scan-planning functions. However, in <command>INSERT</> the target
*** a/doc/src/sgml/postgres-fdw.sgml
--- b/doc/src/sgml/postgres-fdw.sgml
***************
*** 484,489 ****
--- 484,498 ----
extension that's listed in the foreign server's <literal>extensions</>
option. Operators and functions in such clauses must
be <literal>IMMUTABLE</> as well.
+ For an <command>UPDATE</> or <command>DELETE</> query,
+ <filename>postgres_fdw</> attempts to optimize the query execution by
+ sending the whole query to the remote server if there are no query
+ <literal>WHERE</> clauses that cannot be sent to the remote server,
+ no local joins for the query, and no row-level local <literal>BEFORE</> or
+ <literal>AFTER</> triggers on the target table. In <command>UPDATE</>,
+ expressions to assign to target columns must use only built-in data types,
+ <literal>IMMUTABLE</> operators, or <literal>IMMUTABLE</> functions,
+ to reduce the risk of misexecution of the query.
</para>
<para>
*** a/src/backend/commands/explain.c
--- b/src/backend/commands/explain.c
***************
*** 900,906 **** ExplainNode(PlanState *planstate, List *ancestors,
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
! pname = sname = "Foreign Scan";
break;
case T_CustomScan:
sname = "Custom Scan";
--- 900,928 ----
pname = sname = "WorkTable Scan";
break;
case T_ForeignScan:
! sname = "Foreign Scan";
! switch (((ForeignScan *) plan)->operation)
! {
! case CMD_SELECT:
! pname = "Foreign Scan";
! operation = "Select";
! break;
! case CMD_INSERT:
! pname = "Foreign Insert";
! operation = "Insert";
! break;
! case CMD_UPDATE:
! pname = "Foreign Update";
! operation = "Update";
! break;
! case CMD_DELETE:
! pname = "Foreign Delete";
! operation = "Delete";
! break;
! default:
! pname = "???";
! break;
! }
break;
case T_CustomScan:
sname = "Custom Scan";
***************
*** 1648,1653 **** show_plan_tlist(PlanState *planstate, List *ancestors, ExplainState *es)
--- 1670,1688 ----
return;
if (IsA(plan, RecursiveUnion))
return;
+ /*
+ * Likewise for ForeignScan that executes a direct INSERT/UPDATE/DELETE
+ *
+ * Note: the tlist for a ForeignScan that executes a direct INSERT/UPDATE
+ * might contain subplan output expressions that are confusing in this
+ * context. The tlist for a ForeignScan that executes a direct UPDATE/
+ * DELETE always contains "junk" target columns to identify the exact row
+ * to update or delete, which would be confusing in this context. So, we
+ * suppress it in all the cases.
+ */
+ if (IsA(plan, ForeignScan) &&
+ ((ForeignScan *) plan)->operation != CMD_SELECT)
+ return;
/* Set up deparsing context */
context = set_deparse_context_planstate(es->deparse_cxt,
***************
*** 2236,2243 **** show_foreignscan_info(ForeignScanState *fsstate, ExplainState *es)
FdwRoutine *fdwroutine = fsstate->fdwroutine;
/* Let the FDW emit whatever fields it wants */
! if (fdwroutine->ExplainForeignScan != NULL)
! fdwroutine->ExplainForeignScan(fsstate, es);
}
/*
--- 2271,2286 ----
FdwRoutine *fdwroutine = fsstate->fdwroutine;
/* Let the FDW emit whatever fields it wants */
! if (((ForeignScan *) fsstate->ss.ps.plan)->operation != CMD_SELECT)
! {
! if (fdwroutine->ExplainDirectModify != NULL)
! fdwroutine->ExplainDirectModify(fsstate, es);
! }
! else
! {
! if (fdwroutine->ExplainForeignScan != NULL)
! fdwroutine->ExplainForeignScan(fsstate, es);
! }
}
/*
***************
*** 2623,2630 **** show_modifytable_info(ModifyTableState *mtstate, List *ancestors,
}
}
! /* Give FDW a chance */
! if (fdwroutine && fdwroutine->ExplainForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, j);
--- 2666,2675 ----
}
}
! /* Give FDW a chance if needed */
! if (!resultRelInfo->ri_usesFdwDirectModify &&
! fdwroutine != NULL &&
! fdwroutine->ExplainForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, j);
*** a/src/backend/executor/execMain.c
--- b/src/backend/executor/execMain.c
***************
*** 1245,1250 **** InitResultRelInfo(ResultRelInfo *resultRelInfo,
--- 1245,1251 ----
else
resultRelInfo->ri_FdwRoutine = NULL;
resultRelInfo->ri_FdwState = NULL;
+ resultRelInfo->ri_usesFdwDirectModify = false;
resultRelInfo->ri_ConstraintExprs = NULL;
resultRelInfo->ri_junkFilter = NULL;
resultRelInfo->ri_projectReturning = NULL;
*** a/src/backend/executor/nodeForeignscan.c
--- b/src/backend/executor/nodeForeignscan.c
***************
*** 48,54 **** ForeignNext(ForeignScanState *node)
/* Call the Iterate function in short-lived context */
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! slot = node->fdwroutine->IterateForeignScan(node);
MemoryContextSwitchTo(oldcontext);
/*
--- 48,57 ----
/* Call the Iterate function in short-lived context */
oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
! if (plan->operation != CMD_SELECT)
! slot = node->fdwroutine->IterateDirectModify(node);
! else
! slot = node->fdwroutine->IterateForeignScan(node);
MemoryContextSwitchTo(oldcontext);
/*
***************
*** 226,232 **** ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
/*
* Tell the FDW to initialize the scan.
*/
! fdwroutine->BeginForeignScan(scanstate, eflags);
return scanstate;
}
--- 229,238 ----
/*
* Tell the FDW to initialize the scan.
*/
! if (node->operation != CMD_SELECT)
! fdwroutine->BeginDirectModify(scanstate, eflags);
! else
! fdwroutine->BeginForeignScan(scanstate, eflags);
return scanstate;
}
***************
*** 240,247 **** ExecInitForeignScan(ForeignScan *node, EState *estate, int eflags)
void
ExecEndForeignScan(ForeignScanState *node)
{
/* Let the FDW shut down */
! node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
--- 246,258 ----
void
ExecEndForeignScan(ForeignScanState *node)
{
+ ForeignScan *plan = (ForeignScan *) node->ss.ps.plan;
+
/* Let the FDW shut down */
! if (plan->operation != CMD_SELECT)
! node->fdwroutine->EndDirectModify(node);
! else
! node->fdwroutine->EndForeignScan(node);
/* Shut down any outer plan. */
if (outerPlanState(node))
*** a/src/backend/executor/nodeModifyTable.c
--- b/src/backend/executor/nodeModifyTable.c
***************
*** 138,150 **** ExecCheckPlanOutput(Relation resultRel, List *targetList)
* tupleSlot: slot holding tuple actually inserted/updated/deleted
* planSlot: slot holding tuple returned by top subplan node
*
* Returns a slot holding the result tuple
*/
static TupleTableSlot *
! ExecProcessReturning(ProjectionInfo *projectReturning,
TupleTableSlot *tupleSlot,
TupleTableSlot *planSlot)
{
ExprContext *econtext = projectReturning->pi_exprContext;
/*
--- 138,154 ----
* tupleSlot: slot holding tuple actually inserted/updated/deleted
* planSlot: slot holding tuple returned by top subplan node
*
+ * Note: If tupleSlot is NULL, the FDW should have already provided econtext's
+ * scan tuple.
+ *
* Returns a slot holding the result tuple
*/
static TupleTableSlot *
! ExecProcessReturning(ResultRelInfo *resultRelInfo,
TupleTableSlot *tupleSlot,
TupleTableSlot *planSlot)
{
+ ProjectionInfo *projectReturning = resultRelInfo->ri_projectReturning;
ExprContext *econtext = projectReturning->pi_exprContext;
/*
***************
*** 154,160 **** ExecProcessReturning(ProjectionInfo *projectReturning,
ResetExprContext(econtext);
/* Make tuple and any needed join variables available to ExecProject */
! econtext->ecxt_scantuple = tupleSlot;
econtext->ecxt_outertuple = planSlot;
/* Compute the RETURNING expressions */
--- 158,177 ----
ResetExprContext(econtext);
/* Make tuple and any needed join variables available to ExecProject */
! if (tupleSlot)
! econtext->ecxt_scantuple = tupleSlot;
! else
! {
! HeapTuple tuple;
!
! /*
! * RETURNING expressions might reference the tableoid column, so
! * initialize t_tableOid before evaluating them.
! */
! Assert(!TupIsNull(econtext->ecxt_scantuple));
! tuple = ExecMaterializeSlot(econtext->ecxt_scantuple);
! tuple->t_tableOid = RelationGetRelid(resultRelInfo->ri_RelationDesc);
! }
econtext->ecxt_outertuple = planSlot;
/* Compute the RETURNING expressions */
***************
*** 496,503 **** ExecInsert(ModifyTableState *mtstate,
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo->ri_projectReturning,
! slot, planSlot);
return NULL;
}
--- 513,519 ----
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo, slot, planSlot);
return NULL;
}
***************
*** 738,745 **** ldelete:;
ExecStoreTuple(&deltuple, slot, InvalidBuffer, false);
}
! rslot = ExecProcessReturning(resultRelInfo->ri_projectReturning,
! slot, planSlot);
/*
* Before releasing the target tuple again, make sure rslot has a
--- 754,760 ----
ExecStoreTuple(&deltuple, slot, InvalidBuffer, false);
}
! rslot = ExecProcessReturning(resultRelInfo, slot, planSlot);
/*
* Before releasing the target tuple again, make sure rslot has a
***************
*** 1024,1031 **** lreplace:;
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo->ri_projectReturning,
! slot, planSlot);
return NULL;
}
--- 1039,1045 ----
/* Process RETURNING if present */
if (resultRelInfo->ri_projectReturning)
! return ExecProcessReturning(resultRelInfo, slot, planSlot);
return NULL;
}
***************
*** 1380,1385 **** ExecModifyTable(ModifyTableState *node)
--- 1394,1419 ----
break;
}
+ /*
+ * If resultRelInfo->ri_usesFdwDirectModify is true, all we need to do
+ * here is compute the RETURNING expressions.
+ */
+ if (resultRelInfo->ri_usesFdwDirectModify)
+ {
+ Assert(resultRelInfo->ri_projectReturning);
+
+ /*
+ * A scan slot containing the data that was actually inserted,
+ * updated or deleted has already been made available to
+ * ExecProcessReturning by IterateDirectModify, so no need to
+ * provide it here.
+ */
+ slot = ExecProcessReturning(resultRelInfo, NULL, planSlot);
+
+ estate->es_result_relation_info = saved_resultRelInfo;
+ return slot;
+ }
+
EvalPlanQualSetSlot(&node->mt_epqstate, planSlot);
slot = planSlot;
***************
*** 1559,1564 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
--- 1593,1602 ----
{
subplan = (Plan *) lfirst(l);
+ /* Initialize the usesFdwDirectModify flag */
+ resultRelInfo->ri_usesFdwDirectModify = bms_is_member(i,
+ node->fdwDirectModifyPlans);
+
/*
* Verify result relation is a valid target for the current operation
*/
***************
*** 1583,1589 **** ExecInitModifyTable(ModifyTable *node, EState *estate, int eflags)
mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
/* Also let FDWs init themselves for foreign-table result rels */
! if (resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, i);
--- 1621,1628 ----
mtstate->mt_plans[i] = ExecInitNode(subplan, estate, eflags);
/* Also let FDWs init themselves for foreign-table result rels */
! if (!resultRelInfo->ri_usesFdwDirectModify &&
! resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->BeginForeignModify != NULL)
{
List *fdw_private = (List *) list_nth(node->fdwPrivLists, i);
***************
*** 1910,1916 **** ExecEndModifyTable(ModifyTableState *node)
{
ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
! if (resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->EndForeignModify != NULL)
resultRelInfo->ri_FdwRoutine->EndForeignModify(node->ps.state,
resultRelInfo);
--- 1949,1956 ----
{
ResultRelInfo *resultRelInfo = node->resultRelInfo + i;
! if (!resultRelInfo->ri_usesFdwDirectModify &&
! resultRelInfo->ri_FdwRoutine != NULL &&
resultRelInfo->ri_FdwRoutine->EndForeignModify != NULL)
resultRelInfo->ri_FdwRoutine->EndForeignModify(node->ps.state,
resultRelInfo);
*** a/src/backend/nodes/copyfuncs.c
--- b/src/backend/nodes/copyfuncs.c
***************
*** 188,193 **** _copyModifyTable(const ModifyTable *from)
--- 188,194 ----
COPY_NODE_FIELD(withCheckOptionLists);
COPY_NODE_FIELD(returningLists);
COPY_NODE_FIELD(fdwPrivLists);
+ COPY_BITMAPSET_FIELD(fdwDirectModifyPlans);
COPY_NODE_FIELD(rowMarks);
COPY_SCALAR_FIELD(epqParam);
COPY_SCALAR_FIELD(onConflictAction);
***************
*** 648,653 **** _copyForeignScan(const ForeignScan *from)
--- 649,655 ----
/*
* copy remainder of node
*/
+ COPY_SCALAR_FIELD(operation);
COPY_SCALAR_FIELD(fs_server);
COPY_NODE_FIELD(fdw_exprs);
COPY_NODE_FIELD(fdw_private);
*** a/src/backend/nodes/outfuncs.c
--- b/src/backend/nodes/outfuncs.c
***************
*** 356,361 **** _outModifyTable(StringInfo str, const ModifyTable *node)
--- 356,362 ----
WRITE_NODE_FIELD(withCheckOptionLists);
WRITE_NODE_FIELD(returningLists);
WRITE_NODE_FIELD(fdwPrivLists);
+ WRITE_BITMAPSET_FIELD(fdwDirectModifyPlans);
WRITE_NODE_FIELD(rowMarks);
WRITE_INT_FIELD(epqParam);
WRITE_ENUM_FIELD(onConflictAction, OnConflictAction);
***************
*** 608,613 **** _outForeignScan(StringInfo str, const ForeignScan *node)
--- 609,615 ----
_outScanInfo(str, (const Scan *) node);
+ WRITE_ENUM_FIELD(operation, CmdType);
WRITE_OID_FIELD(fs_server);
WRITE_NODE_FIELD(fdw_exprs);
WRITE_NODE_FIELD(fdw_private);
*** a/src/backend/nodes/readfuncs.c
--- b/src/backend/nodes/readfuncs.c
***************
*** 1481,1486 **** _readModifyTable(void)
--- 1481,1487 ----
READ_NODE_FIELD(withCheckOptionLists);
READ_NODE_FIELD(returningLists);
READ_NODE_FIELD(fdwPrivLists);
+ READ_BITMAPSET_FIELD(fdwDirectModifyPlans);
READ_NODE_FIELD(rowMarks);
READ_INT_FIELD(epqParam);
READ_ENUM_FIELD(onConflictAction, OnConflictAction);
*** a/src/backend/optimizer/plan/createplan.c
--- b/src/backend/optimizer/plan/createplan.c
***************
*** 4903,4908 **** make_foreignscan(List *qptlist,
--- 4903,4909 ----
plan->lefttree = outer_plan;
plan->righttree = NULL;
node->scan.scanrelid = scanrelid;
+ node->operation = CMD_SELECT;
/* fs_server will be filled in by create_foreignscan_plan */
node->fs_server = InvalidOid;
node->fdw_exprs = fdw_exprs;
***************
*** 6018,6023 **** make_modifytable(PlannerInfo *root,
--- 6019,6025 ----
{
ModifyTable *node = makeNode(ModifyTable);
List *fdw_private_list;
+ Bitmapset *direct_modify_plans;
ListCell *lc;
int i;
***************
*** 6075,6086 **** make_modifytable(PlannerInfo *root,
--- 6077,6090 ----
* construct private plan data, and accumulate it all into a list.
*/
fdw_private_list = NIL;
+ direct_modify_plans = NULL;
i = 0;
foreach(lc, resultRelations)
{
Index rti = lfirst_int(lc);
FdwRoutine *fdwroutine;
List *fdw_private;
+ bool direct_modify;
/*
* If possible, we want to get the FdwRoutine from our RelOptInfo for
***************
*** 6107,6113 **** make_modifytable(PlannerInfo *root,
--- 6111,6133 ----
fdwroutine = NULL;
}
+ /*
+ * If the target foreign table has any row-level triggers, we can't
+ * modify the foreign table directly.
+ */
+ direct_modify = false;
if (fdwroutine != NULL &&
+ fdwroutine->PlanDirectModify != NULL &&
+ fdwroutine->BeginDirectModify != NULL &&
+ fdwroutine->IterateDirectModify != NULL &&
+ fdwroutine->EndDirectModify != NULL &&
+ !has_row_triggers(root, rti, operation))
+ direct_modify = fdwroutine->PlanDirectModify(root, node, rti, i);
+ if (direct_modify)
+ direct_modify_plans = bms_add_member(direct_modify_plans, i);
+
+ if (!direct_modify &&
+ fdwroutine != NULL &&
fdwroutine->PlanForeignModify != NULL)
fdw_private = fdwroutine->PlanForeignModify(root, node, rti, i);
else
***************
*** 6116,6121 **** make_modifytable(PlannerInfo *root,
--- 6136,6142 ----
i++;
}
node->fdwPrivLists = fdw_private_list;
+ node->fdwDirectModifyPlans = direct_modify_plans;
return node;
}
*** a/src/backend/optimizer/util/plancat.c
--- b/src/backend/optimizer/util/plancat.c
***************
*** 1540,1542 **** has_unique_index(RelOptInfo *rel, AttrNumber attno)
--- 1540,1589 ----
}
return false;
}
+
+
+ /*
+ * has_row_triggers
+ *
+ * Detect whether the specified relation has any row-level triggers for event.
+ */
+ bool
+ has_row_triggers(PlannerInfo *root, Index rti, CmdType event)
+ {
+ RangeTblEntry *rte = planner_rt_fetch(rti, root);
+ Relation relation;
+ TriggerDesc *trigDesc;
+ bool result = false;
+
+ /* Assume we already have adequate lock */
+ relation = heap_open(rte->relid, NoLock);
+
+ trigDesc = relation->trigdesc;
+ switch (event)
+ {
+ case CMD_INSERT:
+ if (trigDesc &&
+ (trigDesc->trig_insert_after_row ||
+ trigDesc->trig_insert_before_row))
+ result = true;
+ break;
+ case CMD_UPDATE:
+ if (trigDesc &&
+ (trigDesc->trig_update_after_row ||
+ trigDesc->trig_update_before_row))
+ result = true;
+ break;
+ case CMD_DELETE:
+ if (trigDesc &&
+ (trigDesc->trig_delete_after_row ||
+ trigDesc->trig_delete_before_row))
+ result = true;
+ break;
+ default:
+ elog(ERROR, "unrecognized CmdType: %d", (int) event);
+ break;
+ }
+
+ heap_close(relation, NoLock);
+ return result;
+ }
*** a/src/include/foreign/fdwapi.h
--- b/src/include/foreign/fdwapi.h
***************
*** 97,102 **** typedef void (*EndForeignModify_function) (EState *estate,
--- 97,114 ----
typedef int (*IsForeignRelUpdatable_function) (Relation rel);
+ typedef bool (*PlanDirectModify_function) (PlannerInfo *root,
+ ModifyTable *plan,
+ Index resultRelation,
+ int subplan_index);
+
+ typedef void (*BeginDirectModify_function) (ForeignScanState *node,
+ int eflags);
+
+ typedef TupleTableSlot *(*IterateDirectModify_function) (ForeignScanState *node);
+
+ typedef void (*EndDirectModify_function) (ForeignScanState *node);
+
typedef RowMarkType (*GetForeignRowMarkType_function) (RangeTblEntry *rte,
LockClauseStrength strength);
***************
*** 114,119 **** typedef void (*ExplainForeignModify_function) (ModifyTableState *mtstate,
--- 126,134 ----
int subplan_index,
struct ExplainState *es);
+ typedef void (*ExplainDirectModify_function) (ForeignScanState *node,
+ struct ExplainState *es);
+
typedef int (*AcquireSampleRowsFunc) (Relation relation, int elevel,
HeapTuple *rows, int targrows,
double *totalrows,
***************
*** 181,186 **** typedef struct FdwRoutine
--- 196,205 ----
ExecForeignDelete_function ExecForeignDelete;
EndForeignModify_function EndForeignModify;
IsForeignRelUpdatable_function IsForeignRelUpdatable;
+ PlanDirectModify_function PlanDirectModify;
+ BeginDirectModify_function BeginDirectModify;
+ IterateDirectModify_function IterateDirectModify;
+ EndDirectModify_function EndDirectModify;
/* Functions for SELECT FOR UPDATE/SHARE row locking */
GetForeignRowMarkType_function GetForeignRowMarkType;
***************
*** 190,195 **** typedef struct FdwRoutine
--- 209,215 ----
/* Support functions for EXPLAIN */
ExplainForeignScan_function ExplainForeignScan;
ExplainForeignModify_function ExplainForeignModify;
+ ExplainDirectModify_function ExplainDirectModify;
/* Support functions for ANALYZE */
AnalyzeForeignTable_function AnalyzeForeignTable;
*** a/src/include/nodes/execnodes.h
--- b/src/include/nodes/execnodes.h
***************
*** 311,316 **** typedef struct JunkFilter
--- 311,317 ----
* TrigInstrument optional runtime measurements for triggers
* FdwRoutine FDW callback functions, if foreign table
* FdwState available to save private state of FDW
+ * usesFdwDirectModify true when modifying foreign table directly
* WithCheckOptions list of WithCheckOption's to be checked
* WithCheckOptionExprs list of WithCheckOption expr states
* ConstraintExprs array of constraint-checking expr states
***************
*** 334,339 **** typedef struct ResultRelInfo
--- 335,341 ----
Instrumentation *ri_TrigInstrument;
struct FdwRoutine *ri_FdwRoutine;
void *ri_FdwState;
+ bool ri_usesFdwDirectModify;
List *ri_WithCheckOptions;
List *ri_WithCheckOptionExprs;
List **ri_ConstraintExprs;
*** a/src/include/nodes/pg_list.h
--- b/src/include/nodes/pg_list.h
***************
*** 134,149 **** list_length(const List *l)
--- 134,152 ----
#define list_make2(x1,x2) lcons(x1, list_make1(x2))
#define list_make3(x1,x2,x3) lcons(x1, list_make2(x2, x3))
#define list_make4(x1,x2,x3,x4) lcons(x1, list_make3(x2, x3, x4))
+ #define list_make5(x1,x2,x3,x4,x5) lcons(x1, list_make4(x2, x3, x4, x5))
#define list_make1_int(x1) lcons_int(x1, NIL)
#define list_make2_int(x1,x2) lcons_int(x1, list_make1_int(x2))
#define list_make3_int(x1,x2,x3) lcons_int(x1, list_make2_int(x2, x3))
#define list_make4_int(x1,x2,x3,x4) lcons_int(x1, list_make3_int(x2, x3, x4))
+ #define list_make5_int(x1,x2,x3,x4,x5) lcons_int(x1, list_make4_int(x2, x3, x4, x5))
#define list_make1_oid(x1) lcons_oid(x1, NIL)
#define list_make2_oid(x1,x2) lcons_oid(x1, list_make1_oid(x2))
#define list_make3_oid(x1,x2,x3) lcons_oid(x1, list_make2_oid(x2, x3))
#define list_make4_oid(x1,x2,x3,x4) lcons_oid(x1, list_make3_oid(x2, x3, x4))
+ #define list_make5_oid(x1,x2,x3,x4,x5) lcons_oid(x1, list_make4_oid(x2, x3, x4, x5))
/*
* foreach -
*** a/src/include/nodes/plannodes.h
--- b/src/include/nodes/plannodes.h
***************
*** 189,194 **** typedef struct ModifyTable
--- 189,195 ----
List *withCheckOptionLists; /* per-target-table WCO lists */
List *returningLists; /* per-target-table RETURNING tlists */
List *fdwPrivLists; /* per-target-table FDW private data lists */
+ Bitmapset *fdwDirectModifyPlans; /* indices of FDW DM plans */
List *rowMarks; /* PlanRowMarks (non-locking only) */
int epqParam; /* ID of Param for EvalPlanQual re-eval */
OnConflictAction onConflictAction; /* ON CONFLICT action */
***************
*** 531,536 **** typedef struct WorkTableScan
--- 532,538 ----
typedef struct ForeignScan
{
Scan scan;
+ CmdType operation; /* SELECT/INSERT/UPDATE/DELETE */
Oid fs_server; /* OID of foreign server */
List *fdw_exprs; /* expressions that FDW may evaluate */
List *fdw_private; /* private data for FDW */
*** a/src/include/optimizer/plancat.h
--- b/src/include/optimizer/plancat.h
***************
*** 55,58 **** extern Selectivity join_selectivity(PlannerInfo *root,
--- 55,60 ----
JoinType jointype,
SpecialJoinInfo *sjinfo);
+ extern bool has_row_triggers(PlannerInfo *root, Index rti, CmdType event);
+
#endif /* PLANCAT_H */
On Fri, Mar 18, 2016 at 5:15 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
On 2016/03/10 2:56, Robert Haas wrote:
I see that you went and changed all of the places that tested for !=
CMD_SELECT and made them test for == CMD_INSERT || == CMD_UPDATE || ==
CMD_DELETE instead. I think that's the wrong direction. I think that
we should use the != CMD_SELECT version of the test everywhere.
That's a single test instead of three, so marginally faster, and maybe
marginally more future-proof.I think deparsePushedDownUpdateSql should be renamed to use the new
"direct modify" naming, like deparseDirectUpdateSql, maybe.I would suggest not numbering the tests in postgresPlanDirectModify.
That just becomes a nuisance to keep up to date as things change.Agreed. I updated the patch to address these comments. Attached is the
updated version of the patch.
Committed.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/03/19 3:30, Robert Haas wrote:
On Fri, Mar 18, 2016 at 5:15 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:Attached is the updated version of the patch.
Committed.
Thank you.
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 22 March 2016 at 02:30, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp> wrote:
On 2016/03/19 3:30, Robert Haas wrote:
On Fri, Mar 18, 2016 at 5:15 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:Attached is the updated version of the patch.
I've noticed that you now can't cancel a query if there's DML pushdown
to a foreign server. This previously worked while it was sending
individual statements as it interrupted and rolled it back.
Here's what the local server sees when trying to cancel:
# DELETE FROM remote.contacts;
^CCancel request sent
DELETE 5000000
This should probably be fixed.
Thom
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Mar 23, 2016 at 10:05 PM, Thom Brown <thom@linux.com> wrote:
I've noticed that you now can't cancel a query if there's DML pushdown
to a foreign server. This previously worked while it was sending
individual statements as it interrupted and rolled it back.Here's what the local server sees when trying to cancel:
# DELETE FROM remote.contacts;
^CCancel request sent
DELETE 5000000This should probably be fixed.
Looking at what has been committed, execute_dml_stmt is using
PQexecParams, so we'd want to use an asynchronous call and loop on
PQgetResult with CHECK_FOR_INTERRUPTS() in it.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/03/24 11:14, Michael Paquier wrote:
On Wed, Mar 23, 2016 at 10:05 PM, Thom Brown <thom@linux.com> wrote:
I've noticed that you now can't cancel a query if there's DML pushdown
to a foreign server. This previously worked while it was sending
individual statements as it interrupted and rolled it back.Here's what the local server sees when trying to cancel:
# DELETE FROM remote.contacts;
^CCancel request sent
DELETE 5000000This should probably be fixed.
Looking at what has been committed, execute_dml_stmt is using
PQexecParams, so we'd want to use an asynchronous call and loop on
PQgetResult with CHECK_FOR_INTERRUPTS() in it.
Will fix.
Thanks for the report, Thom! Thanks for the advice, Michael!
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Mar 24, 2016 at 1:02 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
On 2016/03/24 11:14, Michael Paquier wrote:
On Wed, Mar 23, 2016 at 10:05 PM, Thom Brown <thom@linux.com> wrote:
I've noticed that you now can't cancel a query if there's DML pushdown
to a foreign server. This previously worked while it was sending
individual statements as it interrupted and rolled it back.Here's what the local server sees when trying to cancel:
# DELETE FROM remote.contacts;
^CCancel request sent
DELETE 5000000This should probably be fixed.
Looking at what has been committed, execute_dml_stmt is using
PQexecParams, so we'd want to use an asynchronous call and loop on
PQgetResult with CHECK_FOR_INTERRUPTS() in it.Will fix.
Thanks for the report, Thom! Thanks for the advice, Michael!
I am adding that to the list of open items of 9.6 to not forget about it.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Mar 25, 2016 at 3:10 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:
On Thu, Mar 24, 2016 at 1:02 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:Thanks for the report, Thom! Thanks for the advice, Michael!
I am adding that to the list of open items of 9.6 to not forget about it.
My bad. Thom did it already :p
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Mar 24, 2016 at 01:02:57PM +0900, Etsuro Fujita wrote:
On 2016/03/24 11:14, Michael Paquier wrote:
On Wed, Mar 23, 2016 at 10:05 PM, Thom Brown <thom@linux.com> wrote:
I've noticed that you now can't cancel a query if there's DML pushdown
to a foreign server. This previously worked while it was sending
individual statements as it interrupted and rolled it back.Here's what the local server sees when trying to cancel:
# DELETE FROM remote.contacts;
^CCancel request sent
DELETE 5000000This should probably be fixed.
Looking at what has been committed, execute_dml_stmt is using
PQexecParams, so we'd want to use an asynchronous call and loop on
PQgetResult with CHECK_FOR_INTERRUPTS() in it.Will fix.
Thanks for the report, Thom! Thanks for the advice, Michael!
[This is a generic notification.]
The above-described topic is currently a PostgreSQL 9.6 open item. Robert,
since you committed the patch believed to have created it, you own this open
item. If that responsibility lies elsewhere, please let us know whose
responsibility it is to fix this. Since new open items may be discovered at
any time and I want to plan to have them all fixed well in advance of the ship
date, I will appreciate your efforts toward speedy resolution. Please
present, within 72 hours, a plan to fix the defect within seven days of this
message. Thanks.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/03/31 14:07, Noah Misch wrote:
On Thu, Mar 24, 2016 at 01:02:57PM +0900, Etsuro Fujita wrote:
On 2016/03/24 11:14, Michael Paquier wrote:
On Wed, Mar 23, 2016 at 10:05 PM, Thom Brown <thom@linux.com> wrote:
I've noticed that you now can't cancel a query if there's DML pushdown
to a foreign server. This previously worked while it was sending
individual statements as it interrupted and rolled it back.Here's what the local server sees when trying to cancel:
# DELETE FROM remote.contacts;
^CCancel request sent
DELETE 5000000This should probably be fixed.
Looking at what has been committed, execute_dml_stmt is using
PQexecParams, so we'd want to use an asynchronous call and loop on
PQgetResult with CHECK_FOR_INTERRUPTS() in it.
Will fix.
[This is a generic notification.]
The above-described topic is currently a PostgreSQL 9.6 open item. Robert,
since you committed the patch believed to have created it, you own this open
item. If that responsibility lies elsewhere, please let us know whose
responsibility it is to fix this. Since new open items may be discovered at
any time and I want to plan to have them all fixed well in advance of the ship
date, I will appreciate your efforts toward speedy resolution. Please
present, within 72 hours, a plan to fix the defect within seven days of this
message. Thanks.
Sorry for not having taken any action. I've been busy with another task
lately, but I started working on this. I plan to post a patch early
next week.
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/03/31 16:38, Etsuro Fujita wrote:
On 2016/03/31 14:07, Noah Misch wrote:
On Thu, Mar 24, 2016 at 01:02:57PM +0900, Etsuro Fujita wrote:
On 2016/03/24 11:14, Michael Paquier wrote:
On Wed, Mar 23, 2016 at 10:05 PM, Thom Brown <thom@linux.com> wrote:
I've noticed that you now can't cancel a query if there's DML pushdown
to a foreign server. This previously worked while it was sending
individual statements as it interrupted and rolled it back.Here's what the local server sees when trying to cancel:
# DELETE FROM remote.contacts;
^CCancel request sent
DELETE 5000000This should probably be fixed.
Looking at what has been committed, execute_dml_stmt is using
PQexecParams, so we'd want to use an asynchronous call and loop on
PQgetResult with CHECK_FOR_INTERRUPTS() in it.
Will fix.
[This is a generic notification.]
Sorry for not having taken any action. I've been busy with another task
lately, but I started working on this. I plan to post a patch early
next week.
Here is a patch to fix this issue. As proposed by Michael, I modified
execute_dml_stmt so that it uses PQsendQueryParams, not PQexecParams.
Any comments are welcome.
Best regards,
Etsuro Fujita
Attachments:
pgfdw-direct-modify.patchtext/x-patch; name=pgfdw-direct-modify.patchDownload
*** a/contrib/postgres_fdw/connection.c
--- b/contrib/postgres_fdw/connection.c
***************
*** 12,17 ****
--- 12,20 ----
*/
#include "postgres.h"
+ #include <unistd.h>
+ #include <sys/time.h>
+
#include "postgres_fdw.h"
#include "access/xact.h"
***************
*** 20,25 ****
--- 23,38 ----
#include "utils/hsearch.h"
#include "utils/memutils.h"
+ #ifdef HAVE_POLL_H
+ #include <poll.h>
+ #endif
+ #ifdef HAVE_SYS_POLL_H
+ #include <sys/poll.h>
+ #endif
+ #ifdef HAVE_SYS_SELECT_H
+ #include <sys/select.h>
+ #endif
+
/*
* Connection cache hash table entry
***************
*** 417,422 **** ReleaseConnection(PGconn *conn)
--- 430,484 ----
}
/*
+ * Wait until we can read data.
+ *
+ * Returns true if data has become available for reading, false if interrupted
+ * by signal.
+ *
+ * This is based on pqSocketCheck.
+ */
+ bool
+ CheckSocket(PGconn *conn)
+ {
+ int ret;
+
+ Assert(conn != NULL);
+ if (PQsocket(conn) < 0)
+ ereport(ERROR,
+ (errcode_for_socket_access(),
+ errmsg("invalid socket: %s", PQerrorMessage(conn))));
+
+ /* We use poll(2) if available, otherwise select(2) */
+ {
+ #ifdef HAVE_POLL
+ struct pollfd input_fd;
+
+ input_fd.fd = PQsocket(conn);
+ input_fd.events = POLLIN | POLLERR;
+ input_fd.revents = 0;
+
+ ret = poll(&input_fd, 1, -1);
+ #else /* !HAVE_POLL */
+ fd_set input_mask;
+
+ FD_ZERO(&input_mask);
+ FD_SET(PQsocket(conn), &input_mask);
+
+ ret = select(PQsocket(conn) + 1, &input_mask, NULL, NULL, NULL);
+ #endif /* HAVE_POLL */
+ }
+
+ Assert(ret != 0);
+ if (ret < 0 && errno == EINTR)
+ return false;
+ if (ret < 0)
+ ereport(ERROR,
+ (errcode_for_socket_access(),
+ errmsg("select() failed: %s", strerror(errno))));
+ return true;
+ }
+
+ /*
* Assign a "unique" number for a cursor.
*
* These really only need to be unique per connection within a transaction.
***************
*** 598,603 **** pgfdw_xact_callback(XactEvent event, void *arg)
--- 660,684 ----
case XACT_EVENT_ABORT:
/* Assume we might have lost track of prepared statements */
entry->have_error = true;
+ /*
+ * If we executed a query asynchronously, the query might
+ * have not yet completed. Check to see if a command is
+ * still being processed by the remote server, and if so,
+ * request cancellation of the command; if not, abort
+ * gracefully.
+ */
+ if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
+ {
+ PGcancel *cancel;
+ char errbuf[256];
+
+ if ((cancel = PQgetCancel(entry->conn)))
+ {
+ PQcancel(cancel, errbuf, sizeof(errbuf));
+ PQfreeCancel(cancel);
+ }
+ break;
+ }
/* If we're aborting, abort all remote transactions too */
res = PQexec(entry->conn, "ABORT TRANSACTION");
/* Note: can't throw ERROR, it would be infinite loop */
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
***************
*** 3151,3158 **** execute_dml_stmt(ForeignScanState *node)
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
! dmstate->result = PQexecParams(dmstate->conn, dmstate->query,
! numParams, NULL, values, NULL, NULL, 0);
if (PQresultStatus(dmstate->result) !=
(dmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, dmstate->result, dmstate->conn, true,
--- 3151,3181 ----
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
! if (!PQsendQueryParams(dmstate->conn, dmstate->query, numParams,
! NULL, values, NULL, NULL, 0))
! pgfdw_report_error(ERROR, NULL, dmstate->conn, true, dmstate->query);
!
! /*
! * Receive data until PQgetResult is ready to get the result without
! * blocking.
! */
! while (PQisBusy(dmstate->conn))
! {
! /* Wait until data is available. */
! if (!CheckSocket(dmstate->conn))
! {
! /* We are interrupted by signal. */
! CHECK_FOR_INTERRUPTS();
! continue;
! }
! /* Data is available. */
! if (!PQconsumeInput(dmstate->conn))
! pgfdw_report_error(ERROR, NULL, dmstate->conn, true,
! dmstate->query);
! }
!
! /* Get the result */
! dmstate->result = PQgetResult(dmstate->conn);
if (PQresultStatus(dmstate->result) !=
(dmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, dmstate->result, dmstate->conn, true,
*** a/contrib/postgres_fdw/postgres_fdw.h
--- b/contrib/postgres_fdw/postgres_fdw.h
***************
*** 101,106 **** extern void reset_transmission_modes(int nestlevel);
--- 101,107 ----
/* in connection.c */
extern PGconn *GetConnection(UserMapping *user, bool will_prep_stmt);
extern void ReleaseConnection(PGconn *conn);
+ extern bool CheckSocket(PGconn *conn);
extern unsigned int GetCursorNumber(PGconn *conn);
extern unsigned int GetPrepStmtNumber(PGconn *conn);
extern void pgfdw_report_error(int elevel, PGresult *res, PGconn *conn,
On Mon, Apr 4, 2016 at 7:49 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
On 2016/03/31 16:38, Etsuro Fujita wrote:
On 2016/03/31 14:07, Noah Misch wrote:
On Thu, Mar 24, 2016 at 01:02:57PM +0900, Etsuro Fujita wrote:
On 2016/03/24 11:14, Michael Paquier wrote:
On Wed, Mar 23, 2016 at 10:05 PM, Thom Brown <thom@linux.com> wrote:
I've noticed that you now can't cancel a query if there's DML pushdown
to a foreign server. This previously worked while it was sending
individual statements as it interrupted and rolled it back.Here's what the local server sees when trying to cancel:
# DELETE FROM remote.contacts;
^CCancel request sent
DELETE 5000000This should probably be fixed.
Looking at what has been committed, execute_dml_stmt is using
PQexecParams, so we'd want to use an asynchronous call and loop on
PQgetResult with CHECK_FOR_INTERRUPTS() in it.Will fix.
[This is a generic notification.]
Sorry for not having taken any action. I've been busy with another task
lately, but I started working on this. I plan to post a patch early
next week.Here is a patch to fix this issue. As proposed by Michael, I modified
execute_dml_stmt so that it uses PQsendQueryParams, not PQexecParams. Any
comments are welcome.
+ * This is based on pqSocketCheck.
+ */
+ bool
+ CheckSocket(PGconn *conn)
+ {
+ int ret;
+
+ Assert(conn != NULL);
Instead of copying again pqSocketQuery, which is as well copied in
libpqwalreceiver.c, wouldn't it be better to use WaitLatchOrSocket
with the socket returned by PQsocket?
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/04/04 20:35, Michael Paquier wrote:
On Mon, Apr 4, 2016 at 7:49 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:Here is a patch to fix this issue. As proposed by Michael, I modified
execute_dml_stmt so that it uses PQsendQueryParams, not PQexecParams. Any
comments are welcome.
+ * This is based on pqSocketCheck. + */ + bool + CheckSocket(PGconn *conn) + { + int ret; + + Assert(conn != NULL); Instead of copying again pqSocketQuery, which is as well copied in libpqwalreceiver.c, wouldn't it be better to use WaitLatchOrSocket with the socket returned by PQsocket?
Will check. Thanks for the comment!
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Apr 05, 2016 at 03:22:03PM +0900, Etsuro Fujita wrote:
On 2016/04/04 20:35, Michael Paquier wrote:
On Mon, Apr 4, 2016 at 7:49 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:Here is a patch to fix this issue. As proposed by Michael, I modified
execute_dml_stmt so that it uses PQsendQueryParams, not PQexecParams. Any
comments are welcome.+ * This is based on pqSocketCheck. + */ + bool + CheckSocket(PGconn *conn) + { + int ret; + + Assert(conn != NULL); Instead of copying again pqSocketQuery, which is as well copied in libpqwalreceiver.c, wouldn't it be better to use WaitLatchOrSocket with the socket returned by PQsocket?Will check. Thanks for the comment!
What do you think? This open item's seven-day deadline has passed. It would
help keep things moving to know whether you consider your latest patch optimal
or whether you wish to change it the way Michael described.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/04/08 13:42, Noah Misch wrote:
On Tue, Apr 05, 2016 at 03:22:03PM +0900, Etsuro Fujita wrote:
On 2016/04/04 20:35, Michael Paquier wrote:
On Mon, Apr 4, 2016 at 7:49 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:Here is a patch to fix this issue. As proposed by Michael, I modified
execute_dml_stmt so that it uses PQsendQueryParams, not PQexecParams. Any
comments are welcome.
+ * This is based on pqSocketCheck. + */ + bool + CheckSocket(PGconn *conn) + { + int ret; + + Assert(conn != NULL); Instead of copying again pqSocketQuery, which is as well copied in libpqwalreceiver.c, wouldn't it be better to use WaitLatchOrSocket with the socket returned by PQsocket?
Will check. Thanks for the comment!
What do you think? This open item's seven-day deadline has passed. It would
help keep things moving to know whether you consider your latest patch optimal
or whether you wish to change it the way Michael described.
I wish to change it that way because it not only avoids the duplicate
but fixes a bug in the previous patch that I overlooked that there is a
race condition if a signal arrives just before entering the CheckSocket.
Attached is an updated version of the patch.
Sorry for the delay.
Best regards,
Etsuro Fujita
Attachments:
pgfdw-direct-modify-v2.patchtext/x-patch; name=pgfdw-direct-modify-v2.patchDownload
*** a/contrib/postgres_fdw/connection.c
--- b/contrib/postgres_fdw/connection.c
***************
*** 598,603 **** pgfdw_xact_callback(XactEvent event, void *arg)
--- 598,623 ----
case XACT_EVENT_ABORT:
/* Assume we might have lost track of prepared statements */
entry->have_error = true;
+ /*
+ * If we had submitted a command to the remote server using
+ * an asynchronous execution function, the command might
+ * have not yet completed. Check to see if a command is
+ * still being processed by the remote server, and if so,
+ * request cancellation of the command; if not, abort
+ * gracefully.
+ */
+ if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
+ {
+ PGcancel *cancel;
+ char errbuf[256];
+
+ if ((cancel = PQgetCancel(entry->conn)))
+ {
+ PQcancel(cancel, errbuf, sizeof(errbuf));
+ PQfreeCancel(cancel);
+ }
+ break;
+ }
/* If we're aborting, abort all remote transactions too */
res = PQexec(entry->conn, "ABORT TRANSACTION");
/* Note: can't throw ERROR, it would be infinite loop */
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
***************
*** 32,37 ****
--- 32,38 ----
#include "optimizer/var.h"
#include "optimizer/tlist.h"
#include "parser/parsetree.h"
+ #include "storage/latch.h"
#include "utils/builtins.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
***************
*** 3131,3136 **** execute_dml_stmt(ForeignScanState *node)
--- 3132,3138 ----
ExprContext *econtext = node->ss.ps.ps_ExprContext;
int numParams = dmstate->numParams;
const char **values = dmstate->param_values;
+ int wc;
/*
* Construct array of query parameter values in text format.
***************
*** 3151,3158 **** execute_dml_stmt(ForeignScanState *node)
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
! dmstate->result = PQexecParams(dmstate->conn, dmstate->query,
! numParams, NULL, values, NULL, NULL, 0);
if (PQresultStatus(dmstate->result) !=
(dmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, dmstate->result, dmstate->conn, true,
--- 3153,3188 ----
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
! if (!PQsendQueryParams(dmstate->conn, dmstate->query, numParams,
! NULL, values, NULL, NULL, 0))
! pgfdw_report_error(ERROR, NULL, dmstate->conn, true, dmstate->query);
!
! /*
! * Receive data until PQgetResult is ready to get the result without
! * blocking.
! */
! while (PQisBusy(dmstate->conn))
! {
! /* Sleep until there's something to do */
! wc = WaitLatchOrSocket(MyLatch,
! WL_LATCH_SET | WL_SOCKET_READABLE,
! PQsocket(dmstate->conn),
! -1L);
! ResetLatch(MyLatch);
!
! CHECK_FOR_INTERRUPTS();
!
! /* Data available in socket */
! if (wc & WL_SOCKET_READABLE)
! {
! if (!PQconsumeInput(dmstate->conn))
! pgfdw_report_error(ERROR, NULL, dmstate->conn, true,
! dmstate->query);
! }
! }
!
! /* Get the result */
! dmstate->result = PQgetResult(dmstate->conn);
if (PQresultStatus(dmstate->result) !=
(dmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, dmstate->result, dmstate->conn, true,
On Fri, Apr 8, 2016 at 3:05 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
What do you think? This open item's seven-day deadline has passed. It
would
help keep things moving to know whether you consider your latest patch
optimal
or whether you wish to change it the way Michael described.I wish to change it that way because it not only avoids the duplicate but
fixes a bug in the previous patch that I overlooked that there is a race
condition if a signal arrives just before entering the CheckSocket.Attached is an updated version of the patch.
The comment just before the second hunk in the patch says:
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
But the patch adds a whole bunch of new things there that seem like
they can error out, like CHECK_FOR_INTERRUPTS(), for example. Isn't
that a problem?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Apr 8, 2016 at 6:28 PM, Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Apr 8, 2016 at 3:05 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:What do you think? This open item's seven-day deadline has passed. It
would
help keep things moving to know whether you consider your latest patch
optimal
or whether you wish to change it the way Michael described.I wish to change it that way because it not only avoids the duplicate but
fixes a bug in the previous patch that I overlooked that there is a race
condition if a signal arrives just before entering the CheckSocket.Attached is an updated version of the patch.
The comment just before the second hunk in the patch says:
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.But the patch adds a whole bunch of new things there that seem like
they can error out, like CHECK_FOR_INTERRUPTS(), for example. Isn't
that a problem?
Basically we fetching the PGresult, after the newly added hunk, so there
should not be any problem.
But yes comment is definitely at wrong place.
PFA patch with correction.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Rushabh Lathia
Attachments:
pgfdw-direct-modify-v3.patchapplication/x-patch; name=pgfdw-direct-modify-v3.patchDownload
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 189f290..8820597 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -598,6 +598,26 @@ pgfdw_xact_callback(XactEvent event, void *arg)
case XACT_EVENT_ABORT:
/* Assume we might have lost track of prepared statements */
entry->have_error = true;
+ /*
+ * If we had submitted a command to the remote server using
+ * an asynchronous execution function, the command might
+ * have not yet completed. Check to see if a command is
+ * still being processed by the remote server, and if so,
+ * request cancellation of the command; if not, abort
+ * gracefully.
+ */
+ if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
+ {
+ PGcancel *cancel;
+ char errbuf[256];
+
+ if ((cancel = PQgetCancel(entry->conn)))
+ {
+ PQcancel(cancel, errbuf, sizeof(errbuf));
+ PQfreeCancel(cancel);
+ }
+ break;
+ }
/* If we're aborting, abort all remote transactions too */
res = PQexec(entry->conn, "ABORT TRANSACTION");
/* Note: can't throw ERROR, it would be infinite loop */
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index ee0220a..2b6a61b 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -32,6 +32,7 @@
#include "optimizer/var.h"
#include "optimizer/tlist.h"
#include "parser/parsetree.h"
+#include "storage/latch.h"
#include "utils/builtins.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -3131,6 +3132,7 @@ execute_dml_stmt(ForeignScanState *node)
ExprContext *econtext = node->ss.ps.ps_ExprContext;
int numParams = dmstate->numParams;
const char **values = dmstate->param_values;
+ int wc;
/*
* Construct array of query parameter values in text format.
@@ -3147,12 +3149,42 @@ execute_dml_stmt(ForeignScanState *node)
* parameter (see deparse.c), the "inference" is trivial and will produce
* the desired result. This allows us to avoid assuming that the remote
* server has the same OIDs we do for the parameters' types.
+ */
+ if (!PQsendQueryParams(dmstate->conn, dmstate->query, numParams,
+ NULL, values, NULL, NULL, 0))
+ pgfdw_report_error(ERROR, NULL, dmstate->conn, true, dmstate->query);
+
+ /*
+ * Receive data until PQgetResult is ready to get the result without
+ * blocking.
+ */
+ while (PQisBusy(dmstate->conn))
+ {
+ /* Sleep until there's something to do */
+ wc = WaitLatchOrSocket(MyLatch,
+ WL_LATCH_SET | WL_SOCKET_READABLE,
+ PQsocket(dmstate->conn),
+ -1L);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+
+ /* Data available in socket */
+ if (wc & WL_SOCKET_READABLE)
+ {
+ if (!PQconsumeInput(dmstate->conn))
+ pgfdw_report_error(ERROR, NULL, dmstate->conn, true,
+ dmstate->query);
+ }
+ }
+
+ /*
+ * Get the result
*
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- dmstate->result = PQexecParams(dmstate->conn, dmstate->query,
- numParams, NULL, values, NULL, NULL, 0);
+ dmstate->result = PQgetResult(dmstate->conn);
if (PQresultStatus(dmstate->result) !=
(dmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, dmstate->result, dmstate->conn, true,
On 2016/04/08 22:21, Rushabh Lathia wrote:
On Fri, Apr 8, 2016 at 6:28 PM, Robert Haas <robertmhaas@gmail.com
<mailto:robertmhaas@gmail.com>> wrote:
The comment just before the second hunk in the patch says:
* We don't use a PG_TRY block here, so be careful not to
throw error
* without releasing the PGresult.But the patch adds a whole bunch of new things there that seem like
they can error out, like CHECK_FOR_INTERRUPTS(), for example. Isn't
that a problem?
Basically we fetching the PGresult, after the newly added hunk, so there
should not be any problem.But yes comment is definitely at wrong place.
PFA patch with correction.
I agree with Rushabh. Thanks for updating the patch!
Another thing I'd like to propose to revise the patch is to call
pgfdw_report_error in the newly added hunk, with the clear argument set
to *false*. The PGresult argument is NULL there, so no need to release
the PGresult.
Attached is an updated patch.
Best regards,
Etsuro Fujita
Attachments:
pgfdw-direct-modify-v4.patchtext/x-patch; name=pgfdw-direct-modify-v4.patchDownload
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 189f290..8820597 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -598,6 +598,26 @@ pgfdw_xact_callback(XactEvent event, void *arg)
case XACT_EVENT_ABORT:
/* Assume we might have lost track of prepared statements */
entry->have_error = true;
+ /*
+ * If we had submitted a command to the remote server using
+ * an asynchronous execution function, the command might
+ * have not yet completed. Check to see if a command is
+ * still being processed by the remote server, and if so,
+ * request cancellation of the command; if not, abort
+ * gracefully.
+ */
+ if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
+ {
+ PGcancel *cancel;
+ char errbuf[256];
+
+ if ((cancel = PQgetCancel(entry->conn)))
+ {
+ PQcancel(cancel, errbuf, sizeof(errbuf));
+ PQfreeCancel(cancel);
+ }
+ break;
+ }
/* If we're aborting, abort all remote transactions too */
res = PQexec(entry->conn, "ABORT TRANSACTION");
/* Note: can't throw ERROR, it would be infinite loop */
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index ee0220a..0378f1d 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -32,6 +32,7 @@
#include "optimizer/var.h"
#include "optimizer/tlist.h"
#include "parser/parsetree.h"
+#include "storage/latch.h"
#include "utils/builtins.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -3131,6 +3132,7 @@ execute_dml_stmt(ForeignScanState *node)
ExprContext *econtext = node->ss.ps.ps_ExprContext;
int numParams = dmstate->numParams;
const char **values = dmstate->param_values;
+ int wc;
/*
* Construct array of query parameter values in text format.
@@ -3147,12 +3149,42 @@ execute_dml_stmt(ForeignScanState *node)
* parameter (see deparse.c), the "inference" is trivial and will produce
* the desired result. This allows us to avoid assuming that the remote
* server has the same OIDs we do for the parameters' types.
+ */
+ if (!PQsendQueryParams(dmstate->conn, dmstate->query, numParams,
+ NULL, values, NULL, NULL, 0))
+ pgfdw_report_error(ERROR, NULL, dmstate->conn, false, dmstate->query);
+
+ /*
+ * Receive data until PQgetResult is ready to get the result without
+ * blocking.
+ */
+ while (PQisBusy(dmstate->conn))
+ {
+ /* Sleep until there's something to do */
+ wc = WaitLatchOrSocket(MyLatch,
+ WL_LATCH_SET | WL_SOCKET_READABLE,
+ PQsocket(dmstate->conn),
+ -1L);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+
+ /* Data available in socket */
+ if (wc & WL_SOCKET_READABLE)
+ {
+ if (!PQconsumeInput(dmstate->conn))
+ pgfdw_report_error(ERROR, NULL, dmstate->conn, false,
+ dmstate->query);
+ }
+ }
+
+ /*
+ * Get the result
*
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- dmstate->result = PQexecParams(dmstate->conn, dmstate->query,
- numParams, NULL, values, NULL, NULL, 0);
+ dmstate->result = PQgetResult(dmstate->conn);
if (PQresultStatus(dmstate->result) !=
(dmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, dmstate->result, dmstate->conn, true,
On Mon, Apr 11, 2016 at 11:30 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
I agree with Rushabh. Thanks for updating the patch!
Yes, not using a PG_TRY/CATCH block is better. We are not doing
anything that need to clean up PGresult in case of an unplanned
failure.
Another thing I'd like to propose to revise the patch is to call
pgfdw_report_error in the newly added hunk, with the clear argument set to
*false*. The PGresult argument is NULL there, so no need to release the
PGresult.
Sure, this saves a couple of cycles. PQclear is anyway smart enough to
handle NULL results correctly, but this way is better.
Attached is an updated patch.
+ if (wc & WL_SOCKET_READABLE)
+ {
+ if (!PQconsumeInput(dmstate->conn))
+ pgfdw_report_error(ERROR, NULL, dmstate->conn, false,
+ dmstate->query);
+ }
OK, so here we would fail with ERRCODE_CONNECTION_FAILURE in the case
where the socket is readable but no data can be consumed. I guess it
makes sense.
+ if ((cancel = PQgetCancel(entry->conn)))
+ {
+ PQcancel(cancel, errbuf, sizeof(errbuf));
+ PQfreeCancel(cancel);
+ }
Wouldn't it be better to issue a WARNING here if PQcancel does not succeed?
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/04/11 12:30, Michael Paquier wrote:
+ if ((cancel = PQgetCancel(entry->conn))) + { + PQcancel(cancel, errbuf, sizeof(errbuf)); + PQfreeCancel(cancel); + } Wouldn't it be better to issue a WARNING here if PQcancel does not succeed?
Seems like a good idea. Attached is an updated version of the patch.
Thanks for the review!
Best regards,
Etsuro Fujita
Attachments:
pgfdw-direct-modify-v5.patchtext/x-patch; name=pgfdw-direct-modify-v5.patchDownload
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 189f290..da1758b 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -598,6 +598,34 @@ pgfdw_xact_callback(XactEvent event, void *arg)
case XACT_EVENT_ABORT:
/* Assume we might have lost track of prepared statements */
entry->have_error = true;
+ /*
+ * If we had submitted a command to the remote server using
+ * an asynchronous execution function, the command might
+ * have not yet completed. Check to see if a command is
+ * still being processed by the remote server, and if so,
+ * request cancellation of the command; if not, abort
+ * gracefully.
+ */
+ if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
+ {
+ PGcancel *cancel;
+ char errbuf[256];
+
+ if ((cancel = PQgetCancel(entry->conn)))
+ {
+ /*
+ * Note: can't throw ERROR, it would be infinite
+ * loop
+ */
+ if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ errbuf)));
+ PQfreeCancel(cancel);
+ }
+ break;
+ }
/* If we're aborting, abort all remote transactions too */
res = PQexec(entry->conn, "ABORT TRANSACTION");
/* Note: can't throw ERROR, it would be infinite loop */
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index ee0220a..0378f1d 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -32,6 +32,7 @@
#include "optimizer/var.h"
#include "optimizer/tlist.h"
#include "parser/parsetree.h"
+#include "storage/latch.h"
#include "utils/builtins.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -3131,6 +3132,7 @@ execute_dml_stmt(ForeignScanState *node)
ExprContext *econtext = node->ss.ps.ps_ExprContext;
int numParams = dmstate->numParams;
const char **values = dmstate->param_values;
+ int wc;
/*
* Construct array of query parameter values in text format.
@@ -3147,12 +3149,42 @@ execute_dml_stmt(ForeignScanState *node)
* parameter (see deparse.c), the "inference" is trivial and will produce
* the desired result. This allows us to avoid assuming that the remote
* server has the same OIDs we do for the parameters' types.
+ */
+ if (!PQsendQueryParams(dmstate->conn, dmstate->query, numParams,
+ NULL, values, NULL, NULL, 0))
+ pgfdw_report_error(ERROR, NULL, dmstate->conn, false, dmstate->query);
+
+ /*
+ * Receive data until PQgetResult is ready to get the result without
+ * blocking.
+ */
+ while (PQisBusy(dmstate->conn))
+ {
+ /* Sleep until there's something to do */
+ wc = WaitLatchOrSocket(MyLatch,
+ WL_LATCH_SET | WL_SOCKET_READABLE,
+ PQsocket(dmstate->conn),
+ -1L);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+
+ /* Data available in socket */
+ if (wc & WL_SOCKET_READABLE)
+ {
+ if (!PQconsumeInput(dmstate->conn))
+ pgfdw_report_error(ERROR, NULL, dmstate->conn, false,
+ dmstate->query);
+ }
+ }
+
+ /*
+ * Get the result
*
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- dmstate->result = PQexecParams(dmstate->conn, dmstate->query,
- numParams, NULL, values, NULL, NULL, 0);
+ dmstate->result = PQgetResult(dmstate->conn);
if (PQresultStatus(dmstate->result) !=
(dmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, dmstate->result, dmstate->conn, true,
On Mon, Apr 11, 2016 at 5:16 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
On 2016/04/11 12:30, Michael Paquier wrote:
+ if ((cancel = PQgetCancel(entry->conn))) + { + PQcancel(cancel, errbuf, sizeof(errbuf)); + PQfreeCancel(cancel); + } Wouldn't it be better to issue a WARNING here if PQcancel does not succeed?Seems like a good idea. Attached is an updated version of the patch.
Thanks for the new version. The patch looks good to me.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Apr 11, 2016 at 9:45 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:
On Mon, Apr 11, 2016 at 5:16 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:On 2016/04/11 12:30, Michael Paquier wrote:
+ if ((cancel = PQgetCancel(entry->conn))) + { + PQcancel(cancel, errbuf, sizeof(errbuf)); + PQfreeCancel(cancel); + } Wouldn't it be better to issue a WARNING here if PQcancel does not succeed?Seems like a good idea. Attached is an updated version of the patch.
Thanks for the new version. The patch looks good to me.
I'm wondering why we are fixing this specific case and not any of the
other calls to PQexec() or PQexecParams() in postgres_fdw.c.
I mean, many of those instances are cases where the query isn't likely
to run for very long, but certainly "FETCH %d FROM c%u" is in theory
just as bad as the new code introduced in 9.6. In practice, it
probably isn't, because we're probably only fetching 50 rows at a time
rather than potentially a lot more, but if we're fixing this code up
to be interrupt-safe, maybe we should fix it all at the same time.
Even for the short-running queries like CLOSE and DEALLOCATE, it seems
possible that there could be a network-related hang which you might
want to interrupt.
How about we encapsulate the while (PQisBusy(...)) loop into a new
function pgfdw_get_result(), which can be called after first calling
PQsendQueryParams()? So then this code will say dmstate->result =
pgfdw_get_result(dmstate->conn). And we can do something similar for
the other call to PQexecParams() in create_cursor(). Then let's also
add something like pgfdw_exec_query() which calls PQsendQuery() and
then pgfdw_get_result, and use that to replace all of the existing
calls to PQexec().
Then all the SQL postgres_fdw executes would be interruptible, not
just the new stuff.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/04/13 3:14, Robert Haas wrote:
I'm wondering why we are fixing this specific case and not any of the
other calls to PQexec() or PQexecParams() in postgres_fdw.c.I mean, many of those instances are cases where the query isn't likely
to run for very long, but certainly "FETCH %d FROM c%u" is in theory
just as bad as the new code introduced in 9.6. In practice, it
probably isn't, because we're probably only fetching 50 rows at a time
rather than potentially a lot more, but if we're fixing this code up
to be interrupt-safe, maybe we should fix it all at the same time.
Even for the short-running queries like CLOSE and DEALLOCATE, it seems
possible that there could be a network-related hang which you might
want to interrupt.
Actually, I was wondering, too, but I didn't propose that because, as
far as I know, there are no reports from the field. But I agree with you.
How about we encapsulate the while (PQisBusy(...)) loop into a new
function pgfdw_get_result(), which can be called after first calling
PQsendQueryParams()? So then this code will say dmstate->result =
pgfdw_get_result(dmstate->conn). And we can do something similar for
the other call to PQexecParams() in create_cursor(). Then let's also
add something like pgfdw_exec_query() which calls PQsendQuery() and
then pgfdw_get_result, and use that to replace all of the existing
calls to PQexec().Then all the SQL postgres_fdw executes would be interruptible, not
just the new stuff.
Seems like a good idea. Will do.
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Apr 13, 2016 at 11:24 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
On 2016/04/13 3:14, Robert Haas wrote:
I'm wondering why we are fixing this specific case and not any of the
other calls to PQexec() or PQexecParams() in postgres_fdw.c.I mean, many of those instances are cases where the query isn't likely
to run for very long, but certainly "FETCH %d FROM c%u" is in theory
just as bad as the new code introduced in 9.6. In practice, it
probably isn't, because we're probably only fetching 50 rows at a time
rather than potentially a lot more, but if we're fixing this code up
to be interrupt-safe, maybe we should fix it all at the same time.
Even for the short-running queries like CLOSE and DEALLOCATE, it seems
possible that there could be a network-related hang which you might
want to interrupt.Actually, I was wondering, too, but I didn't propose that because, as far as
I know, there are no reports from the field. But I agree with you.
For something that is HEAD-only that's a great idea to put everything
into the same flag like that.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Apr 12, 2016 at 10:24 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
How about we encapsulate the while (PQisBusy(...)) loop into a new
function pgfdw_get_result(), which can be called after first calling
PQsendQueryParams()? So then this code will say dmstate->result =
pgfdw_get_result(dmstate->conn). And we can do something similar for
the other call to PQexecParams() in create_cursor(). Then let's also
add something like pgfdw_exec_query() which calls PQsendQuery() and
then pgfdw_get_result, and use that to replace all of the existing
calls to PQexec().Then all the SQL postgres_fdw executes would be interruptible, not
just the new stuff.Seems like a good idea. Will do.
When will you do this? We are on a bit of a time budget here.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Apr 13, 2016 at 9:46 PM, Robert Haas <robertmhaas@gmail.com> wrote:
On Tue, Apr 12, 2016 at 10:24 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:How about we encapsulate the while (PQisBusy(...)) loop into a new
function pgfdw_get_result(), which can be called after first calling
PQsendQueryParams()? So then this code will say dmstate->result =
pgfdw_get_result(dmstate->conn). And we can do something similar for
the other call to PQexecParams() in create_cursor(). Then let's also
add something like pgfdw_exec_query() which calls PQsendQuery() and
then pgfdw_get_result, and use that to replace all of the existing
calls to PQexec().Then all the SQL postgres_fdw executes would be interruptible, not
just the new stuff.Seems like a good idea. Will do.
When will you do this? We are on a bit of a time budget here.
Fujita-san, I can code that tomorrow or in two days if need be. That
should not be an issue from here.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/04/13 21:50, Michael Paquier wrote:
On Wed, Apr 13, 2016 at 9:46 PM, Robert Haas <robertmhaas@gmail.com> wrote:
On Tue, Apr 12, 2016 at 10:24 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:How about we encapsulate the while (PQisBusy(...)) loop into a new
function pgfdw_get_result(), which can be called after first calling
PQsendQueryParams()? So then this code will say dmstate->result =
pgfdw_get_result(dmstate->conn). And we can do something similar for
the other call to PQexecParams() in create_cursor(). Then let's also
add something like pgfdw_exec_query() which calls PQsendQuery() and
then pgfdw_get_result, and use that to replace all of the existing
calls to PQexec().Then all the SQL postgres_fdw executes would be interruptible, not
just the new stuff.
Seems like a good idea. Will do.
When will you do this? We are on a bit of a time budget here.
Fujita-san, I can code that tomorrow or in two days if need be. That
should not be an issue from here.
I would be happy if you work on that.
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Apr 14, 2016 at 10:44 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
On 2016/04/13 21:50, Michael Paquier wrote:
On Wed, Apr 13, 2016 at 9:46 PM, Robert Haas <robertmhaas@gmail.com>
wrote:On Tue, Apr 12, 2016 at 10:24 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:How about we encapsulate the while (PQisBusy(...)) loop into a new
function pgfdw_get_result(), which can be called after first calling
PQsendQueryParams()? So then this code will say dmstate->result =
pgfdw_get_result(dmstate->conn). And we can do something similar for
the other call to PQexecParams() in create_cursor(). Then let's also
add something like pgfdw_exec_query() which calls PQsendQuery() and
then pgfdw_get_result, and use that to replace all of the existing
calls to PQexec().Then all the SQL postgres_fdw executes would be interruptible, not
just the new stuff.I would be happy if you work on that.
OK, so I have finished with the attached. One thing that I noticed in
the previous patch version is that it completely ignored cases where
multiple PGresult could be returned by server, say when multiple
queries are sent in the same string: PQexec gets always the last one,
so I think that we had better do the same here. I have switched all
the PQexec calls to a custom routine that combines
PQsendQuery/PQgetResult, and PQexecParams is switched to
PQsendQueryParams/PQgetResult. This structure allows all queries run
though postgres_fdw.c to be interruptible.
--
Michael
Attachments:
pgfdw-interrupt-v2.patchtext/x-patch; charset=US-ASCII; name=pgfdw-interrupt-v2.patchDownload
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 189f290..64a00b4 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -17,6 +17,7 @@
#include "access/xact.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
+#include "storage/latch.h"
#include "utils/hsearch.h"
#include "utils/memutils.h"
@@ -448,6 +449,67 @@ GetPrepStmtNumber(PGconn *conn)
}
/*
+ * Execute query in non-blocking mode and fetch back result
+ */
+PGresult *
+pgfdw_exec_query(PGconn *conn, const char *query)
+{
+ if (!PQsendQuery(conn, query))
+ pgfdw_report_error(ERROR, NULL, conn, false, query);
+
+ return pgfdw_get_result(conn, query);
+}
+
+/*
+ * Scan for results of query run in non-waiting mode
+ *
+ * This should be run after executing a query on the remote server and
+ * offers quick responsiveness by checking for any interruptions. The
+ * last result is returned back to caller, and previous results are
+ * consumed. Caller is responsible for the error handling on the fetched
+ * result.
+ */
+PGresult *
+pgfdw_get_result(PGconn *conn, const char *query)
+{
+ PGresult *last_res;
+
+ for (;;)
+ {
+ PGresult *res;
+
+ while (PQisBusy(conn))
+ {
+ int wc;
+
+ /* Sleep until there's something to do */
+ wc = WaitLatchOrSocket(MyLatch,
+ WL_LATCH_SET | WL_SOCKET_READABLE,
+ PQsocket(conn),
+ -1L);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+
+ /* Data available in socket */
+ if (wc & WL_SOCKET_READABLE)
+ {
+ if (!PQconsumeInput(conn))
+ pgfdw_report_error(ERROR, NULL, conn, false, query);
+ }
+ }
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ break;
+
+ last_res = res;
+ }
+
+ return last_res;
+}
+
+/*
* Report an error we got from the remote server.
*
* elevel: error level to use (typically ERROR, but might be less)
@@ -598,6 +660,32 @@ pgfdw_xact_callback(XactEvent event, void *arg)
case XACT_EVENT_ABORT:
/* Assume we might have lost track of prepared statements */
entry->have_error = true;
+
+ /*
+ * If a command has been submitted to the remote server
+ * using an asynchronous execution function, the command
+ * might not have yet completed. Check to see if a command
+ * is still being processed by the remote server, and if so,
+ * request cancellation of the command; if not, abort
+ * gracefully.
+ */
+ if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
+ {
+ PGcancel *cancel;
+ char errbuf[256];
+
+ if ((cancel = PQgetCancel(entry->conn)))
+ {
+ if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ errbuf)));
+ PQfreeCancel(cancel);
+ }
+ break;
+ }
+
/* If we're aborting, abort all remote transactions too */
res = PQexec(entry->conn, "ABORT TRANSACTION");
/* Note: can't throw ERROR, it would be infinite loop */
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index ee0220a..31f5bb2 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -1421,7 +1421,7 @@ postgresReScanForeignScan(ForeignScanState *node)
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexec(fsstate->conn, sql);
+ res = pgfdw_exec_query(fsstate->conn, sql);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, fsstate->conn, true, sql);
PQclear(res);
@@ -1950,7 +1950,7 @@ postgresEndForeignModify(EState *estate,
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexec(fmstate->conn, sql);
+ res = pgfdw_exec_query(fmstate->conn, sql);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, fmstate->conn, true, sql);
PQclear(res);
@@ -2712,7 +2712,7 @@ get_remote_estimate(const char *sql, PGconn *conn,
/*
* Execute EXPLAIN remotely.
*/
- res = PQexec(conn, sql);
+ res = pgfdw_exec_query(conn, sql);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, sql);
@@ -2821,8 +2821,11 @@ create_cursor(ForeignScanState *node)
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexecParams(conn, buf.data, numParams, NULL, values,
- NULL, NULL, 0);
+ if (!PQsendQueryParams(conn, buf.data, numParams,
+ NULL, values, NULL, NULL, 0))
+ pgfdw_report_error(ERROR, NULL, conn, false, buf.data);
+
+ res = pgfdw_get_result(conn, buf.data);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, conn, true, fsstate->query);
PQclear(res);
@@ -2868,7 +2871,7 @@ fetch_more_data(ForeignScanState *node)
snprintf(sql, sizeof(sql), "FETCH %d FROM c%u",
fsstate->fetch_size, fsstate->cursor_number);
- res = PQexec(conn, sql);
+ res = pgfdw_exec_query(conn, sql);
/* On error, report the original query, not the FETCH. */
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, fsstate->query);
@@ -2978,7 +2981,7 @@ close_cursor(PGconn *conn, unsigned int cursor_number)
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexec(conn, sql);
+ res = pgfdw_exec_query(conn, sql);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, conn, true, sql);
PQclear(res);
@@ -3151,8 +3154,12 @@ execute_dml_stmt(ForeignScanState *node)
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- dmstate->result = PQexecParams(dmstate->conn, dmstate->query,
- numParams, NULL, values, NULL, NULL, 0);
+ if (!PQsendQueryParams(dmstate->conn, dmstate->query, numParams,
+ NULL, values, NULL, NULL, 0))
+ pgfdw_report_error(ERROR, NULL, dmstate->conn, false, dmstate->query);
+
+ dmstate->result = pgfdw_get_result(dmstate->conn, dmstate->query);
+
if (PQresultStatus(dmstate->result) !=
(dmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, dmstate->result, dmstate->conn, true,
@@ -3355,7 +3362,7 @@ postgresAnalyzeForeignTable(Relation relation,
/* In what follows, do not risk leaking any PGresults. */
PG_TRY();
{
- res = PQexec(conn, sql.data);
+ res = pgfdw_exec_query(conn, sql.data);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, sql.data);
@@ -3449,7 +3456,7 @@ postgresAcquireSampleRowsFunc(Relation relation, int elevel,
/* In what follows, do not risk leaking any PGresults. */
PG_TRY();
{
- res = PQexec(conn, sql.data);
+ res = pgfdw_exec_query(conn, sql.data);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, conn, false, sql.data);
PQclear(res);
@@ -3500,7 +3507,7 @@ postgresAcquireSampleRowsFunc(Relation relation, int elevel,
snprintf(fetch_sql, sizeof(fetch_sql), "FETCH %d FROM c%u",
fetch_size, cursor_number);
- res = PQexec(conn, fetch_sql);
+ res = pgfdw_exec_query(conn, fetch_sql);
/* On error, report the original query, not the FETCH. */
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, sql.data);
@@ -3675,7 +3682,7 @@ postgresImportForeignSchema(ImportForeignSchemaStmt *stmt, Oid serverOid)
appendStringInfoString(&buf, "SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname = ");
deparseStringLiteral(&buf, stmt->remote_schema);
- res = PQexec(conn, buf.data);
+ res = pgfdw_exec_query(conn, buf.data);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, buf.data);
@@ -3774,7 +3781,7 @@ postgresImportForeignSchema(ImportForeignSchemaStmt *stmt, Oid serverOid)
appendStringInfoString(&buf, " ORDER BY c.relname, a.attnum");
/* Fetch the data */
- res = PQexec(conn, buf.data);
+ res = pgfdw_exec_query(conn, buf.data);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, buf.data);
diff --git a/contrib/postgres_fdw/postgres_fdw.h b/contrib/postgres_fdw/postgres_fdw.h
index 3a11d99..574b07d 100644
--- a/contrib/postgres_fdw/postgres_fdw.h
+++ b/contrib/postgres_fdw/postgres_fdw.h
@@ -103,6 +103,8 @@ extern PGconn *GetConnection(UserMapping *user, bool will_prep_stmt);
extern void ReleaseConnection(PGconn *conn);
extern unsigned int GetCursorNumber(PGconn *conn);
extern unsigned int GetPrepStmtNumber(PGconn *conn);
+extern PGresult *pgfdw_get_result(PGconn *conn, const char *query);
+extern PGresult *pgfdw_exec_query(PGconn *conn, const char *query);
extern void pgfdw_report_error(int elevel, PGresult *res, PGconn *conn,
bool clear, const char *sql);
On 2016/04/15 14:31, Michael Paquier wrote:
On Thu, Apr 14, 2016 at 10:44 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:On 2016/04/13 21:50, Michael Paquier wrote:
On Wed, Apr 13, 2016 at 9:46 PM, Robert Haas <robertmhaas@gmail.com>
wrote:On Tue, Apr 12, 2016 at 10:24 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:How about we encapsulate the while (PQisBusy(...)) loop into a new
function pgfdw_get_result(), which can be called after first calling
PQsendQueryParams()? So then this code will say dmstate->result =
pgfdw_get_result(dmstate->conn). And we can do something similar for
the other call to PQexecParams() in create_cursor(). Then let's also
add something like pgfdw_exec_query() which calls PQsendQuery() and
then pgfdw_get_result, and use that to replace all of the existing
calls to PQexec().Then all the SQL postgres_fdw executes would be interruptible, not
just the new stuff.
I would be happy if you work on that.
OK, so I have finished with the attached.
Thank you for working on that!
One thing that I noticed in
the previous patch version is that it completely ignored cases where
multiple PGresult could be returned by server, say when multiple
queries are sent in the same string: PQexec gets always the last one,
so I think that we had better do the same here.
Seems reasonable.
so I think that we had better do the same here. I have switched all
the PQexec calls to a custom routine that combines
PQsendQuery/PQgetResult, and PQexecParams is switched to
PQsendQueryParams/PQgetResult. This structure allows all queries run
though postgres_fdw.c to be interruptible.
How about doing something similar for PQprepare/PQexecPrepared in
postgresExecForeignInsert, postgresExecForeignUpdate, and
postgresExecForeignDelete? Also, how about doing that for PQexec in
connection.c?
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Apr 15, 2016 at 8:25 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
How about doing something similar for PQprepare/PQexecPrepared in
postgresExecForeignInsert, postgresExecForeignUpdate, and
postgresExecForeignDelete?
Yes, I hesitated to touch those, but they are good candidates for this
new interface, and actually it has proved not to be complicated to
plug in the new routines in those code paths.
Also, how about doing that for PQexec in connection.c?
Here I disagree, this is not adapted. All the PQexec calls are part of
callbacks that are triggered on failures, and we rely on such a
callback when issuing PQcancel. do_sql_command runs commands that take
a short amount of time, so I think as well that it is fine as-is with
PQexec.
--
Michael
Attachments:
pgfdw-interrupt-v3.patchtext/x-diff; charset=US-ASCII; name=pgfdw-interrupt-v3.patchDownload
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 189f290..64a00b4 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -17,6 +17,7 @@
#include "access/xact.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
+#include "storage/latch.h"
#include "utils/hsearch.h"
#include "utils/memutils.h"
@@ -448,6 +449,67 @@ GetPrepStmtNumber(PGconn *conn)
}
/*
+ * Execute query in non-blocking mode and fetch back result
+ */
+PGresult *
+pgfdw_exec_query(PGconn *conn, const char *query)
+{
+ if (!PQsendQuery(conn, query))
+ pgfdw_report_error(ERROR, NULL, conn, false, query);
+
+ return pgfdw_get_result(conn, query);
+}
+
+/*
+ * Scan for results of query run in non-waiting mode
+ *
+ * This should be run after executing a query on the remote server and
+ * offers quick responsiveness by checking for any interruptions. The
+ * last result is returned back to caller, and previous results are
+ * consumed. Caller is responsible for the error handling on the fetched
+ * result.
+ */
+PGresult *
+pgfdw_get_result(PGconn *conn, const char *query)
+{
+ PGresult *last_res;
+
+ for (;;)
+ {
+ PGresult *res;
+
+ while (PQisBusy(conn))
+ {
+ int wc;
+
+ /* Sleep until there's something to do */
+ wc = WaitLatchOrSocket(MyLatch,
+ WL_LATCH_SET | WL_SOCKET_READABLE,
+ PQsocket(conn),
+ -1L);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+
+ /* Data available in socket */
+ if (wc & WL_SOCKET_READABLE)
+ {
+ if (!PQconsumeInput(conn))
+ pgfdw_report_error(ERROR, NULL, conn, false, query);
+ }
+ }
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ break;
+
+ last_res = res;
+ }
+
+ return last_res;
+}
+
+/*
* Report an error we got from the remote server.
*
* elevel: error level to use (typically ERROR, but might be less)
@@ -598,6 +660,32 @@ pgfdw_xact_callback(XactEvent event, void *arg)
case XACT_EVENT_ABORT:
/* Assume we might have lost track of prepared statements */
entry->have_error = true;
+
+ /*
+ * If a command has been submitted to the remote server
+ * using an asynchronous execution function, the command
+ * might not have yet completed. Check to see if a command
+ * is still being processed by the remote server, and if so,
+ * request cancellation of the command; if not, abort
+ * gracefully.
+ */
+ if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
+ {
+ PGcancel *cancel;
+ char errbuf[256];
+
+ if ((cancel = PQgetCancel(entry->conn)))
+ {
+ if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ errbuf)));
+ PQfreeCancel(cancel);
+ }
+ break;
+ }
+
/* If we're aborting, abort all remote transactions too */
res = PQexec(entry->conn, "ABORT TRANSACTION");
/* Note: can't throw ERROR, it would be infinite loop */
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index ee0220a..4566cf4 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -1421,7 +1421,7 @@ postgresReScanForeignScan(ForeignScanState *node)
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexec(fsstate->conn, sql);
+ res = pgfdw_exec_query(fsstate->conn, sql);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, fsstate->conn, true, sql);
PQclear(res);
@@ -1754,13 +1754,16 @@ postgresExecForeignInsert(EState *estate,
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexecPrepared(fmstate->conn,
- fmstate->p_name,
- fmstate->p_nums,
- p_values,
- NULL,
- NULL,
- 0);
+ if (!PQsendQueryPrepared(fmstate->conn,
+ fmstate->p_name,
+ fmstate->p_nums,
+ p_values,
+ NULL,
+ NULL,
+ 0))
+ pgfdw_report_error(ERROR, NULL, fmstate->conn, true, fmstate->query);
+
+ res = pgfdw_get_result(fmstate->conn, fmstate->query);
if (PQresultStatus(res) !=
(fmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
@@ -1824,13 +1827,16 @@ postgresExecForeignUpdate(EState *estate,
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexecPrepared(fmstate->conn,
- fmstate->p_name,
- fmstate->p_nums,
- p_values,
- NULL,
- NULL,
- 0);
+ if (!PQsendQueryPrepared(fmstate->conn,
+ fmstate->p_name,
+ fmstate->p_nums,
+ p_values,
+ NULL,
+ NULL,
+ 0))
+ pgfdw_report_error(ERROR, NULL, fmstate->conn, true, fmstate->query);
+
+ res = pgfdw_get_result(fmstate->conn, fmstate->query);;
if (PQresultStatus(res) !=
(fmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
@@ -1894,13 +1900,16 @@ postgresExecForeignDelete(EState *estate,
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexecPrepared(fmstate->conn,
- fmstate->p_name,
- fmstate->p_nums,
- p_values,
- NULL,
- NULL,
- 0);
+ if (!PQsendQueryPrepared(fmstate->conn,
+ fmstate->p_name,
+ fmstate->p_nums,
+ p_values,
+ NULL,
+ NULL,
+ 0))
+ pgfdw_report_error(ERROR, NULL, fmstate->conn, true, fmstate->query);
+
+ res = pgfdw_get_result(fmstate->conn, fmstate->query);
if (PQresultStatus(res) !=
(fmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
@@ -1950,7 +1959,7 @@ postgresEndForeignModify(EState *estate,
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexec(fmstate->conn, sql);
+ res = pgfdw_exec_query(fmstate->conn, sql);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, fmstate->conn, true, sql);
PQclear(res);
@@ -2712,7 +2721,7 @@ get_remote_estimate(const char *sql, PGconn *conn,
/*
* Execute EXPLAIN remotely.
*/
- res = PQexec(conn, sql);
+ res = pgfdw_exec_query(conn, sql);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, sql);
@@ -2821,8 +2830,11 @@ create_cursor(ForeignScanState *node)
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexecParams(conn, buf.data, numParams, NULL, values,
- NULL, NULL, 0);
+ if (!PQsendQueryParams(conn, buf.data, numParams,
+ NULL, values, NULL, NULL, 0))
+ pgfdw_report_error(ERROR, NULL, conn, false, buf.data);
+
+ res = pgfdw_get_result(conn, buf.data);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, conn, true, fsstate->query);
PQclear(res);
@@ -2868,7 +2880,7 @@ fetch_more_data(ForeignScanState *node)
snprintf(sql, sizeof(sql), "FETCH %d FROM c%u",
fsstate->fetch_size, fsstate->cursor_number);
- res = PQexec(conn, sql);
+ res = pgfdw_exec_query(conn, sql);
/* On error, report the original query, not the FETCH. */
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, fsstate->query);
@@ -2978,7 +2990,7 @@ close_cursor(PGconn *conn, unsigned int cursor_number)
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexec(conn, sql);
+ res = pgfdw_exec_query(conn, sql);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, conn, true, sql);
PQclear(res);
@@ -3010,12 +3022,14 @@ prepare_foreign_modify(PgFdwModifyState *fmstate)
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQprepare(fmstate->conn,
- p_name,
- fmstate->query,
- 0,
- NULL);
-
+ if (!PQsendPrepare(fmstate->conn,
+ p_name,
+ fmstate->query,
+ 0,
+ NULL))
+ pgfdw_report_error(ERROR, NULL, fmstate->conn, true, fmstate->query);
+
+ res = pgfdw_get_result(fmstate->conn, fmstate->query);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
PQclear(res);
@@ -3151,8 +3165,12 @@ execute_dml_stmt(ForeignScanState *node)
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- dmstate->result = PQexecParams(dmstate->conn, dmstate->query,
- numParams, NULL, values, NULL, NULL, 0);
+ if (!PQsendQueryParams(dmstate->conn, dmstate->query, numParams,
+ NULL, values, NULL, NULL, 0))
+ pgfdw_report_error(ERROR, NULL, dmstate->conn, false, dmstate->query);
+
+ dmstate->result = pgfdw_get_result(dmstate->conn, dmstate->query);
+
if (PQresultStatus(dmstate->result) !=
(dmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, dmstate->result, dmstate->conn, true,
@@ -3355,7 +3373,7 @@ postgresAnalyzeForeignTable(Relation relation,
/* In what follows, do not risk leaking any PGresults. */
PG_TRY();
{
- res = PQexec(conn, sql.data);
+ res = pgfdw_exec_query(conn, sql.data);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, sql.data);
@@ -3449,7 +3467,7 @@ postgresAcquireSampleRowsFunc(Relation relation, int elevel,
/* In what follows, do not risk leaking any PGresults. */
PG_TRY();
{
- res = PQexec(conn, sql.data);
+ res = pgfdw_exec_query(conn, sql.data);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, conn, false, sql.data);
PQclear(res);
@@ -3500,7 +3518,7 @@ postgresAcquireSampleRowsFunc(Relation relation, int elevel,
snprintf(fetch_sql, sizeof(fetch_sql), "FETCH %d FROM c%u",
fetch_size, cursor_number);
- res = PQexec(conn, fetch_sql);
+ res = pgfdw_exec_query(conn, fetch_sql);
/* On error, report the original query, not the FETCH. */
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, sql.data);
@@ -3675,7 +3693,7 @@ postgresImportForeignSchema(ImportForeignSchemaStmt *stmt, Oid serverOid)
appendStringInfoString(&buf, "SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname = ");
deparseStringLiteral(&buf, stmt->remote_schema);
- res = PQexec(conn, buf.data);
+ res = pgfdw_exec_query(conn, buf.data);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, buf.data);
@@ -3774,7 +3792,7 @@ postgresImportForeignSchema(ImportForeignSchemaStmt *stmt, Oid serverOid)
appendStringInfoString(&buf, " ORDER BY c.relname, a.attnum");
/* Fetch the data */
- res = PQexec(conn, buf.data);
+ res = pgfdw_exec_query(conn, buf.data);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, buf.data);
diff --git a/contrib/postgres_fdw/postgres_fdw.h b/contrib/postgres_fdw/postgres_fdw.h
index 3a11d99..574b07d 100644
--- a/contrib/postgres_fdw/postgres_fdw.h
+++ b/contrib/postgres_fdw/postgres_fdw.h
@@ -103,6 +103,8 @@ extern PGconn *GetConnection(UserMapping *user, bool will_prep_stmt);
extern void ReleaseConnection(PGconn *conn);
extern unsigned int GetCursorNumber(PGconn *conn);
extern unsigned int GetPrepStmtNumber(PGconn *conn);
+extern PGresult *pgfdw_get_result(PGconn *conn, const char *query);
+extern PGresult *pgfdw_exec_query(PGconn *conn, const char *query);
extern void pgfdw_report_error(int elevel, PGresult *res, PGconn *conn,
bool clear, const char *sql);
On Fri, Apr 15, 2016 at 9:46 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:
On Fri, Apr 15, 2016 at 8:25 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:How about doing something similar for PQprepare/PQexecPrepared in
postgresExecForeignInsert, postgresExecForeignUpdate, and
postgresExecForeignDelete?Yes, I hesitated to touch those, but they are good candidates for this
new interface, and actually it has proved not to be complicated to
plug in the new routines in those code paths.Also, how about doing that for PQexec in connection.c?
Here I disagree, this is not adapted. All the PQexec calls are part of
callbacks that are triggered on failures, and we rely on such a
callback when issuing PQcancel. do_sql_command runs commands that take
a short amount of time, so I think as well that it is fine as-is with
PQexec.
Here is a new version. I just recalled that I forgot a PQclear() call
to clean up results.
--
Michael
Attachments:
pgfdw-interrupt-v4.patchtext/x-diff; charset=US-ASCII; name=pgfdw-interrupt-v4.patchDownload
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 189f290..33b7363 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -17,6 +17,7 @@
#include "access/xact.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
+#include "storage/latch.h"
#include "utils/hsearch.h"
#include "utils/memutils.h"
@@ -448,6 +449,68 @@ GetPrepStmtNumber(PGconn *conn)
}
/*
+ * Execute query in non-blocking mode and fetch back result
+ */
+PGresult *
+pgfdw_exec_query(PGconn *conn, const char *query)
+{
+ if (!PQsendQuery(conn, query))
+ pgfdw_report_error(ERROR, NULL, conn, false, query);
+
+ return pgfdw_get_result(conn, query);
+}
+
+/*
+ * Scan for results of query run in non-waiting mode
+ *
+ * This should be run after executing a query on the remote server and
+ * offers quick responsiveness by checking for any interruptions. The
+ * last result is returned back to caller, and previous results are
+ * consumed. Caller is responsible for the error handling on the fetched
+ * result.
+ */
+PGresult *
+pgfdw_get_result(PGconn *conn, const char *query)
+{
+ PGresult *last_res = NULL;
+
+ for (;;)
+ {
+ PGresult *res;
+
+ while (PQisBusy(conn))
+ {
+ int wc;
+
+ /* Sleep until there's something to do */
+ wc = WaitLatchOrSocket(MyLatch,
+ WL_LATCH_SET | WL_SOCKET_READABLE,
+ PQsocket(conn),
+ -1L);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+
+ /* Data available in socket */
+ if (wc & WL_SOCKET_READABLE)
+ {
+ if (!PQconsumeInput(conn))
+ pgfdw_report_error(ERROR, NULL, conn, false, query);
+ }
+ }
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ break;
+
+ PQclear(last_res);
+ last_res = res;
+ }
+
+ return last_res;
+}
+
+/*
* Report an error we got from the remote server.
*
* elevel: error level to use (typically ERROR, but might be less)
@@ -598,6 +661,32 @@ pgfdw_xact_callback(XactEvent event, void *arg)
case XACT_EVENT_ABORT:
/* Assume we might have lost track of prepared statements */
entry->have_error = true;
+
+ /*
+ * If a command has been submitted to the remote server
+ * using an asynchronous execution function, the command
+ * might not have yet completed. Check to see if a command
+ * is still being processed by the remote server, and if so,
+ * request cancellation of the command; if not, abort
+ * gracefully.
+ */
+ if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
+ {
+ PGcancel *cancel;
+ char errbuf[256];
+
+ if ((cancel = PQgetCancel(entry->conn)))
+ {
+ if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ errbuf)));
+ PQfreeCancel(cancel);
+ }
+ break;
+ }
+
/* If we're aborting, abort all remote transactions too */
res = PQexec(entry->conn, "ABORT TRANSACTION");
/* Note: can't throw ERROR, it would be infinite loop */
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index ee0220a..4566cf4 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -1421,7 +1421,7 @@ postgresReScanForeignScan(ForeignScanState *node)
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexec(fsstate->conn, sql);
+ res = pgfdw_exec_query(fsstate->conn, sql);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, fsstate->conn, true, sql);
PQclear(res);
@@ -1754,13 +1754,16 @@ postgresExecForeignInsert(EState *estate,
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexecPrepared(fmstate->conn,
- fmstate->p_name,
- fmstate->p_nums,
- p_values,
- NULL,
- NULL,
- 0);
+ if (!PQsendQueryPrepared(fmstate->conn,
+ fmstate->p_name,
+ fmstate->p_nums,
+ p_values,
+ NULL,
+ NULL,
+ 0))
+ pgfdw_report_error(ERROR, NULL, fmstate->conn, true, fmstate->query);
+
+ res = pgfdw_get_result(fmstate->conn, fmstate->query);
if (PQresultStatus(res) !=
(fmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
@@ -1824,13 +1827,16 @@ postgresExecForeignUpdate(EState *estate,
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexecPrepared(fmstate->conn,
- fmstate->p_name,
- fmstate->p_nums,
- p_values,
- NULL,
- NULL,
- 0);
+ if (!PQsendQueryPrepared(fmstate->conn,
+ fmstate->p_name,
+ fmstate->p_nums,
+ p_values,
+ NULL,
+ NULL,
+ 0))
+ pgfdw_report_error(ERROR, NULL, fmstate->conn, true, fmstate->query);
+
+ res = pgfdw_get_result(fmstate->conn, fmstate->query);;
if (PQresultStatus(res) !=
(fmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
@@ -1894,13 +1900,16 @@ postgresExecForeignDelete(EState *estate,
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexecPrepared(fmstate->conn,
- fmstate->p_name,
- fmstate->p_nums,
- p_values,
- NULL,
- NULL,
- 0);
+ if (!PQsendQueryPrepared(fmstate->conn,
+ fmstate->p_name,
+ fmstate->p_nums,
+ p_values,
+ NULL,
+ NULL,
+ 0))
+ pgfdw_report_error(ERROR, NULL, fmstate->conn, true, fmstate->query);
+
+ res = pgfdw_get_result(fmstate->conn, fmstate->query);
if (PQresultStatus(res) !=
(fmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
@@ -1950,7 +1959,7 @@ postgresEndForeignModify(EState *estate,
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexec(fmstate->conn, sql);
+ res = pgfdw_exec_query(fmstate->conn, sql);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, fmstate->conn, true, sql);
PQclear(res);
@@ -2712,7 +2721,7 @@ get_remote_estimate(const char *sql, PGconn *conn,
/*
* Execute EXPLAIN remotely.
*/
- res = PQexec(conn, sql);
+ res = pgfdw_exec_query(conn, sql);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, sql);
@@ -2821,8 +2830,11 @@ create_cursor(ForeignScanState *node)
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexecParams(conn, buf.data, numParams, NULL, values,
- NULL, NULL, 0);
+ if (!PQsendQueryParams(conn, buf.data, numParams,
+ NULL, values, NULL, NULL, 0))
+ pgfdw_report_error(ERROR, NULL, conn, false, buf.data);
+
+ res = pgfdw_get_result(conn, buf.data);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, conn, true, fsstate->query);
PQclear(res);
@@ -2868,7 +2880,7 @@ fetch_more_data(ForeignScanState *node)
snprintf(sql, sizeof(sql), "FETCH %d FROM c%u",
fsstate->fetch_size, fsstate->cursor_number);
- res = PQexec(conn, sql);
+ res = pgfdw_exec_query(conn, sql);
/* On error, report the original query, not the FETCH. */
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, fsstate->query);
@@ -2978,7 +2990,7 @@ close_cursor(PGconn *conn, unsigned int cursor_number)
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexec(conn, sql);
+ res = pgfdw_exec_query(conn, sql);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, conn, true, sql);
PQclear(res);
@@ -3010,12 +3022,14 @@ prepare_foreign_modify(PgFdwModifyState *fmstate)
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQprepare(fmstate->conn,
- p_name,
- fmstate->query,
- 0,
- NULL);
-
+ if (!PQsendPrepare(fmstate->conn,
+ p_name,
+ fmstate->query,
+ 0,
+ NULL))
+ pgfdw_report_error(ERROR, NULL, fmstate->conn, true, fmstate->query);
+
+ res = pgfdw_get_result(fmstate->conn, fmstate->query);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
PQclear(res);
@@ -3151,8 +3165,12 @@ execute_dml_stmt(ForeignScanState *node)
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- dmstate->result = PQexecParams(dmstate->conn, dmstate->query,
- numParams, NULL, values, NULL, NULL, 0);
+ if (!PQsendQueryParams(dmstate->conn, dmstate->query, numParams,
+ NULL, values, NULL, NULL, 0))
+ pgfdw_report_error(ERROR, NULL, dmstate->conn, false, dmstate->query);
+
+ dmstate->result = pgfdw_get_result(dmstate->conn, dmstate->query);
+
if (PQresultStatus(dmstate->result) !=
(dmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, dmstate->result, dmstate->conn, true,
@@ -3355,7 +3373,7 @@ postgresAnalyzeForeignTable(Relation relation,
/* In what follows, do not risk leaking any PGresults. */
PG_TRY();
{
- res = PQexec(conn, sql.data);
+ res = pgfdw_exec_query(conn, sql.data);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, sql.data);
@@ -3449,7 +3467,7 @@ postgresAcquireSampleRowsFunc(Relation relation, int elevel,
/* In what follows, do not risk leaking any PGresults. */
PG_TRY();
{
- res = PQexec(conn, sql.data);
+ res = pgfdw_exec_query(conn, sql.data);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, conn, false, sql.data);
PQclear(res);
@@ -3500,7 +3518,7 @@ postgresAcquireSampleRowsFunc(Relation relation, int elevel,
snprintf(fetch_sql, sizeof(fetch_sql), "FETCH %d FROM c%u",
fetch_size, cursor_number);
- res = PQexec(conn, fetch_sql);
+ res = pgfdw_exec_query(conn, fetch_sql);
/* On error, report the original query, not the FETCH. */
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, sql.data);
@@ -3675,7 +3693,7 @@ postgresImportForeignSchema(ImportForeignSchemaStmt *stmt, Oid serverOid)
appendStringInfoString(&buf, "SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname = ");
deparseStringLiteral(&buf, stmt->remote_schema);
- res = PQexec(conn, buf.data);
+ res = pgfdw_exec_query(conn, buf.data);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, buf.data);
@@ -3774,7 +3792,7 @@ postgresImportForeignSchema(ImportForeignSchemaStmt *stmt, Oid serverOid)
appendStringInfoString(&buf, " ORDER BY c.relname, a.attnum");
/* Fetch the data */
- res = PQexec(conn, buf.data);
+ res = pgfdw_exec_query(conn, buf.data);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, buf.data);
diff --git a/contrib/postgres_fdw/postgres_fdw.h b/contrib/postgres_fdw/postgres_fdw.h
index 3a11d99..574b07d 100644
--- a/contrib/postgres_fdw/postgres_fdw.h
+++ b/contrib/postgres_fdw/postgres_fdw.h
@@ -103,6 +103,8 @@ extern PGconn *GetConnection(UserMapping *user, bool will_prep_stmt);
extern void ReleaseConnection(PGconn *conn);
extern unsigned int GetCursorNumber(PGconn *conn);
extern unsigned int GetPrepStmtNumber(PGconn *conn);
+extern PGresult *pgfdw_get_result(PGconn *conn, const char *query);
+extern PGresult *pgfdw_exec_query(PGconn *conn, const char *query);
extern void pgfdw_report_error(int elevel, PGresult *res, PGconn *conn,
bool clear, const char *sql);
On Sat, Apr 16, 2016 at 08:59:40AM +0900, Michael Paquier wrote:
On Fri, Apr 15, 2016 at 9:46 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:On Fri, Apr 15, 2016 at 8:25 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:How about doing something similar for PQprepare/PQexecPrepared in
postgresExecForeignInsert, postgresExecForeignUpdate, and
postgresExecForeignDelete?Yes, I hesitated to touch those, but they are good candidates for this
new interface, and actually it has proved not to be complicated to
plug in the new routines in those code paths.Also, how about doing that for PQexec in connection.c?
Here I disagree, this is not adapted. All the PQexec calls are part of
callbacks that are triggered on failures, and we rely on such a
callback when issuing PQcancel. do_sql_command runs commands that take
a short amount of time, so I think as well that it is fine as-is with
PQexec.Here is a new version. I just recalled that I forgot a PQclear() call
to clean up results.
Robert, the deadline to fix this open item expired eleven days ago. The
thread had been seeing regular activity, but it has now been quiet for three
days. Do you have an updated plan for fixing this open item?
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Apr 19, 2016 at 12:16 PM, Noah Misch <noah@leadboat.com> wrote:
On Sat, Apr 16, 2016 at 08:59:40AM +0900, Michael Paquier wrote:
On Fri, Apr 15, 2016 at 9:46 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:On Fri, Apr 15, 2016 at 8:25 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:How about doing something similar for PQprepare/PQexecPrepared in
postgresExecForeignInsert, postgresExecForeignUpdate, and
postgresExecForeignDelete?Yes, I hesitated to touch those, but they are good candidates for this
new interface, and actually it has proved not to be complicated to
plug in the new routines in those code paths.Also, how about doing that for PQexec in connection.c?
Here I disagree, this is not adapted. All the PQexec calls are part of
callbacks that are triggered on failures, and we rely on such a
callback when issuing PQcancel. do_sql_command runs commands that take
a short amount of time, so I think as well that it is fine as-is with
PQexec.Here is a new version. I just recalled that I forgot a PQclear() call
to clean up results.Robert, the deadline to fix this open item expired eleven days ago. The
thread had been seeing regular activity, but it has now been quiet for three
days. Do you have an updated plan for fixing this open item?
Note for Robert: pgfdw_get_result copycats PQexec by discarding all
PQresult received except the last one. I think that's fine for the
purposes of postgres_fdw, but perhaps you have a different opinion on
the matter.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/04/19 12:26, Michael Paquier wrote:
On Tue, Apr 19, 2016 at 12:16 PM, Noah Misch <noah@leadboat.com> wrote:
On Sat, Apr 16, 2016 at 08:59:40AM +0900, Michael Paquier wrote:
Here is a new version. I just recalled that I forgot a PQclear() call
to clean up results.
Thanks for updating the patch!
Robert, the deadline to fix this open item expired eleven days ago. The
thread had been seeing regular activity, but it has now been quiet for three
days. Do you have an updated plan for fixing this open item?
Note for Robert: pgfdw_get_result copycats PQexec by discarding all
PQresult received except the last one. I think that's fine for the
purposes of postgres_fdw, but perhaps you have a different opinion on
the matter.
That seemed reasonable to me, but sorry, on second thought, I'm not sure
that's still a good idea. One reason is (1) I think it's better for the
in-postgres_fdw.c functions using pgfdw_get_result to verify that there
are no more results, in itself. I think that would improve the
robustness of those functions. Another reason is I don't think
pgfdw_report_error, which is used in pgfdw_get_result, works well for
cases where the query contains multiple SQL commands. So, +1 for the
idea of simply encapsulating the while (PQisBusy(...)) loop into a new
function pgfdw_get_result().
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/04/19 12:45, Etsuro Fujita wrote:
On 2016/04/19 12:26, Michael Paquier wrote:
Note for Robert: pgfdw_get_result copycats PQexec by discarding all
PQresult received except the last one. I think that's fine for the
purposes of postgres_fdw, but perhaps you have a different opinion on
the matter.
That seemed reasonable to me, but sorry, on second thought, I'm not sure
that's still a good idea. One reason is (1) I think it's better for the
in-postgres_fdw.c functions using pgfdw_get_result to verify that there
are no more results, in itself. I think that would improve the
robustness of those functions. Another reason is I don't think
pgfdw_report_error, which is used in pgfdw_get_result, works well for
cases where the query contains multiple SQL commands. So, +1 for the
idea of simply encapsulating the while (PQisBusy(...)) loop into a new
function pgfdw_get_result().
Here is a proposed patch for that.
Other changes:
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexecPrepared(fmstate->conn,
- fmstate->p_name,
- fmstate->p_nums,
- p_values,
- NULL,
- NULL,
- 0);
+ if (!PQsendQueryPrepared(fmstate->conn,
+ fmstate->p_name,
+ fmstate->p_nums,
+ p_values,
+ NULL,
+ NULL,
+ 0))
+ pgfdw_report_error(ERROR, NULL, fmstate->conn, true, fmstate->query);
The comment "We don't use a PG_TRY block here ..." seems to be wrongly
placed, so I moved that comment. Also, I think it'd be better to call
pgfdw_report_error() with the clear argument set to false, not true,
since we don't need to clear the PGresult. Same for
postgresExecForeignUpdate, postgresExecForeignUpdate, and
prepare_foreign_modify.
What do you think about that?
Best regards,
Etsuro Fujita
Attachments:
pgfdw-interrupt-efujita.patchtext/x-patch; name=pgfdw-interrupt-efujita.patchDownload
*** a/contrib/postgres_fdw/connection.c
--- b/contrib/postgres_fdw/connection.c
***************
*** 17,22 ****
--- 17,23 ----
#include "access/xact.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
+ #include "storage/latch.h"
#include "utils/hsearch.h"
#include "utils/memutils.h"
***************
*** 448,453 **** GetPrepStmtNumber(PGconn *conn)
--- 449,537 ----
}
/*
+ * Send a query using PQsendQuery, and wait for the results.
+ *
+ * This function is interruptible by signals.
+ *
+ * Note: we assume that the query string contains a single SQL command.
+ */
+ PGresult *
+ pgfdw_exec_query(PGconn *conn, const char *query)
+ {
+ PGresult *res;
+ PGresult *last_res = NULL;
+
+ /*
+ * Submit a query. Since we don't use non-blocking mode, this also can
+ * block. But its risk is relatively small, so we ignore that for now.
+ */
+ if (!PQsendQuery(conn, query))
+ pgfdw_report_error(ERROR, NULL, conn, false, query);
+
+ /* Wait for the result */
+ res = pgfdw_get_result(conn, query);
+ if (res == NULL)
+ pgfdw_report_error(ERROR, NULL, conn, false, query);
+ last_res = res;
+
+ /*
+ * Verify that there are no more results
+ *
+ * We don't use a PG_TRY block here, so be careful not to throw error
+ * without releasing the PGresult.
+ */
+ res = pgfdw_get_result(conn, query);
+ if (res != NULL)
+ {
+ PQclear(last_res);
+ pgfdw_report_error(ERROR, res, conn, true, query);
+ }
+
+ return last_res;
+ }
+
+ /*
+ * Wait for the next result from a prior asynchronous execution function call,
+ * and return it.
+ *
+ * This function offers quick responsiveness by checking for any interruptions.
+ *
+ * Caller is responsible for the error handling on the fetched result.
+ *
+ * Note: we assume that the query string contains a single SQL command.
+ */
+ PGresult *
+ pgfdw_get_result(PGconn *conn, const char *query)
+ {
+ /*
+ * Receive data until PQgetResult is ready to get the result without
+ * blocking.
+ */
+ while (PQisBusy(conn))
+ {
+ int wc;
+
+ /* Sleep until there's something to do */
+ wc = WaitLatchOrSocket(MyLatch,
+ WL_LATCH_SET | WL_SOCKET_READABLE,
+ PQsocket(conn),
+ -1L);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+
+ /* Data available in socket */
+ if (wc & WL_SOCKET_READABLE)
+ {
+ if (!PQconsumeInput(conn))
+ pgfdw_report_error(ERROR, NULL, conn, false, query);
+ }
+ }
+
+ return PQgetResult(conn);
+ }
+
+ /*
* Report an error we got from the remote server.
*
* elevel: error level to use (typically ERROR, but might be less)
***************
*** 598,603 **** pgfdw_xact_callback(XactEvent event, void *arg)
--- 682,713 ----
case XACT_EVENT_ABORT:
/* Assume we might have lost track of prepared statements */
entry->have_error = true;
+
+ /*
+ * If a command has been submitted to the remote server
+ * using an asynchronous execution function, the command
+ * might not have yet completed. Check to see if a command
+ * is still being processed by the remote server, and if so,
+ * request cancellation of the command; if not, abort
+ * gracefully.
+ */
+ if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
+ {
+ PGcancel *cancel;
+ char errbuf[256];
+
+ if ((cancel = PQgetCancel(entry->conn)))
+ {
+ if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ errbuf)));
+ PQfreeCancel(cancel);
+ }
+ break;
+ }
+
/* If we're aborting, abort all remote transactions too */
res = PQexec(entry->conn, "ABORT TRANSACTION");
/* Note: can't throw ERROR, it would be infinite loop */
*** a/contrib/postgres_fdw/postgres_fdw.c
--- b/contrib/postgres_fdw/postgres_fdw.c
***************
*** 1421,1427 **** postgresReScanForeignScan(ForeignScanState *node)
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
! res = PQexec(fsstate->conn, sql);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, fsstate->conn, true, sql);
PQclear(res);
--- 1421,1427 ----
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
! res = pgfdw_exec_query(fsstate->conn, sql);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, fsstate->conn, true, sql);
PQclear(res);
***************
*** 1749,1766 **** postgresExecForeignInsert(EState *estate,
p_values = convert_prep_stmt_params(fmstate, NULL, slot);
/*
! * Execute the prepared statement, and check for success.
*
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
! res = PQexecPrepared(fmstate->conn,
! fmstate->p_name,
! fmstate->p_nums,
! p_values,
! NULL,
! NULL,
! 0);
if (PQresultStatus(res) !=
(fmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
--- 1749,1772 ----
p_values = convert_prep_stmt_params(fmstate, NULL, slot);
/*
! * Execute the prepared statement
! */
! if (!PQsendQueryPrepared(fmstate->conn,
! fmstate->p_name,
! fmstate->p_nums,
! p_values,
! NULL,
! NULL,
! 0))
! pgfdw_report_error(ERROR, NULL, fmstate->conn, false, fmstate->query);
!
! /*
! * Get the result, and check for success
*
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
! res = pgfdw_get_result(fmstate->conn, fmstate->query);
if (PQresultStatus(res) !=
(fmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
***************
*** 1778,1783 **** postgresExecForeignInsert(EState *estate,
--- 1784,1794 ----
/* And clean up */
PQclear(res);
+ /* Verify that there are no more results */
+ res = pgfdw_get_result(fmstate->conn, fmstate->query);
+ if (res != NULL)
+ pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
+
MemoryContextReset(fmstate->temp_cxt);
/* Return NULL if nothing was inserted on the remote end */
***************
*** 1819,1836 **** postgresExecForeignUpdate(EState *estate,
slot);
/*
! * Execute the prepared statement, and check for success.
*
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
! res = PQexecPrepared(fmstate->conn,
! fmstate->p_name,
! fmstate->p_nums,
! p_values,
! NULL,
! NULL,
! 0);
if (PQresultStatus(res) !=
(fmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
--- 1830,1853 ----
slot);
/*
! * Execute the prepared statement
! */
! if (!PQsendQueryPrepared(fmstate->conn,
! fmstate->p_name,
! fmstate->p_nums,
! p_values,
! NULL,
! NULL,
! 0))
! pgfdw_report_error(ERROR, NULL, fmstate->conn, false, fmstate->query);
!
! /*
! * Get the result, and check for success
*
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
! res = pgfdw_get_result(fmstate->conn, fmstate->query);
if (PQresultStatus(res) !=
(fmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
***************
*** 1848,1853 **** postgresExecForeignUpdate(EState *estate,
--- 1865,1875 ----
/* And clean up */
PQclear(res);
+ /* Verify that there are no more results */
+ res = pgfdw_get_result(fmstate->conn, fmstate->query);
+ if (res != NULL)
+ pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
+
MemoryContextReset(fmstate->temp_cxt);
/* Return NULL if nothing was updated on the remote end */
***************
*** 1889,1906 **** postgresExecForeignDelete(EState *estate,
NULL);
/*
! * Execute the prepared statement, and check for success.
*
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
! res = PQexecPrepared(fmstate->conn,
! fmstate->p_name,
! fmstate->p_nums,
! p_values,
! NULL,
! NULL,
! 0);
if (PQresultStatus(res) !=
(fmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
--- 1911,1934 ----
NULL);
/*
! * Execute the prepared statement
! */
! if (!PQsendQueryPrepared(fmstate->conn,
! fmstate->p_name,
! fmstate->p_nums,
! p_values,
! NULL,
! NULL,
! 0))
! pgfdw_report_error(ERROR, NULL, fmstate->conn, false, fmstate->query);
!
! /*
! * Get the result, and check for success
*
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
! res = pgfdw_get_result(fmstate->conn, fmstate->query);
if (PQresultStatus(res) !=
(fmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
***************
*** 1918,1923 **** postgresExecForeignDelete(EState *estate,
--- 1946,1956 ----
/* And clean up */
PQclear(res);
+ /* Verify that there are no more results */
+ res = pgfdw_get_result(fmstate->conn, fmstate->query);
+ if (res != NULL)
+ pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
+
MemoryContextReset(fmstate->temp_cxt);
/* Return NULL if nothing was deleted on the remote end */
***************
*** 1950,1956 **** postgresEndForeignModify(EState *estate,
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
! res = PQexec(fmstate->conn, sql);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, fmstate->conn, true, sql);
PQclear(res);
--- 1983,1989 ----
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
! res = pgfdw_exec_query(fmstate->conn, sql);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, fmstate->conn, true, sql);
PQclear(res);
***************
*** 2336,2344 **** postgresEndDirectModify(ForeignScanState *node)
if (dmstate == NULL)
return;
! /* Release PGresult */
if (dmstate->result)
PQclear(dmstate->result);
/* Release remote connection */
ReleaseConnection(dmstate->conn);
--- 2369,2388 ----
if (dmstate == NULL)
return;
! /*
! * Verify that there are no more results if necessary
! *
! * We don't use a PG_TRY block here, so be careful not to throw error
! * without releasing the PGresult.
! */
if (dmstate->result)
+ {
PQclear(dmstate->result);
+ dmstate->result = pgfdw_get_result(dmstate->conn, dmstate->query);
+ if (dmstate->result != NULL)
+ pgfdw_report_error(ERROR, dmstate->result, dmstate->conn, true,
+ dmstate->query);
+ }
/* Release remote connection */
ReleaseConnection(dmstate->conn);
***************
*** 2712,2718 **** get_remote_estimate(const char *sql, PGconn *conn,
/*
* Execute EXPLAIN remotely.
*/
! res = PQexec(conn, sql);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, sql);
--- 2756,2762 ----
/*
* Execute EXPLAIN remotely.
*/
! res = pgfdw_exec_query(conn, sql);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, sql);
***************
*** 2817,2832 **** create_cursor(ForeignScanState *node)
* parameter (see deparse.c), the "inference" is trivial and will produce
* the desired result. This allows us to avoid assuming that the remote
* server has the same OIDs we do for the parameters' types.
*
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
! res = PQexecParams(conn, buf.data, numParams, NULL, values,
! NULL, NULL, 0);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, conn, true, fsstate->query);
PQclear(res);
/* Mark the cursor as created, and show no tuples have been retrieved */
fsstate->cursor_exists = true;
fsstate->tuples = NULL;
--- 2861,2887 ----
* parameter (see deparse.c), the "inference" is trivial and will produce
* the desired result. This allows us to avoid assuming that the remote
* server has the same OIDs we do for the parameters' types.
+ */
+ if (!PQsendQueryParams(conn, buf.data, numParams,
+ NULL, values, NULL, NULL, 0))
+ pgfdw_report_error(ERROR, NULL, conn, false, buf.data);
+
+ /*
+ * Get the result, and check for success
*
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
! res = pgfdw_get_result(conn, buf.data);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, conn, true, fsstate->query);
PQclear(res);
+ /* Verify that there are no more results */
+ res = pgfdw_get_result(conn, buf.data);
+ if (res != NULL)
+ pgfdw_report_error(ERROR, res, conn, true, fsstate->query);
+
/* Mark the cursor as created, and show no tuples have been retrieved */
fsstate->cursor_exists = true;
fsstate->tuples = NULL;
***************
*** 2868,2874 **** fetch_more_data(ForeignScanState *node)
snprintf(sql, sizeof(sql), "FETCH %d FROM c%u",
fsstate->fetch_size, fsstate->cursor_number);
! res = PQexec(conn, sql);
/* On error, report the original query, not the FETCH. */
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, fsstate->query);
--- 2923,2929 ----
snprintf(sql, sizeof(sql), "FETCH %d FROM c%u",
fsstate->fetch_size, fsstate->cursor_number);
! res = pgfdw_exec_query(conn, sql);
/* On error, report the original query, not the FETCH. */
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, fsstate->query);
***************
*** 2978,2984 **** close_cursor(PGconn *conn, unsigned int cursor_number)
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
! res = PQexec(conn, sql);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, conn, true, sql);
PQclear(res);
--- 3033,3039 ----
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
! res = pgfdw_exec_query(conn, sql);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, conn, true, sql);
PQclear(res);
***************
*** 3010,3025 **** prepare_foreign_modify(PgFdwModifyState *fmstate)
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
! res = PQprepare(fmstate->conn,
! p_name,
! fmstate->query,
! 0,
! NULL);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
PQclear(res);
/* This action shows that the prepare has been done. */
fmstate->p_name = p_name;
}
--- 3065,3093 ----
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
! if (!PQsendPrepare(fmstate->conn,
! p_name,
! fmstate->query,
! 0,
! NULL))
! pgfdw_report_error(ERROR, NULL, fmstate->conn, false, fmstate->query);
+ /*
+ * Get the result, and check for success
+ *
+ * We don't use a PG_TRY block here, so be careful not to throw error
+ * without releasing the PGresult.
+ */
+ res = pgfdw_get_result(fmstate->conn, fmstate->query);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
PQclear(res);
+ /* Verify that there are no more results */
+ res = pgfdw_get_result(fmstate->conn, fmstate->query);
+ if (res != NULL)
+ pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
+
/* This action shows that the prepare has been done. */
fmstate->p_name = p_name;
}
***************
*** 3147,3158 **** execute_dml_stmt(ForeignScanState *node)
* parameter (see deparse.c), the "inference" is trivial and will produce
* the desired result. This allows us to avoid assuming that the remote
* server has the same OIDs we do for the parameters' types.
*
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
! dmstate->result = PQexecParams(dmstate->conn, dmstate->query,
! numParams, NULL, values, NULL, NULL, 0);
if (PQresultStatus(dmstate->result) !=
(dmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, dmstate->result, dmstate->conn, true,
--- 3215,3232 ----
* parameter (see deparse.c), the "inference" is trivial and will produce
* the desired result. This allows us to avoid assuming that the remote
* server has the same OIDs we do for the parameters' types.
+ */
+ if (!PQsendQueryParams(dmstate->conn, dmstate->query, numParams,
+ NULL, values, NULL, NULL, 0))
+ pgfdw_report_error(ERROR, NULL, dmstate->conn, false, dmstate->query);
+
+ /*
+ * Get the result, and check for success.
*
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
! dmstate->result = pgfdw_get_result(dmstate->conn, dmstate->query);
if (PQresultStatus(dmstate->result) !=
(dmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, dmstate->result, dmstate->conn, true,
***************
*** 3355,3361 **** postgresAnalyzeForeignTable(Relation relation,
/* In what follows, do not risk leaking any PGresults. */
PG_TRY();
{
! res = PQexec(conn, sql.data);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, sql.data);
--- 3429,3435 ----
/* In what follows, do not risk leaking any PGresults. */
PG_TRY();
{
! res = pgfdw_exec_query(conn, sql.data);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, sql.data);
***************
*** 3449,3455 **** postgresAcquireSampleRowsFunc(Relation relation, int elevel,
/* In what follows, do not risk leaking any PGresults. */
PG_TRY();
{
! res = PQexec(conn, sql.data);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, conn, false, sql.data);
PQclear(res);
--- 3523,3529 ----
/* In what follows, do not risk leaking any PGresults. */
PG_TRY();
{
! res = pgfdw_exec_query(conn, sql.data);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, conn, false, sql.data);
PQclear(res);
***************
*** 3500,3506 **** postgresAcquireSampleRowsFunc(Relation relation, int elevel,
snprintf(fetch_sql, sizeof(fetch_sql), "FETCH %d FROM c%u",
fetch_size, cursor_number);
! res = PQexec(conn, fetch_sql);
/* On error, report the original query, not the FETCH. */
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, sql.data);
--- 3574,3580 ----
snprintf(fetch_sql, sizeof(fetch_sql), "FETCH %d FROM c%u",
fetch_size, cursor_number);
! res = pgfdw_exec_query(conn, fetch_sql);
/* On error, report the original query, not the FETCH. */
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, sql.data);
***************
*** 3675,3681 **** postgresImportForeignSchema(ImportForeignSchemaStmt *stmt, Oid serverOid)
appendStringInfoString(&buf, "SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname = ");
deparseStringLiteral(&buf, stmt->remote_schema);
! res = PQexec(conn, buf.data);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, buf.data);
--- 3749,3755 ----
appendStringInfoString(&buf, "SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname = ");
deparseStringLiteral(&buf, stmt->remote_schema);
! res = pgfdw_exec_query(conn, buf.data);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, buf.data);
***************
*** 3774,3780 **** postgresImportForeignSchema(ImportForeignSchemaStmt *stmt, Oid serverOid)
appendStringInfoString(&buf, " ORDER BY c.relname, a.attnum");
/* Fetch the data */
! res = PQexec(conn, buf.data);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, buf.data);
--- 3848,3854 ----
appendStringInfoString(&buf, " ORDER BY c.relname, a.attnum");
/* Fetch the data */
! res = pgfdw_exec_query(conn, buf.data);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, buf.data);
*** a/contrib/postgres_fdw/postgres_fdw.h
--- b/contrib/postgres_fdw/postgres_fdw.h
***************
*** 103,108 **** extern PGconn *GetConnection(UserMapping *user, bool will_prep_stmt);
--- 103,110 ----
extern void ReleaseConnection(PGconn *conn);
extern unsigned int GetCursorNumber(PGconn *conn);
extern unsigned int GetPrepStmtNumber(PGconn *conn);
+ extern PGresult *pgfdw_get_result(PGconn *conn, const char *query);
+ extern PGresult *pgfdw_exec_query(PGconn *conn, const char *query);
extern void pgfdw_report_error(int elevel, PGresult *res, PGconn *conn,
bool clear, const char *sql);
On Tue, Apr 19, 2016 at 1:14 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
The comment "We don't use a PG_TRY block here ..." seems to be wrongly
placed, so I moved that comment. Also, I think it'd be better to call
pgfdw_report_error() with the clear argument set to false, not true, since
we don't need to clear the PGresult. Same for postgresExecForeignUpdate,
postgresExecForeignUpdate, and prepare_foreign_modify.
No objection to moving those comment blocks where pgfdw_get_result is called.
What do you think about that?
+ /* Wait for the result */
+ res = pgfdw_get_result(conn, query);
+ if (res == NULL)
+ pgfdw_report_error(ERROR, NULL, conn, false, query);
+ last_res = res;
+
+ /*
+ * Verify that there are no more results
+ *
+ * We don't use a PG_TRY block here, so be careful not to throw error
+ * without releasing the PGresult.
+ */
+ res = pgfdw_get_result(conn, query);
+ if (res != NULL)
+ {
+ PQclear(last_res);
+ pgfdw_report_error(ERROR, res, conn, true, query);
+ }
But huge objection to that because this fragilizes the current logic
postgres_fdw is based on: PQexec returns the last result to caller,
I'd rather not break that logic for 9.6 stability's sake.
A even better proof of that is the following, which just emulates what
your version of pgfdw_get_result is doing when consuming the results.
+ /* Verify that there are no more results */
+ res = pgfdw_get_result(fmstate->conn, fmstate->query);
+ if (res != NULL)
+ pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
This could even lead to incorrect errors in the future if multiple
queries are combined with those DMLs for a reason or another.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/04/19 13:59, Michael Paquier wrote:
On Tue, Apr 19, 2016 at 1:14 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:What do you think about that?
+ /* Wait for the result */ + res = pgfdw_get_result(conn, query); + if (res == NULL) + pgfdw_report_error(ERROR, NULL, conn, false, query); + last_res = res; + + /* + * Verify that there are no more results + * + * We don't use a PG_TRY block here, so be careful not to throw error + * without releasing the PGresult. + */ + res = pgfdw_get_result(conn, query); + if (res != NULL) + { + PQclear(last_res); + pgfdw_report_error(ERROR, res, conn, true, query); + }But huge objection to that because this fragilizes the current logic
postgres_fdw is based on: PQexec returns the last result to caller,
I'd rather not break that logic for 9.6 stability's sake.
IIUC, I think each query submitted by PQexec in postgres_fdw.c contains
just a single command. Maybe I'm missing something, though.
A even better proof of that is the following, which just emulates what your version of pgfdw_get_result is doing when consuming the results. + /* Verify that there are no more results */ + res = pgfdw_get_result(fmstate->conn, fmstate->query); + if (res != NULL) + pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query); This could even lead to incorrect errors in the future if multiple queries are combined with those DMLs for a reason or another.
I'd like to leave such enhancements for future work...
Thanks for the comment!
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/04/19 14:25, Etsuro Fujita wrote:
On 2016/04/19 13:59, Michael Paquier wrote:
But huge objection to that because this fragilizes the current logic
postgres_fdw is based on: PQexec returns the last result to caller,
I'd rather not break that logic for 9.6 stability's sake.
IIUC, I think each query submitted by PQexec in postgres_fdw.c contains
just a single command. Maybe I'm missing something, though.
A even better proof of that is the following, which just emulates what your version of pgfdw_get_result is doing when consuming the results. + /* Verify that there are no more results */ + res = pgfdw_get_result(fmstate->conn, fmstate->query); + if (res != NULL) + pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query); This could even lead to incorrect errors in the future if multiple queries are combined with those DMLs for a reason or another.
I'd like to leave such enhancements for future work...
On reflection, I'd like to agree with Michael on that point. As
mentioned by him, we should not lose the future extendability. And I
don't think his version of pgfdw_get_result would damage the robustness
of in-postgres_fdw.c functions using that, which was my concern. Sorry
for the noise.
Attached is an updated version of the patch, which modified Michael's
version of the patch, as I proposed in [1]/messages/by-id/5715B08A.2030404@lab.ntt.co.jp (see "Other changes:"). I
modified comments for pgfdw_get_result/pgfdw_exec_query also, mainly
because words like "non-blocking mode" there seems confusing (note that
we have PQsetnonbloking).
Best regards,
Etsuro Fujita
Attachments:
pgfdw-interrupt-v5.patchtext/x-patch; name=pgfdw-interrupt-v5.patchDownload
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 189f290..16ef38f 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -17,6 +17,7 @@
#include "access/xact.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
+#include "storage/latch.h"
#include "utils/hsearch.h"
#include "utils/memutils.h"
@@ -448,6 +449,78 @@ GetPrepStmtNumber(PGconn *conn)
}
/*
+ * Submit a query and wait for the result.
+ *
+ * This function is interruptible by signals.
+ *
+ * Caller is responsible for the error handling on the result.
+ */
+PGresult *
+pgfdw_exec_query(PGconn *conn, const char *query)
+{
+ /*
+ * Submit a query. Since we don't use non-blocking mode, this also can
+ * block. But its risk is relatively small, so we ignore that for now.
+ */
+ if (!PQsendQuery(conn, query))
+ pgfdw_report_error(ERROR, NULL, conn, false, query);
+
+ /* Wait for the result. */
+ return pgfdw_get_result(conn, query);
+}
+
+/*
+ * Wait for the result from a prior asynchronous execution function call.
+ *
+ * This function offers quick responsiveness by checking for any interruptions.
+ *
+ * This function emulates the PQexec()'s behavior of returning the last result
+ * when there are many.
+ *
+ * Caller is responsible for the error handling on the result.
+ */
+PGresult *
+pgfdw_get_result(PGconn *conn, const char *query)
+{
+ PGresult *last_res = NULL;
+
+ for (;;)
+ {
+ PGresult *res;
+
+ while (PQisBusy(conn))
+ {
+ int wc;
+
+ /* Sleep until there's something to do */
+ wc = WaitLatchOrSocket(MyLatch,
+ WL_LATCH_SET | WL_SOCKET_READABLE,
+ PQsocket(conn),
+ -1L);
+ ResetLatch(MyLatch);
+
+ CHECK_FOR_INTERRUPTS();
+
+ /* Data available in socket */
+ if (wc & WL_SOCKET_READABLE)
+ {
+ if (!PQconsumeInput(conn))
+ pgfdw_report_error(ERROR, NULL, conn, false, query);
+ }
+ }
+
+ res = PQgetResult(conn);
+ if (res == NULL)
+ break; /* query is complete */
+
+ PQclear(last_res);
+ last_res = res;
+ }
+
+ return last_res;
+}
+
+/*
* Report an error we got from the remote server.
*
* elevel: error level to use (typically ERROR, but might be less)
@@ -598,6 +671,32 @@ pgfdw_xact_callback(XactEvent event, void *arg)
case XACT_EVENT_ABORT:
/* Assume we might have lost track of prepared statements */
entry->have_error = true;
+
+ /*
+ * If a command has been submitted to the remote server by
+ * using an asynchronous execution function, the command
+ * might not have yet completed. Check to see if a command
+ * is still being processed by the remote server, and if so,
+ * request cancellation of the command; if not, abort
+ * gracefully.
+ */
+ if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
+ {
+ PGcancel *cancel;
+ char errbuf[256];
+
+ if ((cancel = PQgetCancel(entry->conn)))
+ {
+ if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ errbuf)));
+ PQfreeCancel(cancel);
+ }
+ break;
+ }
+
/* If we're aborting, abort all remote transactions too */
res = PQexec(entry->conn, "ABORT TRANSACTION");
/* Note: can't throw ERROR, it would be infinite loop */
diff --git a/contrib/postgres_fdw/postgres_fdw.c b/contrib/postgres_fdw/postgres_fdw.c
index 28093e5..2f49268 100644
--- a/contrib/postgres_fdw/postgres_fdw.c
+++ b/contrib/postgres_fdw/postgres_fdw.c
@@ -1421,7 +1421,7 @@ postgresReScanForeignScan(ForeignScanState *node)
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexec(fsstate->conn, sql);
+ res = pgfdw_exec_query(fsstate->conn, sql);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, fsstate->conn, true, sql);
PQclear(res);
@@ -1749,18 +1749,24 @@ postgresExecForeignInsert(EState *estate,
p_values = convert_prep_stmt_params(fmstate, NULL, slot);
/*
- * Execute the prepared statement, and check for success.
+ * Execute the prepared statement.
+ */
+ if (!PQsendQueryPrepared(fmstate->conn,
+ fmstate->p_name,
+ fmstate->p_nums,
+ p_values,
+ NULL,
+ NULL,
+ 0))
+ pgfdw_report_error(ERROR, NULL, fmstate->conn, false, fmstate->query);
+
+ /*
+ * Get the result, and check for success.
*
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexecPrepared(fmstate->conn,
- fmstate->p_name,
- fmstate->p_nums,
- p_values,
- NULL,
- NULL,
- 0);
+ res = pgfdw_get_result(fmstate->conn, fmstate->query);
if (PQresultStatus(res) !=
(fmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
@@ -1819,18 +1825,24 @@ postgresExecForeignUpdate(EState *estate,
slot);
/*
- * Execute the prepared statement, and check for success.
+ * Execute the prepared statement.
+ */
+ if (!PQsendQueryPrepared(fmstate->conn,
+ fmstate->p_name,
+ fmstate->p_nums,
+ p_values,
+ NULL,
+ NULL,
+ 0))
+ pgfdw_report_error(ERROR, NULL, fmstate->conn, false, fmstate->query);
+
+ /*
+ * Get the result, and check for success.
*
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexecPrepared(fmstate->conn,
- fmstate->p_name,
- fmstate->p_nums,
- p_values,
- NULL,
- NULL,
- 0);
+ res = pgfdw_get_result(fmstate->conn, fmstate->query);
if (PQresultStatus(res) !=
(fmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
@@ -1889,18 +1901,24 @@ postgresExecForeignDelete(EState *estate,
NULL);
/*
- * Execute the prepared statement, and check for success.
+ * Execute the prepared statement.
+ */
+ if (!PQsendQueryPrepared(fmstate->conn,
+ fmstate->p_name,
+ fmstate->p_nums,
+ p_values,
+ NULL,
+ NULL,
+ 0))
+ pgfdw_report_error(ERROR, NULL, fmstate->conn, false, fmstate->query);
+
+ /*
+ * Get the result, and check for success.
*
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexecPrepared(fmstate->conn,
- fmstate->p_name,
- fmstate->p_nums,
- p_values,
- NULL,
- NULL,
- 0);
+ res = pgfdw_get_result(fmstate->conn, fmstate->query);
if (PQresultStatus(res) !=
(fmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
@@ -1950,7 +1968,7 @@ postgresEndForeignModify(EState *estate,
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexec(fmstate->conn, sql);
+ res = pgfdw_exec_query(fmstate->conn, sql);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, fmstate->conn, true, sql);
PQclear(res);
@@ -2712,7 +2730,7 @@ get_remote_estimate(const char *sql, PGconn *conn,
/*
* Execute EXPLAIN remotely.
*/
- res = PQexec(conn, sql);
+ res = pgfdw_exec_query(conn, sql);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, sql);
@@ -2817,12 +2835,18 @@ create_cursor(ForeignScanState *node)
* parameter (see deparse.c), the "inference" is trivial and will produce
* the desired result. This allows us to avoid assuming that the remote
* server has the same OIDs we do for the parameters' types.
+ */
+ if (!PQsendQueryParams(conn, buf.data, numParams,
+ NULL, values, NULL, NULL, 0))
+ pgfdw_report_error(ERROR, NULL, conn, false, buf.data);
+
+ /*
+ * Get the result, and check for success.
*
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexecParams(conn, buf.data, numParams, NULL, values,
- NULL, NULL, 0);
+ res = pgfdw_get_result(conn, buf.data);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, conn, true, fsstate->query);
PQclear(res);
@@ -2868,7 +2892,7 @@ fetch_more_data(ForeignScanState *node)
snprintf(sql, sizeof(sql), "FETCH %d FROM c%u",
fsstate->fetch_size, fsstate->cursor_number);
- res = PQexec(conn, sql);
+ res = pgfdw_exec_query(conn, sql);
/* On error, report the original query, not the FETCH. */
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, fsstate->query);
@@ -2978,7 +3002,7 @@ close_cursor(PGconn *conn, unsigned int cursor_number)
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQexec(conn, sql);
+ res = pgfdw_exec_query(conn, sql);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, conn, true, sql);
PQclear(res);
@@ -3006,16 +3030,21 @@ prepare_foreign_modify(PgFdwModifyState *fmstate)
* with the remote server using different type OIDs than we do. All of
* the prepared statements we use in this module are simple enough that
* the remote server will make the right choices.
+ */
+ if (!PQsendPrepare(fmstate->conn,
+ p_name,
+ fmstate->query,
+ 0,
+ NULL))
+ pgfdw_report_error(ERROR, NULL, fmstate->conn, false, fmstate->query);
+
+ /*
+ * Get the result, and check for success.
*
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- res = PQprepare(fmstate->conn,
- p_name,
- fmstate->query,
- 0,
- NULL);
-
+ res = pgfdw_get_result(fmstate->conn, fmstate->query);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, fmstate->conn, true, fmstate->query);
PQclear(res);
@@ -3147,12 +3176,18 @@ execute_dml_stmt(ForeignScanState *node)
* parameter (see deparse.c), the "inference" is trivial and will produce
* the desired result. This allows us to avoid assuming that the remote
* server has the same OIDs we do for the parameters' types.
+ */
+ if (!PQsendQueryParams(dmstate->conn, dmstate->query, numParams,
+ NULL, values, NULL, NULL, 0))
+ pgfdw_report_error(ERROR, NULL, dmstate->conn, false, dmstate->query);
+
+ /*
+ * Get the result, and check for success.
*
* We don't use a PG_TRY block here, so be careful not to throw error
* without releasing the PGresult.
*/
- dmstate->result = PQexecParams(dmstate->conn, dmstate->query,
- numParams, NULL, values, NULL, NULL, 0);
+ dmstate->result = pgfdw_get_result(dmstate->conn, dmstate->query);
if (PQresultStatus(dmstate->result) !=
(dmstate->has_returning ? PGRES_TUPLES_OK : PGRES_COMMAND_OK))
pgfdw_report_error(ERROR, dmstate->result, dmstate->conn, true,
@@ -3355,7 +3390,7 @@ postgresAnalyzeForeignTable(Relation relation,
/* In what follows, do not risk leaking any PGresults. */
PG_TRY();
{
- res = PQexec(conn, sql.data);
+ res = pgfdw_exec_query(conn, sql.data);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, sql.data);
@@ -3449,7 +3484,7 @@ postgresAcquireSampleRowsFunc(Relation relation, int elevel,
/* In what follows, do not risk leaking any PGresults. */
PG_TRY();
{
- res = PQexec(conn, sql.data);
+ res = pgfdw_exec_query(conn, sql.data);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
pgfdw_report_error(ERROR, res, conn, false, sql.data);
PQclear(res);
@@ -3500,7 +3535,7 @@ postgresAcquireSampleRowsFunc(Relation relation, int elevel,
snprintf(fetch_sql, sizeof(fetch_sql), "FETCH %d FROM c%u",
fetch_size, cursor_number);
- res = PQexec(conn, fetch_sql);
+ res = pgfdw_exec_query(conn, fetch_sql);
/* On error, report the original query, not the FETCH. */
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, sql.data);
@@ -3675,7 +3710,7 @@ postgresImportForeignSchema(ImportForeignSchemaStmt *stmt, Oid serverOid)
appendStringInfoString(&buf, "SELECT 1 FROM pg_catalog.pg_namespace WHERE nspname = ");
deparseStringLiteral(&buf, stmt->remote_schema);
- res = PQexec(conn, buf.data);
+ res = pgfdw_exec_query(conn, buf.data);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, buf.data);
@@ -3774,7 +3809,7 @@ postgresImportForeignSchema(ImportForeignSchemaStmt *stmt, Oid serverOid)
appendStringInfoString(&buf, " ORDER BY c.relname, a.attnum");
/* Fetch the data */
- res = PQexec(conn, buf.data);
+ res = pgfdw_exec_query(conn, buf.data);
if (PQresultStatus(res) != PGRES_TUPLES_OK)
pgfdw_report_error(ERROR, res, conn, false, buf.data);
diff --git a/contrib/postgres_fdw/postgres_fdw.h b/contrib/postgres_fdw/postgres_fdw.h
index 3a11d99..574b07d 100644
--- a/contrib/postgres_fdw/postgres_fdw.h
+++ b/contrib/postgres_fdw/postgres_fdw.h
@@ -103,6 +103,8 @@ extern PGconn *GetConnection(UserMapping *user, bool will_prep_stmt);
extern void ReleaseConnection(PGconn *conn);
extern unsigned int GetCursorNumber(PGconn *conn);
extern unsigned int GetPrepStmtNumber(PGconn *conn);
+extern PGresult *pgfdw_get_result(PGconn *conn, const char *query);
+extern PGresult *pgfdw_exec_query(PGconn *conn, const char *query);
extern void pgfdw_report_error(int elevel, PGresult *res, PGconn *conn,
bool clear, const char *sql);
On Thu, Apr 21, 2016 at 5:22 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
Attached is an updated version of the patch, which modified Michael's
version of the patch, as I proposed in [1] (see "Other changes:"). I
modified comments for pgfdw_get_result/pgfdw_exec_query also, mainly because
words like "non-blocking mode" there seems confusing (note that we have
PQsetnonbloking).
OK, so that is what I sent except that the comments mentioning PG_TRY
are moved to their correct places. That's fine for me. Thanks for
gathering everything in a single patch and correcting it.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Apr 21, 2016 at 8:44 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:
On Thu, Apr 21, 2016 at 5:22 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:Attached is an updated version of the patch, which modified Michael's
version of the patch, as I proposed in [1] (see "Other changes:"). I
modified comments for pgfdw_get_result/pgfdw_exec_query also, mainly because
words like "non-blocking mode" there seems confusing (note that we have
PQsetnonbloking).OK, so that is what I sent except that the comments mentioning PG_TRY
are moved to their correct places. That's fine for me. Thanks for
gathering everything in a single patch and correcting it.
I have committed this patch. Thanks for working on this. Sorry for the delay.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Apr 21, 2016 at 11:53 PM, Robert Haas <robertmhaas@gmail.com> wrote:
On Thu, Apr 21, 2016 at 8:44 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:On Thu, Apr 21, 2016 at 5:22 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:Attached is an updated version of the patch, which modified Michael's
version of the patch, as I proposed in [1] (see "Other changes:"). I
modified comments for pgfdw_get_result/pgfdw_exec_query also, mainly because
words like "non-blocking mode" there seems confusing (note that we have
PQsetnonbloking).OK, so that is what I sent except that the comments mentioning PG_TRY
are moved to their correct places. That's fine for me. Thanks for
gathering everything in a single patch and correcting it.I have committed this patch. Thanks for working on this. Sorry for the delay.
Thanks for the push. open_item--;
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/04/22 9:22, Michael Paquier wrote:
On Thu, Apr 21, 2016 at 11:53 PM, Robert Haas <robertmhaas@gmail.com> wrote:
I have committed this patch. Thanks for working on this. Sorry for the delay.
Thanks for the push. open_item--;
Thank you, Robert and Michael.
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hi,
While re-reviewing the fix, I noticed that since PQcancel we added to
pgfdw_xact_callback to cancel a DML pushdown query isn't followed by a
ROLLBACK, the connection to the remote server will be discarded at the
end of the while loop in that function, which will cause a FATAL error
of "connection to client lost". Probably, that was proposed by me in
the first version of the patch, but I don't think that's a good idea.
Shouldn't we execute ROLLBACK after that PQcancel?
Another thing I noticed is, ISTM that we miss the case where DML
pushdown queries are performed in subtransactions. I think cancellation
logic would also need to be added to pgfdw_subxact_callback.
Comments are welcome!
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/04/26 21:45, Etsuro Fujita wrote:
While re-reviewing the fix, I noticed that since PQcancel we added to
pgfdw_xact_callback to cancel a DML pushdown query isn't followed by a
ROLLBACK, the connection to the remote server will be discarded at the
end of the while loop in that function, which will cause a FATAL error
of "connection to client lost". Probably, that was proposed by me in
the first version of the patch, but I don't think that's a good idea.
Shouldn't we execute ROLLBACK after that PQcancel?Another thing I noticed is, ISTM that we miss the case where DML
pushdown queries are performed in subtransactions. I think cancellation
logic would also need to be added to pgfdw_subxact_callback.
Attached is a patch for that.
Best regards,
Etsuro Fujita
Attachments:
pgfdw-xact-subxact-callbacks.patchtext/x-diff; name=pgfdw-xact-subxact-callbacks.patchDownload
*** a/contrib/postgres_fdw/connection.c
--- b/contrib/postgres_fdw/connection.c
***************
*** 677,684 **** pgfdw_xact_callback(XactEvent event, void *arg)
* using an asynchronous execution function, the command
* might not have yet completed. Check to see if a command
* is still being processed by the remote server, and if so,
! * request cancellation of the command; if not, abort
! * gracefully.
*/
if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
{
--- 677,683 ----
* using an asynchronous execution function, the command
* might not have yet completed. Check to see if a command
* is still being processed by the remote server, and if so,
! * request cancellation of the command.
*/
if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
{
***************
*** 694,700 **** pgfdw_xact_callback(XactEvent event, void *arg)
errbuf)));
PQfreeCancel(cancel);
}
- break;
}
/* If we're aborting, abort all remote transactions too */
--- 693,698 ----
***************
*** 798,803 **** pgfdw_subxact_callback(SubXactEvent event, SubTransactionId mySubid,
--- 796,825 ----
{
/* Assume we might have lost track of prepared statements */
entry->have_error = true;
+
+ /*
+ * If a command has been submitted to the remote server by using an
+ * asynchronous execution function, the command might not have yet
+ * completed. Check to see if a command is still being processed by
+ * the remote server, and if so, request cancellation of the
+ * command.
+ */
+ if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
+ {
+ PGcancel *cancel;
+ char errbuf[256];
+
+ if ((cancel = PQgetCancel(entry->conn)))
+ {
+ if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not send cancel request: %s",
+ errbuf)));
+ PQfreeCancel(cancel);
+ }
+ }
+
/* Rollback all remote subtransactions during abort */
snprintf(sql, sizeof(sql),
"ROLLBACK TO SAVEPOINT s%d; RELEASE SAVEPOINT s%d",
On Wed, Apr 27, 2016 at 12:16 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
On 2016/04/26 21:45, Etsuro Fujita wrote:
While re-reviewing the fix, I noticed that since PQcancel we added to
pgfdw_xact_callback to cancel a DML pushdown query isn't followed by a
ROLLBACK, the connection to the remote server will be discarded at the
end of the while loop in that function, which will cause a FATAL error
of "connection to client lost". Probably, that was proposed by me in
the first version of the patch, but I don't think that's a good idea.
Shouldn't we execute ROLLBACK after that PQcancel?Another thing I noticed is, ISTM that we miss the case where DML
pushdown queries are performed in subtransactions. I think cancellation
logic would also need to be added to pgfdw_subxact_callback.Attached is a patch for that.
I have spent some time looking at that...
And yeah, losing the connection because of that is a little bit
annoying if there are ways to make things clean, and as a START
TRANSACTION is always sent for such queries it seems really better to
issue a ROLLBACK in any case. Actually, by using PQcancel there is no
way to be sure if the cancel will be effective or not. So it could be
possible that the command is still able to complete correctly, or it
could be able to cancel correctly and it would return an ERROR
earlier. In any case, doing the ROLLBACK unconditionally seems adapted
to me because we had better clean up the remote state in both cases.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/04/28 13:45, Michael Paquier wrote:
On Wed, Apr 27, 2016 at 12:16 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:On 2016/04/26 21:45, Etsuro Fujita wrote:
While re-reviewing the fix, I noticed that since PQcancel we added to
pgfdw_xact_callback to cancel a DML pushdown query isn't followed by a
ROLLBACK, the connection to the remote server will be discarded at the
end of the while loop in that function, which will cause a FATAL error
of "connection to client lost". Probably, that was proposed by me in
the first version of the patch, but I don't think that's a good idea.
Shouldn't we execute ROLLBACK after that PQcancel?Another thing I noticed is, ISTM that we miss the case where DML
pushdown queries are performed in subtransactions. I think cancellation
logic would also need to be added to pgfdw_subxact_callback.
Attached is a patch for that.
I have spent some time looking at that...
And yeah, losing the connection because of that is a little bit
annoying if there are ways to make things clean, and as a START
TRANSACTION is always sent for such queries it seems really better to
issue a ROLLBACK in any case. Actually, by using PQcancel there is no
way to be sure if the cancel will be effective or not. So it could be
possible that the command is still able to complete correctly, or it
could be able to cancel correctly and it would return an ERROR
earlier. In any case, doing the ROLLBACK unconditionally seems adapted
to me because we had better clean up the remote state in both cases.
Thanks for the review!
I'll add this to the next CF. I think this should be addressed in
advance of the release of 9.6, though.
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, May 11, 2016 at 3:20 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:
Thanks for the review!
I'll add this to the next CF. I think this should be addressed in advance
of the release of 9.6, though.
I agree. Committed.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2016/05/17 0:25, Robert Haas wrote:
On Wed, May 11, 2016 at 3:20 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:Thanks for the review!
I'll add this to the next CF. I think this should be addressed in advance
of the release of 9.6, though.
I agree. Committed.
Thanks!
Best regards,
Etsuro Fujita
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Apr 21, 2016 at 10:53 AM, Robert Haas <robertmhaas@gmail.com> wrote:
On Thu, Apr 21, 2016 at 8:44 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:On Thu, Apr 21, 2016 at 5:22 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:Attached is an updated version of the patch, which modified Michael's
version of the patch, as I proposed in [1] (see "Other changes:"). I
modified comments for pgfdw_get_result/pgfdw_exec_query also, mainly because
words like "non-blocking mode" there seems confusing (note that we have
PQsetnonbloking).OK, so that is what I sent except that the comments mentioning PG_TRY
are moved to their correct places. That's fine for me. Thanks for
gathering everything in a single patch and correcting it.I have committed this patch. Thanks for working on this. Sorry for the delay.
This 9.6-era patch, as it turns out, has a problem, which is that we
now respond to an interrupt by sending a cancel request and a
NON-interruptible ABORT TRANSACTION command to the remote side. If
the reason that the user is trying to interrupt is that the network
connection has been cut, they interrupt the original query only to get
stuck in a non-interruptible wait for ABORT TRANSACTION. That is
non-optimal.
It is not exactly clear to me how to fix this. Could we get by with
just slamming the remote connection shut, instead of sending an
explicit ABORT TRANSACTION? The remote side ought to treat a
disconnect as equivalent to an ABORT anyway, I think, but maybe our
local state would get confused. (I have not checked.)
Thoughts?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
At Thu, 13 Apr 2017 13:04:12 -0400, Robert Haas <robertmhaas@gmail.com> wrote in <CA+TgmoaxnNmuONgP=bXJojrgbnMPTi6Ms8OSwZBC2YQ2ueUiSg@mail.gmail.com>
On Thu, Apr 21, 2016 at 10:53 AM, Robert Haas <robertmhaas@gmail.com> wrote:
On Thu, Apr 21, 2016 at 8:44 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:On Thu, Apr 21, 2016 at 5:22 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:Attached is an updated version of the patch, which modified Michael's
version of the patch, as I proposed in [1] (see "Other changes:"). I
modified comments for pgfdw_get_result/pgfdw_exec_query also, mainly because
words like "non-blocking mode" there seems confusing (note that we have
PQsetnonbloking).OK, so that is what I sent except that the comments mentioning PG_TRY
are moved to their correct places. That's fine for me. Thanks for
gathering everything in a single patch and correcting it.I have committed this patch. Thanks for working on this. Sorry for the delay.
This 9.6-era patch, as it turns out, has a problem, which is that we
now respond to an interrupt by sending a cancel request and a
NON-interruptible ABORT TRANSACTION command to the remote side. If
the reason that the user is trying to interrupt is that the network
connection has been cut, they interrupt the original query only to get
stuck in a non-interruptible wait for ABORT TRANSACTION. That is
non-optimal.
Agreed.
It is not exactly clear to me how to fix this. Could we get by with
just slamming the remote connection shut, instead of sending an
explicit ABORT TRANSACTION? The remote side ought to treat a
disconnect as equivalent to an ABORT anyway, I think, but maybe our
local state would get confused. (I have not checked.)Thoughts?
Perhaps we will get stuck at query cancellation before ABORT
TRANSACTION in the case. A connection will be shut down when
anything wrong (PQstatus(conn) != CONNECTION_OK and so) on
aborting local transactoin . So I don't think fdw gets confused
or sholdn't be confused by shutting down there.
The most significant issue I can see is that the same thing
happens in the case of graceful ABORT TRANSACTION. It could be a
performance issue.
We could set timeout here but maybe we can just slamming the
connection down instead of sending a query cancellation. It is
caused only by timeout or interrupts so I suppose it is not a
problem *with a few connections*.
Things are a bit diffent with hundreds of connections. The
penalty of reconnection would be very high in the case.
If we are not willing to pay such high penalty, maybe we are to
manage busy-idle time of each connection and trying graceful
abort if it is short enough, maybe having a shoft timeout.
Furthermore, if most or all of the hundreds of connections get
stuck, such timeout will accumulate up like a mountain...
regards,
--
Kyotaro Horiguchi
NTT Open Source Software Center
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Apr 17, 2017 at 1:53 PM, Kyotaro HORIGUCHI
<horiguchi.kyotaro@lab.ntt.co.jp> wrote:
At Thu, 13 Apr 2017 13:04:12 -0400, Robert Haas <robertmhaas@gmail.com> wrote in <CA+TgmoaxnNmuONgP=bXJojrgbnMPTi6Ms8OSwZBC2YQ2ueUiSg@mail.gmail.com>
On Thu, Apr 21, 2016 at 10:53 AM, Robert Haas <robertmhaas@gmail.com> wrote:
On Thu, Apr 21, 2016 at 8:44 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:On Thu, Apr 21, 2016 at 5:22 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:Attached is an updated version of the patch, which modified Michael's
version of the patch, as I proposed in [1] (see "Other changes:"). I
modified comments for pgfdw_get_result/pgfdw_exec_query also, mainly because
words like "non-blocking mode" there seems confusing (note that we have
PQsetnonbloking).OK, so that is what I sent except that the comments mentioning PG_TRY
are moved to their correct places. That's fine for me. Thanks for
gathering everything in a single patch and correcting it.I have committed this patch. Thanks for working on this. Sorry for the delay.
This 9.6-era patch, as it turns out, has a problem, which is that we
now respond to an interrupt by sending a cancel request and a
NON-interruptible ABORT TRANSACTION command to the remote side. If
the reason that the user is trying to interrupt is that the network
connection has been cut, they interrupt the original query only to get
stuck in a non-interruptible wait for ABORT TRANSACTION. That is
non-optimal.Agreed.
It is not exactly clear to me how to fix this. Could we get by with
just slamming the remote connection shut, instead of sending an
explicit ABORT TRANSACTION? The remote side ought to treat a
disconnect as equivalent to an ABORT anyway, I think, but maybe our
local state would get confused. (I have not checked.)Thoughts?
Perhaps we will get stuck at query cancellation before ABORT
TRANSACTION in the case. A connection will be shut down when
anything wrong (PQstatus(conn) != CONNECTION_OK and so) on
aborting local transactoin . So I don't think fdw gets confused
or sholdn't be confused by shutting down there.The most significant issue I can see is that the same thing
happens in the case of graceful ABORT TRANSACTION. It could be a
performance issue.We could set timeout here but maybe we can just slamming the
connection down instead of sending a query cancellation. It is
caused only by timeout or interrupts so I suppose it is not a
problem *with a few connections*.Things are a bit diffent with hundreds of connections. The
penalty of reconnection would be very high in the case.If we are not willing to pay such high penalty, maybe we are to
manage busy-idle time of each connection and trying graceful
abort if it is short enough, maybe having a shoft timeout.Furthermore, if most or all of the hundreds of connections get
stuck, such timeout will accumulate up like a mountain...
Even when the transaction is aborted because a user cancels a query,
we do want to preserve the connection, if possible, to avoid
reconnection. If the request to cancel the query itself fails, we
should certainly drop the connection. Here's the patch to do that.
--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company
Attachments:
0001-Drop-connection-when-query-can-not-be-cancelled.patchapplication/octet-stream; name=0001-Drop-connection-when-query-can-not-be-cancelled.patchDownload
From dabe8c99f635ca6fd7702812c7f8bf1ab4dcf014 Mon Sep 17 00:00:00 2001
From: Ashutosh Bapat <ashutosh.bapat@enterprisedb.com>
Date: Mon, 17 Apr 2017 17:28:36 +0530
Subject: [PATCH] Drop connection when query can not be cancelled.
While aborting a transaction, if an asynchronous query is running on the
connection, postgres_fdw tries to cancel the query. If the query can not be
cancelled, it means that the connection is broken and sending an ABORT command
would get stuck. This is bad when the transaction is being aborted because user
requested to cancel the query running in the given backend. In such a case,
close the connection instead of trying to "ABORT" the transaction.
Ashutosh Bapat.
---
contrib/postgres_fdw/connection.c | 84 ++++++++++++++++++++++++-------------
1 file changed, 56 insertions(+), 28 deletions(-)
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index c6e3d44..9db5e17 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -74,6 +74,7 @@ static void pgfdw_subxact_callback(SubXactEvent event,
SubTransactionId mySubid,
SubTransactionId parentSubid,
void *arg);
+static bool pgfdw_cancel_query(ConnCacheEntry *entry);
/*
@@ -669,22 +670,16 @@ pgfdw_xact_callback(XactEvent event, void *arg)
* might not have yet completed. Check to see if a
* command is still being processed by the remote server,
* and if so, request cancellation of the command.
+ *
+ * If the request can not be completed successfully, the
+ * connection is broken. In that case do not try to send an
+ * ABORT, since that may get blocked. This is bad, if we
+ * are aborting the transaction because the user wanted to
+ * cancel the query running in this backend.
*/
if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
- {
- PGcancel *cancel;
- char errbuf[256];
-
- if ((cancel = PQgetCancel(entry->conn)))
- {
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
- ereport(WARNING,
- (errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
- }
- }
+ if (!pgfdw_cancel_query(entry))
+ break;
/* If we're aborting, abort all remote transactions too */
res = PQexec(entry->conn, "ABORT TRANSACTION");
@@ -794,22 +789,16 @@ pgfdw_subxact_callback(SubXactEvent event, SubTransactionId mySubid,
* yet completed. Check to see if a command is still being
* processed by the remote server, and if so, request cancellation
* of the command.
+ *
+ * If the request can not be completed successfully, the connection
+ * is broken. In that case do not try to send an ABORT, since that
+ * may get blocked. This is bad, if we are aborting the transaction
+ * because the user wanted to cancel the query running in this
+ * backend.
*/
if (PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE)
- {
- PGcancel *cancel;
- char errbuf[256];
-
- if ((cancel = PQgetCancel(entry->conn)))
- {
- if (!PQcancel(cancel, errbuf, sizeof(errbuf)))
- ereport(WARNING,
- (errcode(ERRCODE_CONNECTION_FAILURE),
- errmsg("could not send cancel request: %s",
- errbuf)));
- PQfreeCancel(cancel);
- }
- }
+ if (!pgfdw_cancel_query(entry))
+ continue;
/* Rollback all remote subtransactions during abort */
snprintf(sql, sizeof(sql),
@@ -826,3 +815,42 @@ pgfdw_subxact_callback(SubXactEvent event, SubTransactionId mySubid,
entry->xact_depth--;
}
}
+
+/*
+ * Cancel a running query on the given connection.
+ *
+ * Returns true if query was cancelled successfully, false otherwise. If the
+ * cancellation request fails, it also clears the connection since it will be
+ * broken.
+ */
+static bool
+pgfdw_cancel_query(ConnCacheEntry *entry)
+{
+ PGcancel *cancel;
+ char errbuf[256];
+
+ Assert(PQtransactionStatus(entry->conn) == PQTRANS_ACTIVE);
+
+ if ((cancel = PQgetCancel(entry->conn)))
+ {
+ if (PQcancel(cancel, errbuf, sizeof(errbuf)))
+ {
+ PQfreeCancel(cancel);
+ return true;
+ }
+ }
+
+ ereport(WARNING,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("discarding connection %p because of failed cancel request: %s",
+ entry->conn, errbuf)));
+
+ /*
+ * Clear the broken connection. Other fields in ConnCacheEntry will
+ * be cleared when establishing the connection again in
+ * GetConnection().
+ */
+ PQfinish(entry->conn);
+ entry->conn = NULL;
+ return false;
+}
--
1.7.9.5
At Mon, 17 Apr 2017 17:50:58 +0530, Ashutosh Bapat <ashutosh.bapat@enterprisedb.com> wrote in <CAFjFpRdcWw4h0a-zrL-EiaekkPj8O0GR2M1FwZ1useSRfRm3-g@mail.gmail.com>
On Mon, Apr 17, 2017 at 1:53 PM, Kyotaro HORIGUCHI
<horiguchi.kyotaro@lab.ntt.co.jp> wrote:At Thu, 13 Apr 2017 13:04:12 -0400, Robert Haas <robertmhaas@gmail.com> wrote in <CA+TgmoaxnNmuONgP=bXJojrgbnMPTi6Ms8OSwZBC2YQ2ueUiSg@mail.gmail.com>
On Thu, Apr 21, 2016 at 10:53 AM, Robert Haas <robertmhaas@gmail.com> wrote:
On Thu, Apr 21, 2016 at 8:44 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:On Thu, Apr 21, 2016 at 5:22 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:Attached is an updated version of the patch, which modified Michael's
version of the patch, as I proposed in [1] (see "Other changes:"). I
modified comments for pgfdw_get_result/pgfdw_exec_query also, mainly because
words like "non-blocking mode" there seems confusing (note that we have
PQsetnonbloking).OK, so that is what I sent except that the comments mentioning PG_TRY
are moved to their correct places. That's fine for me. Thanks for
gathering everything in a single patch and correcting it.I have committed this patch. Thanks for working on this. Sorry for the delay.
This 9.6-era patch, as it turns out, has a problem, which is that we
now respond to an interrupt by sending a cancel request and a
NON-interruptible ABORT TRANSACTION command to the remote side. If
the reason that the user is trying to interrupt is that the network
connection has been cut, they interrupt the original query only to get
stuck in a non-interruptible wait for ABORT TRANSACTION. That is
non-optimal.Agreed.
It is not exactly clear to me how to fix this. Could we get by with
just slamming the remote connection shut, instead of sending an
explicit ABORT TRANSACTION? The remote side ought to treat a
disconnect as equivalent to an ABORT anyway, I think, but maybe our
local state would get confused. (I have not checked.)Thoughts?
Perhaps we will get stuck at query cancellation before ABORT
TRANSACTION in the case. A connection will be shut down when
anything wrong (PQstatus(conn) != CONNECTION_OK and so) on
aborting local transactoin . So I don't think fdw gets confused
or sholdn't be confused by shutting down there.The most significant issue I can see is that the same thing
happens in the case of graceful ABORT TRANSACTION. It could be a
performance issue.We could set timeout here but maybe we can just slamming the
connection down instead of sending a query cancellation. It is
caused only by timeout or interrupts so I suppose it is not a
problem *with a few connections*.Things are a bit diffent with hundreds of connections. The
penalty of reconnection would be very high in the case.If we are not willing to pay such high penalty, maybe we are to
manage busy-idle time of each connection and trying graceful
abort if it is short enough, maybe having a shoft timeout.Furthermore, if most or all of the hundreds of connections get
stuck, such timeout will accumulate up like a mountain...Even when the transaction is aborted because a user cancels a query,
we do want to preserve the connection, if possible, to avoid
Yes.
reconnection. If the request to cancel the query itself fails, we
should certainly drop the connection. Here's the patch to do that.
A problem I think on this would be that we still try to make
another connection for canceling and it would stall for several
minutes per connection on a packet stall, which should be done in
a second on ordinary circumstances. Perhaps we might want here is
async canceling with timeout.
regards,
--
Kyotaro Horiguchi
NTT Open Source Software Center
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Apr 18, 2017 at 8:12 AM, Kyotaro HORIGUCHI
<horiguchi.kyotaro@lab.ntt.co.jp> wrote:
At Mon, 17 Apr 2017 17:50:58 +0530, Ashutosh Bapat <ashutosh.bapat@enterprisedb.com> wrote in <CAFjFpRdcWw4h0a-zrL-EiaekkPj8O0GR2M1FwZ1useSRfRm3-g@mail.gmail.com>
On Mon, Apr 17, 2017 at 1:53 PM, Kyotaro HORIGUCHI
<horiguchi.kyotaro@lab.ntt.co.jp> wrote:At Thu, 13 Apr 2017 13:04:12 -0400, Robert Haas <robertmhaas@gmail.com> wrote in <CA+TgmoaxnNmuONgP=bXJojrgbnMPTi6Ms8OSwZBC2YQ2ueUiSg@mail.gmail.com>
On Thu, Apr 21, 2016 at 10:53 AM, Robert Haas <robertmhaas@gmail.com> wrote:
On Thu, Apr 21, 2016 at 8:44 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:On Thu, Apr 21, 2016 at 5:22 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:Attached is an updated version of the patch, which modified Michael's
version of the patch, as I proposed in [1] (see "Other changes:"). I
modified comments for pgfdw_get_result/pgfdw_exec_query also, mainly because
words like "non-blocking mode" there seems confusing (note that we have
PQsetnonbloking).OK, so that is what I sent except that the comments mentioning PG_TRY
are moved to their correct places. That's fine for me. Thanks for
gathering everything in a single patch and correcting it.I have committed this patch. Thanks for working on this. Sorry for the delay.
This 9.6-era patch, as it turns out, has a problem, which is that we
now respond to an interrupt by sending a cancel request and a
NON-interruptible ABORT TRANSACTION command to the remote side. If
the reason that the user is trying to interrupt is that the network
connection has been cut, they interrupt the original query only to get
stuck in a non-interruptible wait for ABORT TRANSACTION. That is
non-optimal.Agreed.
It is not exactly clear to me how to fix this. Could we get by with
just slamming the remote connection shut, instead of sending an
explicit ABORT TRANSACTION? The remote side ought to treat a
disconnect as equivalent to an ABORT anyway, I think, but maybe our
local state would get confused. (I have not checked.)Thoughts?
Perhaps we will get stuck at query cancellation before ABORT
TRANSACTION in the case. A connection will be shut down when
anything wrong (PQstatus(conn) != CONNECTION_OK and so) on
aborting local transactoin . So I don't think fdw gets confused
or sholdn't be confused by shutting down there.The most significant issue I can see is that the same thing
happens in the case of graceful ABORT TRANSACTION. It could be a
performance issue.We could set timeout here but maybe we can just slamming the
connection down instead of sending a query cancellation. It is
caused only by timeout or interrupts so I suppose it is not a
problem *with a few connections*.Things are a bit diffent with hundreds of connections. The
penalty of reconnection would be very high in the case.If we are not willing to pay such high penalty, maybe we are to
manage busy-idle time of each connection and trying graceful
abort if it is short enough, maybe having a shoft timeout.Furthermore, if most or all of the hundreds of connections get
stuck, such timeout will accumulate up like a mountain...Even when the transaction is aborted because a user cancels a query,
we do want to preserve the connection, if possible, to avoidYes.
reconnection. If the request to cancel the query itself fails, we
should certainly drop the connection. Here's the patch to do that.A problem I think on this would be that we still try to make
another connection for canceling and it would stall for several
minutes per connection on a packet stall, which should be done in
a second on ordinary circumstances. Perhaps we might want here is
async canceling with timeout.
I am not sure what do you mean "make another connection for
cancelling.". Do you mean to say that PQcancel would make another
connection?
The patch proposed simply improves the condition when PQcancel has
returned a failure. Right now we ignore that failure and try to ABORT
the transaction, which is most probably going to get stalled or fail
to be dispatched. With the patch such a connection will be reset.
--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
At Tue, 18 Apr 2017 09:12:07 +0530, Ashutosh Bapat <ashutosh.bapat@enterprisedb.com> wrote in <CAFjFpRcRjohUZaCNon1zkP7c6fo2qd5hnfHCT6t_39SGtS_oKQ@mail.gmail.com>
On Tue, Apr 18, 2017 at 8:12 AM, Kyotaro HORIGUCHI
<horiguchi.kyotaro@lab.ntt.co.jp> wrote:At Mon, 17 Apr 2017 17:50:58 +0530, Ashutosh Bapat <ashutosh.bapat@enterprisedb.com> wrote in <CAFjFpRdcWw4h0a-zrL-EiaekkPj8O0GR2M1FwZ1useSRfRm3-g@mail.gmail.com>
On Mon, Apr 17, 2017 at 1:53 PM, Kyotaro HORIGUCHI
<horiguchi.kyotaro@lab.ntt.co.jp> wrote:At Thu, 13 Apr 2017 13:04:12 -0400, Robert Haas <robertmhaas@gmail.com> wrote in <CA+TgmoaxnNmuONgP=bXJojrgbnMPTi6Ms8OSwZBC2YQ2ueUiSg@mail.gmail.com>
On Thu, Apr 21, 2016 at 10:53 AM, Robert Haas <robertmhaas@gmail.com> wrote:
On Thu, Apr 21, 2016 at 8:44 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:On Thu, Apr 21, 2016 at 5:22 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:Attached is an updated version of the patch, which modified Michael's
version of the patch, as I proposed in [1] (see "Other changes:"). I
modified comments for pgfdw_get_result/pgfdw_exec_query also, mainly because
words like "non-blocking mode" there seems confusing (note that we have
PQsetnonbloking).OK, so that is what I sent except that the comments mentioning PG_TRY
are moved to their correct places. That's fine for me. Thanks for
gathering everything in a single patch and correcting it.I have committed this patch. Thanks for working on this. Sorry for the delay.
This 9.6-era patch, as it turns out, has a problem, which is that we
now respond to an interrupt by sending a cancel request and a
NON-interruptible ABORT TRANSACTION command to the remote side. If
the reason that the user is trying to interrupt is that the network
connection has been cut, they interrupt the original query only to get
stuck in a non-interruptible wait for ABORT TRANSACTION. That is
non-optimal.Agreed.
It is not exactly clear to me how to fix this. Could we get by with
just slamming the remote connection shut, instead of sending an
explicit ABORT TRANSACTION? The remote side ought to treat a
disconnect as equivalent to an ABORT anyway, I think, but maybe our
local state would get confused. (I have not checked.)Thoughts?
Perhaps we will get stuck at query cancellation before ABORT
TRANSACTION in the case. A connection will be shut down when
anything wrong (PQstatus(conn) != CONNECTION_OK and so) on
aborting local transactoin . So I don't think fdw gets confused
or sholdn't be confused by shutting down there.The most significant issue I can see is that the same thing
happens in the case of graceful ABORT TRANSACTION. It could be a
performance issue.We could set timeout here but maybe we can just slamming the
connection down instead of sending a query cancellation. It is
caused only by timeout or interrupts so I suppose it is not a
problem *with a few connections*.Things are a bit diffent with hundreds of connections. The
penalty of reconnection would be very high in the case.If we are not willing to pay such high penalty, maybe we are to
manage busy-idle time of each connection and trying graceful
abort if it is short enough, maybe having a shoft timeout.Furthermore, if most or all of the hundreds of connections get
stuck, such timeout will accumulate up like a mountain...Even when the transaction is aborted because a user cancels a query,
we do want to preserve the connection, if possible, to avoidYes.
reconnection. If the request to cancel the query itself fails, we
should certainly drop the connection. Here's the patch to do that.A problem I think on this would be that we still try to make
another connection for canceling and it would stall for several
minutes per connection on a packet stall, which should be done in
a second on ordinary circumstances. Perhaps we might want here is
async canceling with timeout.I am not sure what do you mean "make another connection for
cancelling.". Do you mean to say that PQcancel would make another
connection?
Yes. It will take about 3 minutes on standard settings when no
ACK returned. I thought that this discussion is based on such
situation.
The patch proposed simply improves the condition when PQcancel has
returned a failure. Right now we ignore that failure and try to ABORT
the transaction, which is most probably going to get stalled or fail
to be dispatched. With the patch such a connection will be reset.
Ah, I understand that. It is surely an improvement since it
avoids useless ABORT TRANSACTION that is known to stall.
regards,
--
Kyotaro Horiguchi
NTT Open Source Software Center
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
The subject of this thread is not directly related to this discussion
and we have a new thread [1]/messages/by-id/CAF1DzPU8Kx+fMXEbFoP289xtm3bz3t+ZfxhmKavr98Bh-C0TqQ@mail.gmail.com for relevant discussion. So, let's
discuss this further on that thread.
[1]: /messages/by-id/CAF1DzPU8Kx+fMXEbFoP289xtm3bz3t+ZfxhmKavr98Bh-C0TqQ@mail.gmail.com
On Tue, Apr 18, 2017 at 9:30 AM, Kyotaro HORIGUCHI
<horiguchi.kyotaro@lab.ntt.co.jp> wrote:
At Tue, 18 Apr 2017 09:12:07 +0530, Ashutosh Bapat <ashutosh.bapat@enterprisedb.com> wrote in <CAFjFpRcRjohUZaCNon1zkP7c6fo2qd5hnfHCT6t_39SGtS_oKQ@mail.gmail.com>
On Tue, Apr 18, 2017 at 8:12 AM, Kyotaro HORIGUCHI
<horiguchi.kyotaro@lab.ntt.co.jp> wrote:At Mon, 17 Apr 2017 17:50:58 +0530, Ashutosh Bapat <ashutosh.bapat@enterprisedb.com> wrote in <CAFjFpRdcWw4h0a-zrL-EiaekkPj8O0GR2M1FwZ1useSRfRm3-g@mail.gmail.com>
On Mon, Apr 17, 2017 at 1:53 PM, Kyotaro HORIGUCHI
<horiguchi.kyotaro@lab.ntt.co.jp> wrote:At Thu, 13 Apr 2017 13:04:12 -0400, Robert Haas <robertmhaas@gmail.com> wrote in <CA+TgmoaxnNmuONgP=bXJojrgbnMPTi6Ms8OSwZBC2YQ2ueUiSg@mail.gmail.com>
On Thu, Apr 21, 2016 at 10:53 AM, Robert Haas <robertmhaas@gmail.com> wrote:
On Thu, Apr 21, 2016 at 8:44 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:On Thu, Apr 21, 2016 at 5:22 PM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:Attached is an updated version of the patch, which modified Michael's
version of the patch, as I proposed in [1] (see "Other changes:"). I
modified comments for pgfdw_get_result/pgfdw_exec_query also, mainly because
words like "non-blocking mode" there seems confusing (note that we have
PQsetnonbloking).OK, so that is what I sent except that the comments mentioning PG_TRY
are moved to their correct places. That's fine for me. Thanks for
gathering everything in a single patch and correcting it.I have committed this patch. Thanks for working on this. Sorry for the delay.
This 9.6-era patch, as it turns out, has a problem, which is that we
now respond to an interrupt by sending a cancel request and a
NON-interruptible ABORT TRANSACTION command to the remote side. If
the reason that the user is trying to interrupt is that the network
connection has been cut, they interrupt the original query only to get
stuck in a non-interruptible wait for ABORT TRANSACTION. That is
non-optimal.Agreed.
It is not exactly clear to me how to fix this. Could we get by with
just slamming the remote connection shut, instead of sending an
explicit ABORT TRANSACTION? The remote side ought to treat a
disconnect as equivalent to an ABORT anyway, I think, but maybe our
local state would get confused. (I have not checked.)Thoughts?
Perhaps we will get stuck at query cancellation before ABORT
TRANSACTION in the case. A connection will be shut down when
anything wrong (PQstatus(conn) != CONNECTION_OK and so) on
aborting local transactoin . So I don't think fdw gets confused
or sholdn't be confused by shutting down there.The most significant issue I can see is that the same thing
happens in the case of graceful ABORT TRANSACTION. It could be a
performance issue.We could set timeout here but maybe we can just slamming the
connection down instead of sending a query cancellation. It is
caused only by timeout or interrupts so I suppose it is not a
problem *with a few connections*.Things are a bit diffent with hundreds of connections. The
penalty of reconnection would be very high in the case.If we are not willing to pay such high penalty, maybe we are to
manage busy-idle time of each connection and trying graceful
abort if it is short enough, maybe having a shoft timeout.Furthermore, if most or all of the hundreds of connections get
stuck, such timeout will accumulate up like a mountain...Even when the transaction is aborted because a user cancels a query,
we do want to preserve the connection, if possible, to avoidYes.
reconnection. If the request to cancel the query itself fails, we
should certainly drop the connection. Here's the patch to do that.A problem I think on this would be that we still try to make
another connection for canceling and it would stall for several
minutes per connection on a packet stall, which should be done in
a second on ordinary circumstances. Perhaps we might want here is
async canceling with timeout.I am not sure what do you mean "make another connection for
cancelling.". Do you mean to say that PQcancel would make another
connection?Yes. It will take about 3 minutes on standard settings when no
ACK returned. I thought that this discussion is based on such
situation.The patch proposed simply improves the condition when PQcancel has
returned a failure. Right now we ignore that failure and try to ABORT
the transaction, which is most probably going to get stalled or fail
to be dispatched. With the patch such a connection will be reset.Ah, I understand that. It is surely an improvement since it
avoids useless ABORT TRANSACTION that is known to stall.regards,
--
Kyotaro Horiguchi
NTT Open Source Software Center
--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers