Mishandling of WCO constraints in direct foreign table modification

Started by Etsuro Fujitaover 8 years ago7 messages
#1Etsuro Fujita
fujita.etsuro@lab.ntt.co.jp
1 attachment(s)

Here is an example for $subject using postgres_fdw:

postgres=# create foreign table foreign_tbl (a int, b int) server
loopback options (table_name 'base_tbl');
CREATE FOREIGN TABLE
postgres=# create view rw_view as select * from foreign_tbl where a < b
with check option;
CREATE VIEW
postgres=# insert into rw_view values (0, 10);
INSERT 0 1
postgres=# explain verbose update rw_view set a = 20 where b = 10;
QUERY PLAN
--------------------------------------------------------------------------------------
Update on public.foreign_tbl (cost=100.00..146.21 rows=4 width=14)
-> Foreign Update on public.foreign_tbl (cost=100.00..146.21
rows=4 width=14)
Remote SQL: UPDATE public.base_tbl SET a = 20 WHERE ((a < b))
AND ((b = 10))
(3 rows)

postgres=# update rw_view set a = 20 where b = 10;
UPDATE 1

This is wrong! This should fail. The reason for that is; direct modify
is overlooking checking WITH CHECK OPTION constraints from parent views.
I think we could do direct modify, even if there are any WITH CHECK
OPTIONs, in some way or other, but I think that is a feature. So, I'd
like to propose to fix this by just giving up direct modify if there are
any WITH CHECK OPTIONs. Attached is a patch for that. I'll add it to
the next commitfest.

Best regards,
Etsuro Fujita

Attachments:

fix-direct-modify.patchtext/plain; charset=UTF-8; name=fix-direct-modify.patchDownload
*** a/contrib/postgres_fdw/expected/postgres_fdw.out
--- b/contrib/postgres_fdw/expected/postgres_fdw.out
***************
*** 5856,5861 **** INSERT INTO ft1(c1, c2) VALUES(1111, 2);
--- 5856,5921 ----
  UPDATE ft1 SET c2 = c2 + 1 WHERE c1 = 1;
  ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2negative;
  -- ===================================================================
+ -- test WITH CHECK OPTION constraints
+ -- ===================================================================
+ CREATE TABLE base_tbl (a int, b int);
+ CREATE FOREIGN TABLE foreign_tbl (a int, b int)
+   SERVER loopback OPTIONS(table_name 'base_tbl');
+ CREATE VIEW rw_view AS SELECT * FROM foreign_tbl
+   WHERE a < b WITH CHECK OPTION;
+ \d+ rw_view
+                            View "public.rw_view"
+  Column |  Type   | Collation | Nullable | Default | Storage | Description 
+ --------+---------+-----------+----------+---------+---------+-------------
+  a      | integer |           |          |         | plain   | 
+  b      | integer |           |          |         | plain   | 
+ View definition:
+  SELECT foreign_tbl.a,
+     foreign_tbl.b
+    FROM foreign_tbl
+   WHERE foreign_tbl.a < foreign_tbl.b;
+ Options: check_option=cascaded
+ 
+ INSERT INTO rw_view VALUES (0, 10); -- ok
+ INSERT INTO rw_view VALUES (10, 0); -- should fail
+ ERROR:  new row violates check option for view "rw_view"
+ DETAIL:  Failing row contains (10, 0).
+ EXPLAIN (VERBOSE, COSTS OFF)
+ UPDATE rw_view SET b = 20 WHERE a = 0; -- not pushed down
+                                             QUERY PLAN                                            
+ --------------------------------------------------------------------------------------------------
+  Update on public.foreign_tbl
+    Remote SQL: UPDATE public.base_tbl SET b = $2 WHERE ctid = $1
+    ->  Foreign Scan on public.foreign_tbl
+          Output: foreign_tbl.a, 20, foreign_tbl.ctid
+          Remote SQL: SELECT a, ctid FROM public.base_tbl WHERE ((a < b)) AND ((a = 0)) FOR UPDATE
+ (5 rows)
+ 
+ UPDATE rw_view SET b = 20 WHERE a = 0; -- ok
+ EXPLAIN (VERBOSE, COSTS OFF)
+ UPDATE rw_view SET b = -20 WHERE a = 0; -- not pushed down
+                                             QUERY PLAN                                            
+ --------------------------------------------------------------------------------------------------
+  Update on public.foreign_tbl
+    Remote SQL: UPDATE public.base_tbl SET b = $2 WHERE ctid = $1
+    ->  Foreign Scan on public.foreign_tbl
+          Output: foreign_tbl.a, '-20'::integer, foreign_tbl.ctid
+          Remote SQL: SELECT a, ctid FROM public.base_tbl WHERE ((a < b)) AND ((a = 0)) FOR UPDATE
+ (5 rows)
+ 
+ UPDATE rw_view SET b = -20 WHERE a = 0; -- should fail
+ ERROR:  new row violates check option for view "rw_view"
+ DETAIL:  Failing row contains (0, -20).
+ SELECT * FROM foreign_tbl;
+  a | b  
+ ---+----
+  0 | 20
+ (1 row)
+ 
+ DROP FOREIGN TABLE foreign_tbl CASCADE;
+ NOTICE:  drop cascades to view rw_view
+ DROP TABLE base_tbl;
+ -- ===================================================================
  -- test serial columns (ie, sequence-based defaults)
  -- ===================================================================
  create table loc1 (f1 serial, f2 text);
*** a/contrib/postgres_fdw/sql/postgres_fdw.sql
--- b/contrib/postgres_fdw/sql/postgres_fdw.sql
***************
*** 1158,1163 **** UPDATE ft1 SET c2 = c2 + 1 WHERE c1 = 1;
--- 1158,1187 ----
  ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2negative;
  
  -- ===================================================================
+ -- test WITH CHECK OPTION constraints
+ -- ===================================================================
+ 
+ CREATE TABLE base_tbl (a int, b int);
+ CREATE FOREIGN TABLE foreign_tbl (a int, b int)
+   SERVER loopback OPTIONS(table_name 'base_tbl');
+ CREATE VIEW rw_view AS SELECT * FROM foreign_tbl
+   WHERE a < b WITH CHECK OPTION;
+ \d+ rw_view
+ 
+ INSERT INTO rw_view VALUES (0, 10); -- ok
+ INSERT INTO rw_view VALUES (10, 0); -- should fail
+ EXPLAIN (VERBOSE, COSTS OFF)
+ UPDATE rw_view SET b = 20 WHERE a = 0; -- not pushed down
+ UPDATE rw_view SET b = 20 WHERE a = 0; -- ok
+ EXPLAIN (VERBOSE, COSTS OFF)
+ UPDATE rw_view SET b = -20 WHERE a = 0; -- not pushed down
+ UPDATE rw_view SET b = -20 WHERE a = 0; -- should fail
+ SELECT * FROM foreign_tbl;
+ 
+ DROP FOREIGN TABLE foreign_tbl CASCADE;
+ DROP TABLE base_tbl;
+ 
+ -- ===================================================================
  -- test serial columns (ie, sequence-based defaults)
  -- ===================================================================
  create table loc1 (f1 serial, f2 text);
*** a/src/backend/optimizer/plan/createplan.c
--- b/src/backend/optimizer/plan/createplan.c
***************
*** 6499,6506 **** make_modifytable(PlannerInfo *root,
  		}
  
  		/*
! 		 * If the target foreign table has any row-level triggers, we can't
! 		 * modify the foreign table directly.
  		 */
  		direct_modify = false;
  		if (fdwroutine != NULL &&
--- 6499,6508 ----
  		}
  
  		/*
! 		 * Try to modify the foreign table directly, if (1) the FDW provides
! 		 * callback functions needed for that, (2) there are no row-level
! 		 * triggers on the foreign table, and (3) there are no WITH CHECK
! 		 * OPTIONs from parent views.
  		 */
  		direct_modify = false;
  		if (fdwroutine != NULL &&
***************
*** 6508,6513 **** make_modifytable(PlannerInfo *root,
--- 6510,6516 ----
  			fdwroutine->BeginDirectModify != NULL &&
  			fdwroutine->IterateDirectModify != NULL &&
  			fdwroutine->EndDirectModify != NULL &&
+ 			withCheckOptionLists == NIL &&
  			!has_row_triggers(root, rti, operation))
  			direct_modify = fdwroutine->PlanDirectModify(root, node, rti, i);
  		if (direct_modify)
#2Robert Haas
robertmhaas@gmail.com
In reply to: Etsuro Fujita (#1)
Re: Mishandling of WCO constraints in direct foreign table modification

On Thu, Jul 20, 2017 at 7:40 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:

Here is an example for $subject using postgres_fdw:

postgres=# create foreign table foreign_tbl (a int, b int) server loopback
options (table_name 'base_tbl');
CREATE FOREIGN TABLE
postgres=# create view rw_view as select * from foreign_tbl where a < b with
check option;
CREATE VIEW
postgres=# insert into rw_view values (0, 10);
INSERT 0 1
postgres=# explain verbose update rw_view set a = 20 where b = 10;
QUERY PLAN
--------------------------------------------------------------------------------------
Update on public.foreign_tbl (cost=100.00..146.21 rows=4 width=14)
-> Foreign Update on public.foreign_tbl (cost=100.00..146.21 rows=4
width=14)
Remote SQL: UPDATE public.base_tbl SET a = 20 WHERE ((a < b)) AND
((b = 10))
(3 rows)

postgres=# update rw_view set a = 20 where b = 10;
UPDATE 1

This is wrong! This should fail. The reason for that is; direct modify is
overlooking checking WITH CHECK OPTION constraints from parent views. I
think we could do direct modify, even if there are any WITH CHECK OPTIONs,
in some way or other, but I think that is a feature. So, I'd like to
propose to fix this by just giving up direct modify if there are any WITH
CHECK OPTIONs. Attached is a patch for that. I'll add it to the next
commitfest.

I think that's reasonable. This should be committed and back-patched
to 9.6, right?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#3Etsuro Fujita
fujita.etsuro@lab.ntt.co.jp
In reply to: Robert Haas (#2)
1 attachment(s)
Re: Mishandling of WCO constraints in direct foreign table modification

On 2017/07/21 3:24, Robert Haas wrote:

I think that's reasonable. This should be committed and back-patched
to 9.6, right?

Yeah, because direct modify was introduced in 9.6.

Attached is the second version which updated docs in postgres-fdw.sgml
as well.

Best regards,
Etsuro Fujita

Attachments:

fix-direct-modify-v2.patchtext/plain; charset=UTF-8; name=fix-direct-modify-v2.patchDownload
*** a/contrib/postgres_fdw/expected/postgres_fdw.out
--- b/contrib/postgres_fdw/expected/postgres_fdw.out
***************
*** 5856,5861 **** INSERT INTO ft1(c1, c2) VALUES(1111, 2);
--- 5856,5921 ----
  UPDATE ft1 SET c2 = c2 + 1 WHERE c1 = 1;
  ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2negative;
  -- ===================================================================
+ -- test WITH CHECK OPTION constraints
+ -- ===================================================================
+ CREATE TABLE base_tbl (a int, b int);
+ CREATE FOREIGN TABLE foreign_tbl (a int, b int)
+   SERVER loopback OPTIONS(table_name 'base_tbl');
+ CREATE VIEW rw_view AS SELECT * FROM foreign_tbl
+   WHERE a < b WITH CHECK OPTION;
+ \d+ rw_view
+                            View "public.rw_view"
+  Column |  Type   | Collation | Nullable | Default | Storage | Description 
+ --------+---------+-----------+----------+---------+---------+-------------
+  a      | integer |           |          |         | plain   | 
+  b      | integer |           |          |         | plain   | 
+ View definition:
+  SELECT foreign_tbl.a,
+     foreign_tbl.b
+    FROM foreign_tbl
+   WHERE foreign_tbl.a < foreign_tbl.b;
+ Options: check_option=cascaded
+ 
+ INSERT INTO rw_view VALUES (0, 10); -- ok
+ INSERT INTO rw_view VALUES (10, 0); -- should fail
+ ERROR:  new row violates check option for view "rw_view"
+ DETAIL:  Failing row contains (10, 0).
+ EXPLAIN (VERBOSE, COSTS OFF)
+ UPDATE rw_view SET b = 20 WHERE a = 0; -- not pushed down
+                                             QUERY PLAN                                            
+ --------------------------------------------------------------------------------------------------
+  Update on public.foreign_tbl
+    Remote SQL: UPDATE public.base_tbl SET b = $2 WHERE ctid = $1
+    ->  Foreign Scan on public.foreign_tbl
+          Output: foreign_tbl.a, 20, foreign_tbl.ctid
+          Remote SQL: SELECT a, ctid FROM public.base_tbl WHERE ((a < b)) AND ((a = 0)) FOR UPDATE
+ (5 rows)
+ 
+ UPDATE rw_view SET b = 20 WHERE a = 0; -- ok
+ EXPLAIN (VERBOSE, COSTS OFF)
+ UPDATE rw_view SET b = -20 WHERE a = 0; -- not pushed down
+                                             QUERY PLAN                                            
+ --------------------------------------------------------------------------------------------------
+  Update on public.foreign_tbl
+    Remote SQL: UPDATE public.base_tbl SET b = $2 WHERE ctid = $1
+    ->  Foreign Scan on public.foreign_tbl
+          Output: foreign_tbl.a, '-20'::integer, foreign_tbl.ctid
+          Remote SQL: SELECT a, ctid FROM public.base_tbl WHERE ((a < b)) AND ((a = 0)) FOR UPDATE
+ (5 rows)
+ 
+ UPDATE rw_view SET b = -20 WHERE a = 0; -- should fail
+ ERROR:  new row violates check option for view "rw_view"
+ DETAIL:  Failing row contains (0, -20).
+ SELECT * FROM foreign_tbl;
+  a | b  
+ ---+----
+  0 | 20
+ (1 row)
+ 
+ DROP FOREIGN TABLE foreign_tbl CASCADE;
+ NOTICE:  drop cascades to view rw_view
+ DROP TABLE base_tbl;
+ -- ===================================================================
  -- test serial columns (ie, sequence-based defaults)
  -- ===================================================================
  create table loc1 (f1 serial, f2 text);
*** a/contrib/postgres_fdw/sql/postgres_fdw.sql
--- b/contrib/postgres_fdw/sql/postgres_fdw.sql
***************
*** 1158,1163 **** UPDATE ft1 SET c2 = c2 + 1 WHERE c1 = 1;
--- 1158,1187 ----
  ALTER FOREIGN TABLE ft1 DROP CONSTRAINT ft1_c2negative;
  
  -- ===================================================================
+ -- test WITH CHECK OPTION constraints
+ -- ===================================================================
+ 
+ CREATE TABLE base_tbl (a int, b int);
+ CREATE FOREIGN TABLE foreign_tbl (a int, b int)
+   SERVER loopback OPTIONS(table_name 'base_tbl');
+ CREATE VIEW rw_view AS SELECT * FROM foreign_tbl
+   WHERE a < b WITH CHECK OPTION;
+ \d+ rw_view
+ 
+ INSERT INTO rw_view VALUES (0, 10); -- ok
+ INSERT INTO rw_view VALUES (10, 0); -- should fail
+ EXPLAIN (VERBOSE, COSTS OFF)
+ UPDATE rw_view SET b = 20 WHERE a = 0; -- not pushed down
+ UPDATE rw_view SET b = 20 WHERE a = 0; -- ok
+ EXPLAIN (VERBOSE, COSTS OFF)
+ UPDATE rw_view SET b = -20 WHERE a = 0; -- not pushed down
+ UPDATE rw_view SET b = -20 WHERE a = 0; -- should fail
+ SELECT * FROM foreign_tbl;
+ 
+ DROP FOREIGN TABLE foreign_tbl CASCADE;
+ DROP TABLE base_tbl;
+ 
+ -- ===================================================================
  -- test serial columns (ie, sequence-based defaults)
  -- ===================================================================
  create table loc1 (f1 serial, f2 text);
*** a/doc/src/sgml/postgres-fdw.sgml
--- b/doc/src/sgml/postgres-fdw.sgml
***************
*** 498,505 ****
     <filename>postgres_fdw</> attempts to optimize the query execution by
     sending the whole query to the remote server if there are no query
     <literal>WHERE</> clauses that cannot be sent to the remote server,
!    no local joins for the query, and no row-level local <literal>BEFORE</> or
!    <literal>AFTER</> triggers on the target table.  In <command>UPDATE</>,
     expressions to assign to target columns must use only built-in data types,
     <literal>IMMUTABLE</> operators, or <literal>IMMUTABLE</> functions,
     to reduce the risk of misexecution of the query.
--- 498,507 ----
     <filename>postgres_fdw</> attempts to optimize the query execution by
     sending the whole query to the remote server if there are no query
     <literal>WHERE</> clauses that cannot be sent to the remote server,
!    no local joins for the query, no row-level local <literal>BEFORE</> or
!    <literal>AFTER</> triggers on the target table, and no
!    <literal>CHECK OPTION</> constraints from parent views.
!    In <command>UPDATE</>,
     expressions to assign to target columns must use only built-in data types,
     <literal>IMMUTABLE</> operators, or <literal>IMMUTABLE</> functions,
     to reduce the risk of misexecution of the query.
*** a/src/backend/optimizer/plan/createplan.c
--- b/src/backend/optimizer/plan/createplan.c
***************
*** 6499,6506 **** make_modifytable(PlannerInfo *root,
  		}
  
  		/*
! 		 * If the target foreign table has any row-level triggers, we can't
! 		 * modify the foreign table directly.
  		 */
  		direct_modify = false;
  		if (fdwroutine != NULL &&
--- 6499,6508 ----
  		}
  
  		/*
! 		 * Try to modify the foreign table directly, if (1) the FDW provides
! 		 * callback functions needed for that, (2) there are no row-level
! 		 * triggers on the foreign table, and (3) there are no WITH CHECK
! 		 * OPTIONs from parent views.
  		 */
  		direct_modify = false;
  		if (fdwroutine != NULL &&
***************
*** 6508,6513 **** make_modifytable(PlannerInfo *root,
--- 6510,6516 ----
  			fdwroutine->BeginDirectModify != NULL &&
  			fdwroutine->IterateDirectModify != NULL &&
  			fdwroutine->EndDirectModify != NULL &&
+ 			withCheckOptionLists == NIL &&
  			!has_row_triggers(root, rti, operation))
  			direct_modify = fdwroutine->PlanDirectModify(root, node, rti, i);
  		if (direct_modify)
#4Kyotaro HORIGUCHI
horiguchi.kyotaro@lab.ntt.co.jp
In reply to: Etsuro Fujita (#3)
Re: Mishandling of WCO constraints in direct foreign table modification

At Fri, 21 Jul 2017 12:00:03 +0900, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp> wrote in <15aa9936-9bd8-c9e3-7ca1-3948610734b4@lab.ntt.co.jp>

On 2017/07/21 3:24, Robert Haas wrote:

I think that's reasonable. This should be committed and back-patched
to 9.6, right?

Yeah, because direct modify was introduced in 9.6.

Attached is the second version which updated docs in postgres-fdw.sgml
as well.

! no local joins for the query, no row-level local <literal>BEFORE</> or
! <literal>AFTER</> triggers on the target table, and no
! <literal>CHECK OPTION</> constraints from parent views.
! In <command>UPDATE</>,

Might be a silly question, is CHECK OPTION a "constraint"?

regards,

--
Kyotaro Horiguchi
NTT Open Source Software Center

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#5Etsuro Fujita
fujita.etsuro@lab.ntt.co.jp
In reply to: Kyotaro HORIGUCHI (#4)
Re: Mishandling of WCO constraints in direct foreign table modification

On 2017/07/21 17:18, Kyotaro HORIGUCHI wrote:

At Fri, 21 Jul 2017 12:00:03 +0900, Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp> wrote in <15aa9936-9bd8-c9e3-7ca1-3948610734b4@lab.ntt.co.jp>

Attached is the second version which updated docs in postgres-fdw.sgml
as well.

! no local joins for the query, no row-level local <literal>BEFORE</> or
! <literal>AFTER</> triggers on the target table, and no
! <literal>CHECK OPTION</> constraints from parent views.
! In <command>UPDATE</>,

Might be a silly question, is CHECK OPTION a "constraint"?

I mean constraints derived from WITH CHECK OPTIONs specified for parent
views. We use the words "WITH CHECK OPTION constraints" in comments in
nodeModifyTable.c, so the expression "CHECK OPTION constrains" doesn't
sound not that bad to me. (I used "CHECK OPTION", not "WITH CHECK
OPTION", because we use "CHECK OPTION" a lot more in the documentation
than "WITH CHECK OPTION".)

Best regards,
Etsuro Fujita

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#6Robert Haas
robertmhaas@gmail.com
In reply to: Etsuro Fujita (#5)
Re: Mishandling of WCO constraints in direct foreign table modification

On Fri, Jul 21, 2017 at 6:21 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:

I mean constraints derived from WITH CHECK OPTIONs specified for parent
views. We use the words "WITH CHECK OPTION constraints" in comments in
nodeModifyTable.c, so the expression "CHECK OPTION constrains" doesn't sound
not that bad to me. (I used "CHECK OPTION", not "WITH CHECK OPTION",
because we use "CHECK OPTION" a lot more in the documentation than "WITH
CHECK OPTION".)

Yeah, it seems OK to me, too; if the consensus is otherwise, we also
have the option to change it later. Committed and back-patched as you
had it, but I removed a spurious comma.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#7Etsuro Fujita
fujita.etsuro@lab.ntt.co.jp
In reply to: Robert Haas (#6)
Re: Mishandling of WCO constraints in direct foreign table modification

On 2017/07/25 5:35, Robert Haas wrote:

On Fri, Jul 21, 2017 at 6:21 AM, Etsuro Fujita
<fujita.etsuro@lab.ntt.co.jp> wrote:

I mean constraints derived from WITH CHECK OPTIONs specified for parent
views. We use the words "WITH CHECK OPTION constraints" in comments in
nodeModifyTable.c, so the expression "CHECK OPTION constrains" doesn't sound
not that bad to me. (I used "CHECK OPTION", not "WITH CHECK OPTION",
because we use "CHECK OPTION" a lot more in the documentation than "WITH
CHECK OPTION".)

Yeah, it seems OK to me, too; if the consensus is otherwise, we also
have the option to change it later.

Agreed.

Committed and back-patched as you
had it, but I removed a spurious comma.

Thanks for that, Robert! Thanks for reviewing, Horiguchi-san!

Best regards,
Etsuro Fujita

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers