foreign table creation and NOT VALID check constraints
In f27a6b15e656 (9.6 & later), we decided to "Mark CHECK constraints
declared NOT VALID valid if created with table." In retrospect,
constraints on foreign tables should have been excluded from consideration
in that commit, because the thinking behind the aforementioned commit
(that the constraint is trivially validated because the newly created
table contains no data) does not equally apply to the foreign tables case.
Should we do something about that?
Thanks,
Amit
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 1 August 2017 at 07:16, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> wrote:
In f27a6b15e656 (9.6 & later), we decided to "Mark CHECK constraints
declared NOT VALID valid if created with table." In retrospect,
constraints on foreign tables should have been excluded from consideration
in that commit, because the thinking behind the aforementioned commit
(that the constraint is trivially validated because the newly created
table contains no data) does not equally apply to the foreign tables case.Should we do something about that?
In what way does it not apply? Do you have a failure case?
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2017/08/01 15:22, Simon Riggs wrote:
On 1 August 2017 at 07:16, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> wrote:
In f27a6b15e656 (9.6 & later), we decided to "Mark CHECK constraints
declared NOT VALID valid if created with table." In retrospect,
constraints on foreign tables should have been excluded from consideration
in that commit, because the thinking behind the aforementioned commit
(that the constraint is trivially validated because the newly created
table contains no data) does not equally apply to the foreign tables case.Should we do something about that?
In what way does it not apply? Do you have a failure case?
Sorry for not mentioning the details.
I was thinking that a foreign table starts containing the data of the
remote object it points to the moment it's created (unlike local tables
which contain no data to begin with). If a user is not sure whether a
particular constraint being created in the same command holds for the
remote data, they may mark it as NOT VALID and hope that the system treats
the constraint as such until such a time that they mark it valid by
running ALTER TABLE VALIDATE CONSTRAINT. Since the planner is the only
consumer of pg_constraint.convalidated, that means the user expects the
planner to ignore such a constraint. Since f27a6b15e656, users are no
longer able to expect so.
For example:
create extension postgres_fdw;
create server loopback
foreign data wrapper postgres_fdw
options (dbname 'postgres');
create table foo (a) as select 1;
create foreign table ffoo (
a int,
constraint check_a check (a = 0) not valid
) server loopback options (table_name 'foo');
set constraint_exclusion to on;
-- constraint check_a will exclude the table
select * from ffoo where a = 1;
a
---
(0 rows)
explain select * from ffoo where a = 1;
QUERY PLAN
------------------------------------------
Result (cost=0.00..0.00 rows=0 width=4)
One-Time Filter: false
(2 rows)
The above "issue" doesn't occur if the constraint is created
after-the-fact, whereby it's possible to specify NOT VALID and have the
system actually store it in the catalog as such.
alter foreign table ffoo drop constraint check_a;
alter foreign table ffoo add constraint check_a check (a = 0) not valid;
select * from ffoo where a = 1;
a
---
1
(1 row)
explain select * from ffoo where a = 1;
QUERY PLAN
-------------------------------------------------------------
Foreign Scan on ffoo (cost=100.00..146.86 rows=15 width=4)
Maybe, this is not such a big matter though, because the core system
doesn't actually enforce any constraints of the foreign tables locally
anyway (which is a documented fact), but my concern is the possibility of
some considering this a POLA violation. Lack of complaints so far perhaps
means nobody did.
Thanks,
Amit
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 1 August 2017 at 08:37, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> wrote:
On 2017/08/01 15:22, Simon Riggs wrote:
On 1 August 2017 at 07:16, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> wrote:
In f27a6b15e656 (9.6 & later), we decided to "Mark CHECK constraints
declared NOT VALID valid if created with table." In retrospect,
constraints on foreign tables should have been excluded from consideration
in that commit, because the thinking behind the aforementioned commit
(that the constraint is trivially validated because the newly created
table contains no data) does not equally apply to the foreign tables case.Should we do something about that?
In what way does it not apply? Do you have a failure case?
Sorry for not mentioning the details.
I was thinking that a foreign table starts containing the data of the
remote object it points to the moment it's created (unlike local tables
which contain no data to begin with). If a user is not sure whether a
particular constraint being created in the same command holds for the
remote data, they may mark it as NOT VALID and hope that the system treats
the constraint as such until such a time that they mark it valid by
running ALTER TABLE VALIDATE CONSTRAINT. Since the planner is the only
consumer of pg_constraint.convalidated, that means the user expects the
planner to ignore such a constraint. Since f27a6b15e656, users are no
longer able to expect so.
For Foreign Tables, it sounds like an issue. Don't think it exists for
normal tables.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2017/08/01 17:54, Simon Riggs wrote:
On 1 August 2017 at 08:37, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> wrote:
On 2017/08/01 15:22, Simon Riggs wrote:
On 1 August 2017 at 07:16, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> wrote:
In f27a6b15e656 (9.6 & later), we decided to "Mark CHECK constraints
declared NOT VALID valid if created with table." In retrospect,
constraints on foreign tables should have been excluded from consideration
in that commit, because the thinking behind the aforementioned commit
(that the constraint is trivially validated because the newly created
table contains no data) does not equally apply to the foreign tables case.Should we do something about that?
In what way does it not apply? Do you have a failure case?
Sorry for not mentioning the details.
I was thinking that a foreign table starts containing the data of the
remote object it points to the moment it's created (unlike local tables
which contain no data to begin with). If a user is not sure whether a
particular constraint being created in the same command holds for the
remote data, they may mark it as NOT VALID and hope that the system treats
the constraint as such until such a time that they mark it valid by
running ALTER TABLE VALIDATE CONSTRAINT. Since the planner is the only
consumer of pg_constraint.convalidated, that means the user expects the
planner to ignore such a constraint. Since f27a6b15e656, users are no
longer able to expect so.For Foreign Tables, it sounds like an issue. Don't think it exists for
normal tables.
Yes. I was saying in my first email that we should not disregard user's
request to mark a constraint NOT VALID if the table in question is a
*foreign table*.
So, it's OK that commit f27a6b15e656 changed things to ignore NOT VALID if
it's in the following command:
create table foo (a int, constraint check_a check (a > 0) not valid);
But, not OK in the following command:
create foreign table ffoo (
a int,
constraint check_a check (a > 0) not valid
) server loopback options (table_name 'foo');
Thanks,
Amit
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Aug 1, 2017 at 2:41 PM, Amit Langote
<Langote_Amit_f8@lab.ntt.co.jp> wrote:
On 2017/08/01 17:54, Simon Riggs wrote:
On 1 August 2017 at 08:37, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> wrote:
On 2017/08/01 15:22, Simon Riggs wrote:
On 1 August 2017 at 07:16, Amit Langote <Langote_Amit_f8@lab.ntt.co.jp> wrote:
In f27a6b15e656 (9.6 & later), we decided to "Mark CHECK constraints
declared NOT VALID valid if created with table." In retrospect,
constraints on foreign tables should have been excluded from consideration
in that commit, because the thinking behind the aforementioned commit
(that the constraint is trivially validated because the newly created
table contains no data) does not equally apply to the foreign tables case.Should we do something about that?
In what way does it not apply? Do you have a failure case?
Sorry for not mentioning the details.
I was thinking that a foreign table starts containing the data of the
remote object it points to the moment it's created (unlike local tables
which contain no data to begin with). If a user is not sure whether a
particular constraint being created in the same command holds for the
remote data, they may mark it as NOT VALID and hope that the system treats
the constraint as such until such a time that they mark it valid by
running ALTER TABLE VALIDATE CONSTRAINT. Since the planner is the only
consumer of pg_constraint.convalidated, that means the user expects the
planner to ignore such a constraint. Since f27a6b15e656, users are no
longer able to expect so.For Foreign Tables, it sounds like an issue. Don't think it exists for
normal tables.Yes. I was saying in my first email that we should not disregard user's
request to mark a constraint NOT VALID if the table in question is a
*foreign table*.So, it's OK that commit f27a6b15e656 changed things to ignore NOT VALID if
it's in the following command:create table foo (a int, constraint check_a check (a > 0) not valid);
But, not OK in the following command:
create foreign table ffoo (
a int,
constraint check_a check (a > 0) not valid
) server loopback options (table_name 'foo');
If the user has specified "not valid" for a constraint on the foreign
table, there is high chance that s/he is aware of the fact that the
remote table that the foreign table points to has some rows which will
violet the constraint. So, +1.
--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Aug 2, 2017 at 3:46 AM, Ashutosh Bapat
<ashutosh.bapat@enterprisedb.com> wrote:
If the user has specified "not valid" for a constraint on the foreign
table, there is high chance that s/he is aware of the fact that the
remote table that the foreign table points to has some rows which will
violet the constraint. So, +1.
+1 from me, too.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2017/08/02 20:40, Robert Haas wrote:
On Wed, Aug 2, 2017 at 3:46 AM, Ashutosh Bapat
<ashutosh.bapat@enterprisedb.com> wrote:If the user has specified "not valid" for a constraint on the foreign
table, there is high chance that s/he is aware of the fact that the
remote table that the foreign table points to has some rows which will
violet the constraint. So, +1.+1 from me, too.
Alright, thanks.
Attached is a patch. I think this could be considered a bug-fix,
backpatchable to 9.6 which introduced this behavior change [1]http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=f27a6b15e656.
Thoughts?
Thanks,
Amit
[1]: http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=f27a6b15e656
http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=f27a6b15e656
Attachments:
0001-Honor-NOT-VALID-specification-in-CREATE-FOREIGN-TABL.patchtext/plain; charset=UTF-8; name=0001-Honor-NOT-VALID-specification-in-CREATE-FOREIGN-TABL.patchDownload
From a0967f1a71a65e7802f504002a6dc3dfd1f4a6bb Mon Sep 17 00:00:00 2001
From: amit <amitlangote09@gmail.com>
Date: Thu, 3 Aug 2017 10:31:21 +0900
Subject: [PATCH] Honor NOT VALID specification in CREATE FOREIGN TABLE
It should be possible for a user to mark the constraints on foreign
tables as NOT VALID even when creating the table, because the remote
data they point to may contain constraint-violating rows, which the
database has no way to detect and show an error message for.
---
doc/src/sgml/ref/create_foreign_table.sgml | 7 +++++++
doc/src/sgml/ref/create_table.sgml | 7 +++++++
src/backend/parser/parse_utilcmd.c | 3 ++-
3 files changed, 16 insertions(+), 1 deletion(-)
diff --git a/doc/src/sgml/ref/create_foreign_table.sgml b/doc/src/sgml/ref/create_foreign_table.sgml
index 065c982082..72bf37b8b9 100644
--- a/doc/src/sgml/ref/create_foreign_table.sgml
+++ b/doc/src/sgml/ref/create_foreign_table.sgml
@@ -222,6 +222,13 @@ CHECK ( <replaceable class="PARAMETER">expression</replaceable> ) [ NO INHERIT ]
A constraint marked with <literal>NO INHERIT</> will not propagate to
child tables.
</para>
+
+ <para>
+ It is possible to mark the constraint as <literal>NOT VALID</> if it is
+ specified as a table constraint. If marked as such, the database will
+ not assume that the constraint holds for all the rows in the table until
+ it is validated by using the <literal>VALIDATE CONSTRAINT</> command.
+ See <xref linkend="SQL-ALTERFOREIGNTABLE">.
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index e9c2c49533..72de64a03e 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -652,6 +652,13 @@ FROM ( { <replaceable class="PARAMETER">numeric_literal</replaceable> | <replace
(<productname>PostgreSQL</> versions before 9.5 did not honor any
particular firing order for <literal>CHECK</literal> constraints.)
</para>
+
+ <para>
+ Note that even if the constraint is marked as <literal>NOT VALID</>,
+ it is considered validated as the table that is just created cannot
+ contain any data. In other words, specifying <literal>NOT VALID</> in
+ this case has no effect.
+ </para>
</listitem>
</varlistentry>
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 9f37f1b920..e54322b460 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -165,6 +165,7 @@ transformCreateStmt(CreateStmt *stmt, const char *queryString)
Oid existing_relid;
ParseCallbackState pcbstate;
bool like_found = false;
+ bool is_foreign_table = IsA(stmt, CreateForeignTableStmt);
/*
* We must not scribble on the passed-in CreateStmt, so copy it. (This is
@@ -330,7 +331,7 @@ transformCreateStmt(CreateStmt *stmt, const char *queryString)
/*
* Postprocess check constraints.
*/
- transformCheckConstraints(&cxt, true);
+ transformCheckConstraints(&cxt, !is_foreign_table ? true : false);
/*
* Output results.
--
2.11.0
On Wed, Aug 2, 2017 at 9:41 PM, Amit Langote
<Langote_Amit_f8@lab.ntt.co.jp> wrote:
On 2017/08/02 20:40, Robert Haas wrote:
On Wed, Aug 2, 2017 at 3:46 AM, Ashutosh Bapat
<ashutosh.bapat@enterprisedb.com> wrote:If the user has specified "not valid" for a constraint on the foreign
table, there is high chance that s/he is aware of the fact that the
remote table that the foreign table points to has some rows which will
violet the constraint. So, +1.+1 from me, too.
Alright, thanks.
Attached is a patch. I think this could be considered a bug-fix,
backpatchable to 9.6 which introduced this behavior change [1].
I could go either way on that. It's not inconceivable somebody could
be unhappy about seeing this behavior change in a minor release.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Robert Haas <robertmhaas@gmail.com> writes:
On Wed, Aug 2, 2017 at 9:41 PM, Amit Langote
<Langote_Amit_f8@lab.ntt.co.jp> wrote:Attached is a patch. I think this could be considered a bug-fix,
backpatchable to 9.6 which introduced this behavior change [1].
I could go either way on that. It's not inconceivable somebody could
be unhappy about seeing this behavior change in a minor release.
FWIW, I vote with the camp that this is a clear bug and needs to be
fixed. 9.6 broke a behavior that could be relied on before that.
We do not normally hesitate to fix regressions in minor releases.
(That's not a vote for the patch as submitted; I haven't reviewed it.
But we need to fix this.)
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Aug 3, 2017 at 12:35 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Robert Haas <robertmhaas@gmail.com> writes:
On Wed, Aug 2, 2017 at 9:41 PM, Amit Langote
<Langote_Amit_f8@lab.ntt.co.jp> wrote:Attached is a patch. I think this could be considered a bug-fix,
backpatchable to 9.6 which introduced this behavior change [1].I could go either way on that. It's not inconceivable somebody could
be unhappy about seeing this behavior change in a minor release.FWIW, I vote with the camp that this is a clear bug and needs to be
fixed. 9.6 broke a behavior that could be relied on before that.
We do not normally hesitate to fix regressions in minor releases.(That's not a vote for the patch as submitted; I haven't reviewed it.
But we need to fix this.)
OK. I'm going to commit and back-patch the substantive fix with a
comment change, but I'm not going to include Amit's documentation
changes for now because I'm not sure they are going to be sufficiently
clear. There's not a lot of context for them where he put them.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2017/08/04 2:13, Robert Haas wrote:
On Thu, Aug 3, 2017 at 12:35 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Robert Haas <robertmhaas@gmail.com> writes:
On Wed, Aug 2, 2017 at 9:41 PM, Amit Langote
<Langote_Amit_f8@lab.ntt.co.jp> wrote:Attached is a patch. I think this could be considered a bug-fix,
backpatchable to 9.6 which introduced this behavior change [1].I could go either way on that. It's not inconceivable somebody could
be unhappy about seeing this behavior change in a minor release.FWIW, I vote with the camp that this is a clear bug and needs to be
fixed. 9.6 broke a behavior that could be relied on before that.
We do not normally hesitate to fix regressions in minor releases.(That's not a vote for the patch as submitted; I haven't reviewed it.
But we need to fix this.)OK. I'm going to commit and back-patch the substantive fix with a
comment change, but I'm not going to include Amit's documentation
changes for now because I'm not sure they are going to be sufficiently
clear. There's not a lot of context for them where he put them.
Thanks for committing the code changes.
About the documentation changes, it seems that the only places where any
description of NOT VALID appears is ALTER TABLE, ALTER FOREIGN TABLE, and
ALTER DOMAIN references pages. Even if the CREATE (FOREIGN) TABLE syntax
allows it, NOT VALID does not appear in the syntax synopsis, so it seems
kind of a hidden feature. Maybe, we should fix that first (if at all).
Thanks,
Amit
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Thu, Aug 3, 2017 at 9:29 PM, Amit Langote
<Langote_Amit_f8@lab.ntt.co.jp> wrote:
Thanks for committing the code changes.
About the documentation changes, it seems that the only places where any
description of NOT VALID appears is ALTER TABLE, ALTER FOREIGN TABLE, and
ALTER DOMAIN references pages. Even if the CREATE (FOREIGN) TABLE syntax
allows it, NOT VALID does not appear in the syntax synopsis, so it seems
kind of a hidden feature. Maybe, we should fix that first (if at all).
Yeah. If somebody wants to do a general survey of how NOT VALID is
documented and improve that, cool.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers