CREATE TABLE NOT VALID for check and foreign key

Started by jian heabout 1 year ago9 messages
#1jian he
jian.universality@gmail.com

hi.
I found for foreign keys, check constraints,
you specify it as NOT VALID, it will not be marked as NOT VALID in the
CREATE TABLE statement.

CREATE TABLE s6(id bigint , CONSTRAINT con1 check(id > 1) not valid);
src2=# \d s6
Table "public.s6"
Column | Type | Collation | Nullable | Default
--------+--------+-----------+----------+---------
id | bigint | | |
Check constraints:
"con1" CHECK (id > 1)

create table pk (a int, primary key(a));
create table fk (a int, b int, foreign key(a) references pk( a) not valid);
Table "public.fk"
Column | Type | Collation | Nullable | Default
--------+---------+-----------+----------+---------
a | integer | | |
b | integer | | |
Foreign-key constraints:
"fk_a_fkey" FOREIGN KEY (a) REFERENCES pk(a)

reading transformCheckConstraints, transformFKConstraints comments
appearingly this is intentional?

If so, do we need to document the keywords "NOT VALID"
in create_table.sgml synopsis section?

#2Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: jian he (#1)
Re: CREATE TABLE NOT VALID for check and foreign key

Hello,

On 2024-Dec-05, jian he wrote:

I found for foreign keys, check constraints,
you specify it as NOT VALID, it will not be marked as NOT VALID in the
CREATE TABLE statement.

Uhmm, okay.

reading transformCheckConstraints, transformFKConstraints comments
appearingly this is intentional?

If so, do we need to document the keywords "NOT VALID"
in create_table.sgml synopsis section?

So, the whole point of ALTER TABLE adding constraints marked NOT VALID
is to let the AccessExclusiveLock on the table be held for a very short
time, without requiring a table scan; you follow that with ALTER TABLE
VALIDATE to remove the marking, which takes a weaker lock. This is
great for production-time constraint additions on large tables. But for
CREATE TABLE there's no such argument: it's pointless to mark a
constraint as NOT VALID, because nobody else could be looking at the
table anyway.

Maybe it would have been wise to forbid NOT VALID when used with CREATE
TABLE. But we didn't. Should we do that now? Maybe we can just
document that you can specify it but it doesn't do anything.

Thanks,

--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"We're here to devour each other alive" (Hobbes)

#3Kirill Reshke
reshkekirill@gmail.com
In reply to: Alvaro Herrera (#2)
Re: CREATE TABLE NOT VALID for check and foreign key

On Thu, 5 Dec 2024 at 14:36, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:

Maybe it would have been wise to forbid NOT VALID when used with CREATE
TABLE. But we didn't. Should we do that now? Maybe we can just
document that you can specify it but it doesn't do anything.

+1 on that

--
Best regards,
Kirill Reshke

#4jian he
jian.universality@gmail.com
In reply to: Kirill Reshke (#3)
1 attachment(s)
Re: CREATE TABLE NOT VALID for check and foreign key

turns out this has already been discussed at [1]/messages/by-id/d2b7419f-4a71-cf86-cc99-bfd0f359a1ea@lab.ntt.co.jp
NOT VALID only applies to table_constraint.
Amit Langote actually wrote the doc at[1]/messages/by-id/d2b7419f-4a71-cf86-cc99-bfd0f359a1ea@lab.ntt.co.jp, so I copied some text from there.

in create_table.sgml, i placed right below
<term><literal>INITIALLY IMMEDIATE</literal></term>
<term><literal>INITIALLY DEFERRED</literal></term>
so it looks like this:

<varlistentry id="sql-createtable-parms-notvalid">
<term><literal>NOT VALID</literal></term>
<listitem>
<para>
This applies only to <literal>CHECK</literal> and foreign key constraints.
Note even if the constraint is marked as <literal>NOT VALID</literal>,
it is considered as validated since the newly created table will not
contain any data. In other words, specifying <literal>NOT
VALID</literal> has no effect.
</para>
</listitem>
</varlistentry>

[1]: /messages/by-id/d2b7419f-4a71-cf86-cc99-bfd0f359a1ea@lab.ntt.co.jp

Attachments:

not_valid_create_table.difftext/x-patch; charset=US-ASCII; name=not_valid_create_table.diffDownload
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 70fa929caa..59357bca41 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -84,7 +84,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
   FOREIGN KEY ( <replaceable class="parameter">column_name</replaceable> [, ... ] [, PERIOD <replaceable class="parameter">column_name</replaceable> ] ) REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> [, ... ] [, PERIOD <replaceable class="parameter">refcolumn</replaceable> ] ) ]
     [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ] [ ON DELETE <replaceable
 class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
-[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
+[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ] [NOT VALID]
 
 <phrase>and <replaceable class="parameter">like_option</replaceable> is:</phrase>
 
@@ -1377,6 +1377,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
     </listitem>
    </varlistentry>
 
+   <varlistentry id="sql-createtable-parms-notvalid">
+    <term><literal>NOT VALID</literal></term>
+    <listitem>
+     <para>
+      This applies only to <literal>CHECK</literal> and foreign key constraints.
+      Note even if the constraint is marked as <literal>NOT VALID</literal>,
+      it is considered as validated since the newly created table will not
+      contain any data.  In other words, specifying <literal>NOT VALID</literal> has no effect.
+     </para>
+    </listitem>
+   </varlistentry>
+
    <varlistentry id="sql-createtable-method">
     <term><literal>USING <replaceable class="parameter">method</replaceable></literal></term>
     <listitem>
#5Yasuo Honda
yasuo.honda@gmail.com
In reply to: Alvaro Herrera (#2)
Re: CREATE TABLE NOT VALID for check and foreign key

Hi,

On Mon, Dec 23, 2024 at 10:10 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:

Maybe it would have been wise to forbid NOT VALID when used with CREATE
TABLE. But we didn't. Should we do that now? Maybe we can just
document that you can specify it but it doesn't do anything.

I'd like PostgreSQL to raise errors and/or warnings for the NOT VALID
check constraint for CREATE TABLE.
Ruby on Rails supports creating check constraints with the NOT VALID
option and I was not aware that it is just ignored until
https://github.com/rails/rails/issues/53732 issue is reported.

Rails has implemented a kind of workaround by not dumping the NOT
VALID option, but it does not help for the first execution.
https://github.com/rails/rails/pull/53735

Thanks,
--
Yasuo Honda

#6Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: Yasuo Honda (#5)
Re: CREATE TABLE NOT VALID for check and foreign key

Hello,

On 2025-Jan-07, Yasuo Honda wrote:

I'd like PostgreSQL to raise errors and/or warnings for the NOT VALID
check constraint for CREATE TABLE.
Ruby on Rails supports creating check constraints with the NOT VALID
option and I was not aware that it is just ignored until
https://github.com/rails/rails/issues/53732 issue is reported.

Thanks. I left a comment there. I think raising a WARNING might work.

Rails has implemented a kind of workaround by not dumping the NOT
VALID option, but it does not help for the first execution.
https://github.com/rails/rails/pull/53735

Yeah, that's probably not ideal.

--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"You're _really_ hosed if the person doing the hiring doesn't understand
relational systems: you end up with a whole raft of programmers, none of
whom has had a Date with the clue stick." (Andrew Sullivan)
/messages/by-id/20050809113420.GD2768@phlogiston.dyndns.org

#7Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: Alvaro Herrera (#6)
1 attachment(s)
Re: CREATE TABLE NOT VALID for check and foreign key

On 2025-Jan-08, Alvaro Herrera wrote:

On 2025-Jan-07, Yasuo Honda wrote:

I'd like PostgreSQL to raise errors and/or warnings for the NOT VALID
check constraint for CREATE TABLE.
Ruby on Rails supports creating check constraints with the NOT VALID
option and I was not aware that it is just ignored until
https://github.com/rails/rails/issues/53732 issue is reported.

Thanks. I left a comment there. I think raising a WARNING might work.

I think it'd be something like the rough POC here. I didn't add
warnings to all the CreateConstraintEntry() because some of them aren't
reachable via a path that allows NOT VALID. However, I may have missed
some cases.

I was really surprised to find that we have no test coverage for the
case where CREATE TABLE specifies a NOT VALID constraint.

--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/

Attachments:

0001-Throw-warning-if-NOT-VALID-is-given-during-CREATE-TA.patchtext/x-diff; charset=utf-8Download
From 08dd8b9a65223dfb3662a509b9da54fcb8adb520 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=81lvaro=20Herrera?= <alvherre@alvh.no-ip.org>
Date: Tue, 14 Jan 2025 18:43:47 +0100
Subject: [PATCH] Throw warning if NOT VALID is given during CREATE TABLE

Discussion: https://postgr.es/m/CACJufxEQcHNhN6M18JY1mQcgQq9Gn9ofMeop47SdFDE5B8wbug@mail.gmail.com
---
 src/backend/catalog/heap.c                | 8 ++++++++
 src/backend/commands/tablecmds.c          | 7 +++++++
 src/backend/parser/gram.y                 | 2 ++
 src/include/nodes/parsenodes.h            | 1 +
 src/test/regress/expected/alter_table.out | 1 +
 src/test/regress/expected/foreign_key.out | 3 ++-
 src/test/regress/sql/foreign_key.sql      | 2 +-
 7 files changed, 22 insertions(+), 2 deletions(-)

diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 57ef466acce..8a39dc1c2da 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2510,6 +2510,12 @@ AddRelationNewConstraints(Relation rel,
 				checknames = lappend(checknames, ccname);
 			}
 
+			/* XXX improve error report */
+			if (cdef->was_not_valid && cdef->initially_valid && cdef->is_enforced)
+				ereport(WARNING,
+						errcode(ERRCODE_WARNING),
+						errmsg("NOT VALID flag for constraint \"%s\" ignored", ccname));
+
 			/*
 			 * OK, store it.
 			 */
@@ -2967,6 +2973,8 @@ AddRelationNotNullConstraints(Relation rel, List *constraints,
 										   nnnames);
 		nnnames = lappend(nnnames, conname);
 
+		/* XXX if NOT VALID is given during CREATE TABLE, throw warning */
+
 		StoreRelNotNull(rel, conname,
 						attnum, true, true,
 						inhcount, constr->is_no_inherit);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index f070ffc960e..8e498344e7a 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -10446,6 +10446,13 @@ addFkConstraint(addFkConstraintSides fkside,
 		connoinherit = rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE;
 	}
 
+	/* XXX improve error report */
+	if (fkconstraint->was_not_valid && fkconstraint->initially_valid)
+		ereport(WARNING,
+				errcode(ERRCODE_WARNING),
+				errmsg("NOT VALID flag for constraint \"%s\" ignored",
+					   fkconstraint->conname));
+
 	/*
 	 * Record the FK constraint in pg_constraint.
 	 */
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 6079de70e09..c008a445bd4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -4172,6 +4172,7 @@ ConstraintElem:
 								   NULL, NULL, &n->is_enforced, &n->skip_validation,
 								   &n->is_no_inherit, yyscanner);
 					n->initially_valid = !n->skip_validation;
+					n->was_not_valid = n->skip_validation;
 					$$ = (Node *) n;
 				}
 			| NOT NULL_P ColId ConstraintAttributeSpec
@@ -4306,6 +4307,7 @@ ConstraintElem:
 								   NULL, &n->skip_validation, NULL,
 								   yyscanner);
 					n->initially_valid = !n->skip_validation;
+					n->was_not_valid = n->skip_validation;
 					$$ = (Node *) n;
 				}
 		;
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index b191eaaecab..2f103238d8f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -2762,6 +2762,7 @@ typedef struct Constraint
 	bool		is_enforced;	/* enforced constraint? */
 	bool		skip_validation;	/* skip validation of existing rows? */
 	bool		initially_valid;	/* mark the new constraint as valid? */
+	bool		was_not_valid;	/* did the user request NOT VALID? */
 	bool		is_no_inherit;	/* is constraint non-inheritable? */
 	Node	   *raw_expr;		/* CHECK or DEFAULT expression, as
 								 * untransformed parse tree */
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index dd8cdec2905..8b5918b9eb6 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -574,6 +574,7 @@ DROP TABLE attmp2;
 -- exclusion until validated
 set constraint_exclusion TO 'partition';
 create table nv_parent (d date, check (false) no inherit not valid);
+WARNING:  NOT VALID flag for constraint "nv_parent_check" ignored
 -- not valid constraint added at creation time should automatically become valid
 \d nv_parent
             Table "public.nv_parent"
diff --git a/src/test/regress/expected/foreign_key.out b/src/test/regress/expected/foreign_key.out
index 3f459f70ac1..3b528738302 100644
--- a/src/test/regress/expected/foreign_key.out
+++ b/src/test/regress/expected/foreign_key.out
@@ -61,7 +61,8 @@ DROP TABLE PKTABLE;
 --
 CREATE TABLE PKTABLE ( ptest1 int, ptest2 int, ptest3 text, PRIMARY KEY(ptest1, ptest2) );
 CREATE TABLE FKTABLE ( ftest1 int, ftest2 int, ftest3 int, CONSTRAINT constrname FOREIGN KEY(ftest1, ftest2)
-                       REFERENCES PKTABLE MATCH FULL ON DELETE SET NULL ON UPDATE SET NULL);
+                       REFERENCES PKTABLE MATCH FULL ON DELETE SET NULL ON UPDATE SET NULL NOT VALID);
+WARNING:  NOT VALID flag for constraint "constrname" ignored
 -- Test comments
 COMMENT ON CONSTRAINT constrname_wrong ON FKTABLE IS 'fk constraint comment';
 ERROR:  constraint "constrname_wrong" for table "fktable" does not exist
diff --git a/src/test/regress/sql/foreign_key.sql b/src/test/regress/sql/foreign_key.sql
index 2e710e419c2..46a043e18e1 100644
--- a/src/test/regress/sql/foreign_key.sql
+++ b/src/test/regress/sql/foreign_key.sql
@@ -48,7 +48,7 @@ DROP TABLE PKTABLE;
 --
 CREATE TABLE PKTABLE ( ptest1 int, ptest2 int, ptest3 text, PRIMARY KEY(ptest1, ptest2) );
 CREATE TABLE FKTABLE ( ftest1 int, ftest2 int, ftest3 int, CONSTRAINT constrname FOREIGN KEY(ftest1, ftest2)
-                       REFERENCES PKTABLE MATCH FULL ON DELETE SET NULL ON UPDATE SET NULL);
+                       REFERENCES PKTABLE MATCH FULL ON DELETE SET NULL ON UPDATE SET NULL NOT VALID);
 
 -- Test comments
 COMMENT ON CONSTRAINT constrname_wrong ON FKTABLE IS 'fk constraint comment';
-- 
2.39.5

#8Amul Sul
sulamul@gmail.com
In reply to: Alvaro Herrera (#2)
Re: CREATE TABLE NOT VALID for check and foreign key

On Thu, Dec 5, 2024 at 3:06 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:

Hello,

On 2024-Dec-05, jian he wrote:

I found for foreign keys, check constraints,
you specify it as NOT VALID, it will not be marked as NOT VALID in the
CREATE TABLE statement.

Uhmm, okay.

reading transformCheckConstraints, transformFKConstraints comments
appearingly this is intentional?

If so, do we need to document the keywords "NOT VALID"
in create_table.sgml synopsis section?

So, the whole point of ALTER TABLE adding constraints marked NOT VALID
is to let the AccessExclusiveLock on the table be held for a very short
time, without requiring a table scan; you follow that with ALTER TABLE
VALIDATE to remove the marking, which takes a weaker lock. This is
great for production-time constraint additions on large tables. But for
CREATE TABLE there's no such argument: it's pointless to mark a
constraint as NOT VALID, because nobody else could be looking at the
table anyway.

Maybe it would have been wise to forbid NOT VALID when used with CREATE
TABLE. But we didn't. Should we do that now? Maybe we can just
document that you can specify it but it doesn't do anything.

I might be mistaken, but I believe this behavior is reasonable since
we're creating a new table with no data initially. Future inserts will
be validated against the constraint, ensuring all data added complies
with it. Given that any data in the table at any time will be valid
according to the constraint, marking it as "VALID" seems like the
correct approach, IMO.

However, when a constraint is added via an ALTER TABLE statement, we
don't scan the table, so we can't be sure whether the existing data
complies with the constraint. In this case, marking the constraint as
"NOT VALID" makes sense.

I'm not sure if it's worth documenting or raising a warning about this.

Regards,
Amul

#9Alvaro Herrera
alvherre@alvh.no-ip.org
In reply to: Amul Sul (#8)
Re: CREATE TABLE NOT VALID for check and foreign key

On 2025-Jan-15, Amul Sul wrote:

I might be mistaken, but I believe this behavior is reasonable since
we're creating a new table with no data initially. Future inserts will
be validated against the constraint, ensuring all data added complies
with it. Given that any data in the table at any time will be valid
according to the constraint, marking it as "VALID" seems like the
correct approach, IMO.

Yes to all of the above. But the complaint is not that the behavior is
incorrect. The complaint is that users add NOT VALID and the code
silently behaves as if the parameter was not given, which surprises them
unpleasantly. The WARNING message is there to tell them "look, I
realize you wanted to do this, but I know better, so I decided to ignore
you." That's all.

--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"Use it up, wear it out, make it do, or do without"