NOT VALID for Unique Indexes

Started by Simon Riggsalmost 5 years ago7 messages
#1Simon Riggs
simon.riggs@enterprisedb.com
1 attachment(s)

As you may be aware the NOT VALID qualifier currently only applies to
CHECK and FK constraints, but not yet to unique indexes. I have had
customer requests to change that.

It's a reasonably common requirement to be able to change an index
to/from a unique index, i.e. Unique -> NonUnique or NonUnique to
Unique. Previously, it was easy enough to do that using a catalog
update, but with security concerns and the fact that the optimizer
uses the uniqueness to optimize queries means that there is a gap in
our support. We obviously need to scan the index to see if it actually
can be marked as unique.

In terms of locking we need to exclude writes while we add uniqueness,
so scanning the index to check it is unique would cause problems. So
we need to do the same thing as we do with other constraint types: add
the constraint NOT VALID in one transaction and then later validate it
in a separate transaction (if ever).

I present a WIP patch to show it's a small patch to change Uniqueness
for an index, with docs and tests.

ALTER INDEX SET [NOT] UNIQUE [NOT VALID]
ALTER INDEX VALIDATE UNIQUE

It doesn't do the index validation scan (yet), but I wanted to check
acceptability, syntax and requirements before I do that.

I can also add similar syntax for UNIQUE and PK constraints.

Thoughts please?

--
Simon Riggs http://www.EnterpriseDB.com/

Attachments:

alter_index_set_unique_not_valid.v4.patchapplication/octet-stream; name=alter_index_set_unique_not_valid.v4.patchDownload
diff --git a/doc/src/sgml/ref/alter_index.sgml b/doc/src/sgml/ref/alter_index.sgml
index 214005a86c..7157466a7c 100644
--- a/doc/src/sgml/ref/alter_index.sgml
+++ b/doc/src/sgml/ref/alter_index.sgml
@@ -25,6 +25,8 @@ ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RENA
 ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET TABLESPACE <replaceable class="parameter">tablespace_name</replaceable>
 ALTER INDEX <replaceable class="parameter">name</replaceable> ATTACH PARTITION <replaceable class="parameter">index_name</replaceable>
 ALTER INDEX <replaceable class="parameter">name</replaceable> DEPENDS ON EXTENSION <replaceable class="parameter">extension_name</replaceable>
+ALTER INDEX <replaceable class="parameter">name</replaceable> SET [ NOT ] UNIQUE [ NOT VALID ]
+ALTER INDEX <replaceable class="parameter">name</replaceable> VALIDATE UNIQUE
 ALTER INDEX <replaceable class="parameter">name</replaceable> ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION
 ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> SET ( <replaceable class="parameter">storage_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )
 ALTER INDEX [ IF EXISTS ] <replaceable class="parameter">name</replaceable> RESET ( <replaceable class="parameter">storage_parameter</replaceable> [, ... ] )
@@ -113,6 +115,45 @@ ALTER INDEX ALL IN TABLESPACE <replaceable class="parameter">name</replaceable>
     </listitem>
    </varlistentry>
 
+   <varlistentry>
+    <term><literal>SET UNIQUE</literal></term>
+    <term><literal>SET NOT UNIQUE</literal></term>
+    <listitem>
+     <para>
+      This form alters the uniqueness of the index. The
+      <literal>UNIQUE NOT VALID</literal> option implements uniqueness checking
+      for the specified index immediately, but doesn't validate the uniqueness
+      of the index. This may mean that some optimizations are not yet
+      available because the planner will be unable to rely upon the
+      uniqueness of the index. If the <literal>NOT VALID</literal> option
+      is not specified then the lock is held while a potentially lengthy
+      scan of the index occurs to validate uniqueness.
+     </para>
+     <para>
+      <literal>SET NOT UNIQUE</literal> requires a
+      <literal>ACCESS EXCLUSIVE</literal> lock.
+      <literal>SET UNIQUE</literal> requires a
+      <literal>SHARE ROW EXCLUSIVE</literal> lock.
+     </para>
+    </listitem>
+   </varlistentry>
+
+   <varlistentry>
+    <term><literal>VALIDATE UNIQUE</literal></term>
+    <listitem>
+     <para>
+      This form validates the uniqueness of an index previously altered as
+      <literal>UNIQUE NOT VALID</literal> by scanning the index.
+      After the index's uniqueness has been validated this can be relied
+      upon during optimization in the planner.
+     </para>
+     <para>
+      <literal>VALIDATE UNIQUE</literal> requires a
+      <literal>SHARE UPDATE EXCLUSIVE</literal> lock.
+     </para>
+    </listitem>
+   </varlistentry>
+
    <varlistentry>
     <term><literal>ALTER COLLATION <replaceable class="parameter">collation_name</replaceable> REFRESH VERSION</literal></term>
     <listitem>
diff --git a/src/backend/bootstrap/bootparse.y b/src/backend/bootstrap/bootparse.y
index 5fcd004e1b..3bd2703b9e 100644
--- a/src/backend/bootstrap/bootparse.y
+++ b/src/backend/bootstrap/bootparse.y
@@ -342,6 +342,7 @@ Boot_DeclareUniqueIndexStmt:
 					stmt->oldCreateSubid = InvalidSubTransactionId;
 					stmt->oldFirstRelfilenodeSubid = InvalidSubTransactionId;
 					stmt->unique = true;
+					stmt->uniquevalid = true;
 					stmt->primary = false;
 					stmt->isconstraint = false;
 					stmt->deferrable = false;
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cffbc0ac38..f77bf66f98 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -609,6 +609,7 @@ UpdateIndexRelation(Oid indexoid,
 	values[Anum_pg_index_indisready - 1] = BoolGetDatum(isready);
 	values[Anum_pg_index_indislive - 1] = BoolGetDatum(true);
 	values[Anum_pg_index_indisreplident - 1] = BoolGetDatum(false);
+	values[Anum_pg_index_induniqvalid - 1] = BoolGetDatum(indexInfo->ii_UniqueValid);
 	values[Anum_pg_index_indkey - 1] = PointerGetDatum(indkey);
 	values[Anum_pg_index_indcollation - 1] = PointerGetDatum(indcollation);
 	values[Anum_pg_index_indclass - 1] = PointerGetDatum(indclass);
@@ -1493,6 +1494,7 @@ index_concurrently_create_copy(Relation heapRelation, Oid oldIndexId, const char
 							indexExprs,
 							indexPreds,
 							oldInfo->ii_Unique,
+							oldInfo->ii_UniqueValid,
 							false,	/* not ready for inserts */
 							true);
 
@@ -2491,6 +2493,7 @@ BuildIndexInfo(Relation index)
 					   RelationGetIndexExpressions(index),
 					   RelationGetIndexPredicate(index),
 					   indexStruct->indisunique,
+					   indexStruct->induniqvalid,
 					   indexStruct->indisready,
 					   false);
 
@@ -2550,6 +2553,7 @@ BuildDummyIndexInfo(Relation index)
 					   RelationGetDummyIndexExpressions(index),
 					   NIL,
 					   indexStruct->indisunique,
+					   indexStruct->induniqvalid,
 					   indexStruct->indisready,
 					   false);
 
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 992f4813b4..6bbf96975b 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -217,7 +217,7 @@ CheckIndexCompatible(Oid oldId,
 	 * ii_NumIndexKeyAttrs with same value.
 	 */
 	indexInfo = makeIndexInfo(numberOfAttributes, numberOfAttributes,
-							  accessMethodId, NIL, NIL, false, false, false);
+							  accessMethodId, NIL, NIL, false, false, false, false);
 	typeObjectId = (Oid *) palloc(numberOfAttributes * sizeof(Oid));
 	collationObjectId = (Oid *) palloc(numberOfAttributes * sizeof(Oid));
 	classObjectId = (Oid *) palloc(numberOfAttributes * sizeof(Oid));
@@ -868,6 +868,7 @@ DefineIndex(Oid relationId,
 							  NIL,	/* expressions, NIL for now */
 							  make_ands_implicit((Expr *) stmt->whereClause),
 							  stmt->unique,
+							  stmt->uniquevalid,
 							  !concurrent,
 							  concurrent);
 
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce11d..787bcf646d 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -875,6 +875,7 @@ is_usable_unique_index(Relation indexRel)
 	 * operators.
 	 */
 	if (indexStruct->indisunique &&
+		indexStruct->induniqvalid &&
 		indexStruct->indimmediate &&
 		indexRel->rd_rel->relam == BTREE_AM_OID &&
 		indexStruct->indisvalid &&
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 993da56d43..61f5bfcb70 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -559,6 +559,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
 static List *GetParentedForeignKeyRefs(Relation partition);
 static void ATDetachCheckNoForeignKeyRefs(Relation partition);
 static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
+static void ATExecAlterIndex(Relation rel, bool unique, bool valid);
 
 
 /* ----------------------------------------------------------------
@@ -3790,6 +3791,7 @@ AlterTableGetLockLevel(List *cmds)
 				 */
 			case AT_DropConstraint: /* as DROP INDEX */
 			case AT_DropNotNull:	/* may change some SQL plans */
+			case AT_SetIndexNotUnique:
 				cmd_lockmode = AccessExclusiveLock;
 				break;
 
@@ -3832,6 +3834,8 @@ AlterTableGetLockLevel(List *cmds)
 			case AT_DisableTrig:
 			case AT_DisableTrigAll:
 			case AT_DisableTrigUser:
+			case AT_SetIndexUnique:
+			case AT_SetIndexUniqueNotValid:
 				cmd_lockmode = ShareRowExclusiveLock;
 				break;
 
@@ -3953,6 +3957,7 @@ AlterTableGetLockLevel(List *cmds)
 				break;
 
 			case AT_ValidateConstraint: /* Uses MVCC in getConstraints() */
+			case AT_ValidateIndexUnique:
 				cmd_lockmode = ShareUpdateExclusiveLock;
 				break;
 
@@ -4365,6 +4370,26 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
 			/* No command-specific prep needed */
 			pass = AT_PASS_MISC;
 			break;
+		case AT_SetIndexNotUnique:
+			ATSimplePermissions(rel, ATT_INDEX);
+			/* No command-specific prep needed */
+			pass = AT_PASS_MISC;
+			break;
+		case AT_SetIndexUnique:
+			ATSimplePermissions(rel, ATT_INDEX);
+			/* No command-specific prep needed */
+			pass = AT_PASS_MISC;
+			break;
+		case AT_SetIndexUniqueNotValid:
+			ATSimplePermissions(rel, ATT_INDEX);
+			/* No command-specific prep needed */
+			pass = AT_PASS_MISC;
+			break;
+		case AT_ValidateIndexUnique:
+			ATSimplePermissions(rel, ATT_INDEX);
+			/* No command-specific prep needed */
+			pass = AT_PASS_MISC;
+			break;
 		default:				/* oops */
 			elog(ERROR, "unrecognized alter table type: %d",
 				 (int) cmd->subtype);
@@ -4753,6 +4778,17 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
 			Assert(rel->rd_rel->relkind == RELKIND_INDEX);
 			ATExecAlterCollationRefreshVersion(rel, cmd->object);
 			break;
+		case AT_SetIndexNotUnique:
+			ATExecAlterIndex(rel, false, true);
+			break;
+		case AT_SetIndexUnique:
+			ATExecAlterIndex(rel, true, true);
+			break;
+		case AT_SetIndexUniqueNotValid:
+			ATExecAlterIndex(rel, true, false);
+			break;
+		case AT_ValidateIndexUnique:
+			break;
 		default:				/* oops */
 			elog(ERROR, "unrecognized alter table type: %d",
 				 (int) cmd->subtype);
@@ -10295,7 +10331,7 @@ transformFkeyCheckAttrs(Relation pkrel,
 		/*
 		 * Must have the right number of columns; must be unique and not a
 		 * partial index; forget it if there are any expressions, too. Invalid
-		 * indexes are out as well.
+		 * indexes are out as well. Doesn't matter is uniqueness is valid yet.
 		 */
 		if (indexStruct->indnkeyatts == numattrs &&
 			indexStruct->indisunique &&
@@ -14713,7 +14749,8 @@ ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKMODE lockmode
 						RelationGetRelationName(rel))));
 	/* The AM must support uniqueness, and the index must in fact be unique. */
 	if (!indexRel->rd_indam->amcanunique ||
-		!indexRel->rd_index->indisunique)
+		!indexRel->rd_index->indisunique ||
+		!indexRel->rd_index->induniqvalid)
 		ereport(ERROR,
 				(errcode(ERRCODE_WRONG_OBJECT_TYPE),
 				 errmsg("cannot use non-unique index \"%s\" as replica identity",
@@ -17631,3 +17668,39 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
 	index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
 	CacheInvalidateRelcache(rel);
 }
+
+static void
+ATExecAlterIndex(Relation rel, bool unique, bool valid)
+{
+	Relation	pg_index;
+	HeapTuple	pg_index_tuple;
+	Form_pg_index pg_index_form;
+	bool		dirty = false;
+
+	pg_index = table_open(IndexRelationId, RowExclusiveLock);
+	pg_index_tuple = SearchSysCacheCopy1(INDEXRELID,
+											 ObjectIdGetDatum(RelationGetRelid(rel)));
+	if (!HeapTupleIsValid(pg_index_tuple))
+		elog(ERROR, "cache lookup failed for index %u", RelationGetRelid(rel));
+	pg_index_form = (Form_pg_index) GETSTRUCT(pg_index_tuple);
+
+	if (pg_index_form->indisunique != unique)
+	{
+		pg_index_form->indisunique = unique;
+		dirty = true;
+
+		if (unique && valid)
+			Assert(true); // Call validation
+		pg_index_form->induniqvalid = valid;
+	}
+
+	if (dirty)
+	{
+		CatalogTupleUpdate(pg_index, &pg_index_tuple->t_self, pg_index_tuple);
+		InvokeObjectPostAlterHookArg(IndexRelationId, RelationGetRelid(rel), 0,
+									 InvalidOid, true);
+	}
+
+	heap_freetuple(pg_index_tuple);
+	table_close(pg_index, RowExclusiveLock);
+}
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 2aafcc8f22..3f6429282a 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -379,6 +379,8 @@ ExecInsertIndexTuples(ResultRelInfo *resultRelInfo,
 		 *
 		 * For a speculative insertion (used by INSERT ... ON CONFLICT), do
 		 * the same as for a deferrable unique index.
+		 *
+		 * It doesn't matter here whether uniqueness is valid or not.
 		 */
 		if (!indexRelation->rd_index->indisunique)
 			checkUnique = UNIQUE_CHECK_NO;
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index ba3ccc712c..40015f4f30 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3531,6 +3531,7 @@ _copyIndexStmt(const IndexStmt *from)
 	COPY_SCALAR_FIELD(oldCreateSubid);
 	COPY_SCALAR_FIELD(oldFirstRelfilenodeSubid);
 	COPY_SCALAR_FIELD(unique);
+	COPY_SCALAR_FIELD(uniquevalid);
 	COPY_SCALAR_FIELD(primary);
 	COPY_SCALAR_FIELD(isconstraint);
 	COPY_SCALAR_FIELD(deferrable);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index a2ef853dc2..c0d28287d6 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -1358,6 +1358,7 @@ _equalIndexStmt(const IndexStmt *a, const IndexStmt *b)
 	COMPARE_SCALAR_FIELD(oldCreateSubid);
 	COMPARE_SCALAR_FIELD(oldFirstRelfilenodeSubid);
 	COMPARE_SCALAR_FIELD(unique);
+	COMPARE_SCALAR_FIELD(uniquevalid);
 	COMPARE_SCALAR_FIELD(primary);
 	COMPARE_SCALAR_FIELD(isconstraint);
 	COMPARE_SCALAR_FIELD(deferrable);
diff --git a/src/backend/nodes/makefuncs.c b/src/backend/nodes/makefuncs.c
index 01c110cd2f..ef8083be74 100644
--- a/src/backend/nodes/makefuncs.c
+++ b/src/backend/nodes/makefuncs.c
@@ -741,7 +741,7 @@ make_ands_implicit(Expr *clause)
  */
 IndexInfo *
 makeIndexInfo(int numattrs, int numkeyattrs, Oid amoid, List *expressions,
-			  List *predicates, bool unique, bool isready, bool concurrent)
+			  List *predicates, bool unique, bool uniquevalid, bool isready, bool concurrent)
 {
 	IndexInfo  *n = makeNode(IndexInfo);
 
@@ -750,6 +750,7 @@ makeIndexInfo(int numattrs, int numkeyattrs, Oid amoid, List *expressions,
 	Assert(n->ii_NumIndexKeyAttrs != 0);
 	Assert(n->ii_NumIndexKeyAttrs <= n->ii_NumIndexAttrs);
 	n->ii_Unique = unique;
+	n->ii_UniqueValid = uniquevalid;
 	n->ii_ReadyForInserts = isready;
 	n->ii_Concurrent = concurrent;
 
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8392be6d44..950e1bdade 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2697,6 +2697,7 @@ _outIndexStmt(StringInfo str, const IndexStmt *node)
 	WRITE_UINT_FIELD(oldCreateSubid);
 	WRITE_UINT_FIELD(oldFirstRelfilenodeSubid);
 	WRITE_BOOL_FIELD(unique);
+	WRITE_BOOL_FIELD(uniquevalid);
 	WRITE_BOOL_FIELD(primary);
 	WRITE_BOOL_FIELD(isconstraint);
 	WRITE_BOOL_FIELD(deferrable);
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index da322b453e..cf2ff2d866 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -396,6 +396,7 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
 			info->indrestrictinfo = NIL;	/* set later, in indxpath.c */
 			info->predOK = false;	/* set later, in indxpath.c */
 			info->unique = index->indisunique;
+			info->uniquevalid = index->induniqvalid;
 			info->immediate = index->indimmediate;
 			info->hypothetical = false;
 
@@ -726,7 +727,8 @@ infer_arbiter_indexes(PlannerInfo *root)
 		 * Note that we do not perform a check against indcheckxmin (like e.g.
 		 * get_relation_info()) here to eliminate candidates, because
 		 * uniqueness checking only cares about the most recently committed
-		 * tuple versions.
+		 * tuple versions. So we allow an index that is defined as unique
+		 * even if the uniqueness is not yet valid.
 		 */
 
 		/*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 31c95443a5..2c8f44c09c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -300,7 +300,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
 %type <ival>	add_drop opt_asc_desc opt_nulls_order
 
 %type <node>	alter_table_cmd alter_type_cmd opt_collate_clause
-	   replica_identity partition_cmd index_partition_cmd
+	   replica_identity partition_cmd index_partition_cmd alter_index_cmd
 %type <list>	alter_table_cmds alter_type_cmds
 %type <list>    alter_identity_column_option_list
 %type <defelt>  alter_identity_column_option
@@ -1961,6 +1961,15 @@ AlterTableStmt:
 					n->missing_ok = false;
 					$$ = (Node *)n;
 				}
+		|	ALTER INDEX qualified_name alter_index_cmd
+				{
+					AlterTableStmt *n = makeNode(AlterTableStmt);
+					n->relation = $3;
+					n->cmds = list_make1($4);
+					n->objtype = OBJECT_INDEX;
+					n->missing_ok = false;
+					$$ = (Node *)n;
+				}
 		|	ALTER INDEX ALL IN_P TABLESPACE name SET TABLESPACE name opt_nowait
 				{
 					AlterTableMoveAllStmt *n =
@@ -2129,6 +2138,38 @@ index_partition_cmd:
 				}
 		;
 
+alter_index_cmd:
+			/* ALTER INDEX <name> SET [NOT] UNIQUE [NOT VALID] */
+			SET NOT UNIQUE
+				{
+					AlterTableCmd *n = makeNode(AlterTableCmd);
+
+					n->subtype = AT_SetIndexNotUnique;
+					$$ = (Node *)n;
+				}
+			| SET UNIQUE
+				{
+					AlterTableCmd *n = makeNode(AlterTableCmd);
+
+					n->subtype = AT_SetIndexUnique;
+					$$ = (Node *)n;
+				}
+			| SET UNIQUE NOT VALID
+				{
+					AlterTableCmd *n = makeNode(AlterTableCmd);
+
+					n->subtype = AT_SetIndexUniqueNotValid;
+					$$ = (Node *)n;
+				}
+			| VALIDATE UNIQUE
+				{
+					AlterTableCmd *n = makeNode(AlterTableCmd);
+
+					n->subtype = AT_ValidateIndexUnique;
+					$$ = (Node *)n;
+				}
+		;
+
 alter_table_cmd:
 			/* ALTER TABLE <name> ADD <coldef> */
 			ADD_P columnDef
@@ -7224,6 +7265,7 @@ IndexStmt:	CREATE opt_unique INDEX opt_concurrently opt_index_name
 				{
 					IndexStmt *n = makeNode(IndexStmt);
 					n->unique = $2;
+					n->uniquevalid = $2;
 					n->concurrent = $4;
 					n->idxname = $5;
 					n->relation = $7;
@@ -7254,6 +7296,7 @@ IndexStmt:	CREATE opt_unique INDEX opt_concurrently opt_index_name
 				{
 					IndexStmt *n = makeNode(IndexStmt);
 					n->unique = $2;
+					n->uniquevalid = $2;
 					n->concurrent = $4;
 					n->idxname = $8;
 					n->relation = $10;
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index b31f3afa03..e85c1e79a5 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1570,6 +1570,7 @@ generateClonedIndexStmt(RangeVar *heapRel, Relation source_idx,
 	index->oldCreateSubid = InvalidSubTransactionId;
 	index->oldFirstRelfilenodeSubid = InvalidSubTransactionId;
 	index->unique = idxrec->indisunique;
+	index->uniquevalid = idxrec->induniqvalid;
 	index->primary = idxrec->indisprimary;
 	index->transformed = true;	/* don't need transformIndexStmt */
 	index->concurrent = false;
@@ -2114,6 +2115,7 @@ transformIndexConstraint(Constraint *constraint, CreateStmtContext *cxt)
 	index = makeNode(IndexStmt);
 
 	index->unique = (constraint->contype != CONSTR_EXCLUSION);
+	index->uniquevalid = true;
 	index->primary = (constraint->contype == CONSTR_PRIMARY);
 	if (index->primary)
 	{
diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c
index ba95755867..00454c7ed9 100644
--- a/src/backend/utils/cache/catcache.c
+++ b/src/backend/utils/cache/catcache.c
@@ -1056,6 +1056,7 @@ InitCatCachePhase2(CatCache *cache, bool touch_index)
 		 * about the pg_am indexes not getting tested.
 		 */
 		Assert(idesc->rd_index->indisunique &&
+			   idesc->rd_index->induniqvalid &&
 			   idesc->rd_index->indimmediate);
 
 		index_close(idesc, AccessShareLock);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 7ef510cd01..64a9d23773 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -2261,6 +2261,7 @@ RelationReloadIndexInfo(Relation relation)
 		relation->rd_index->indcheckxmin = index->indcheckxmin;
 		relation->rd_index->indisready = index->indisready;
 		relation->rd_index->indislive = index->indislive;
+		relation->rd_index->induniqvalid = index->induniqvalid;
 
 		/* Copy xmin too, as that is needed to make sense of indcheckxmin */
 		HeapTupleHeaderSetXmin(relation->rd_indextuple->t_data,
@@ -4580,7 +4581,7 @@ RelationGetIndexList(Relation relation)
 		 * so don't check them.
 		 */
 		if (!index->indisvalid || !index->indisunique ||
-			!index->indimmediate ||
+			!index->indimmediate || !index->induniqvalid ||
 			!heap_attisnull(htup, Anum_pg_index_indpred, NULL))
 			continue;
 
@@ -5060,7 +5061,10 @@ restart:
 		else
 			indexPredicate = NULL;
 
-		/* Can this index be referenced by a foreign key? */
+		/*
+		 * Can this index be referenced by a foreign key?
+		 * Yes, even if uniqueness is not yet valid (!induniqvalid)
+		 */
 		isKey = indexDesc->rd_index->indisunique &&
 			indexExpressions == NULL &&
 			indexPredicate == NULL;
diff --git a/src/include/catalog/pg_index.h b/src/include/catalog/pg_index.h
index b6d7ebec55..551499c93f 100644
--- a/src/include/catalog/pg_index.h
+++ b/src/include/catalog/pg_index.h
@@ -42,6 +42,7 @@ CATALOG(pg_index,2610,IndexRelationId) BKI_SCHEMA_MACRO
 	bool		indisready;		/* is this index ready for inserts? */
 	bool		indislive;		/* is this index alive at all? */
 	bool		indisreplident; /* is this index the identity for replication? */
+	bool		induniqvalid;	/* is uniqueness valid for use by queries? */
 
 	/* variable-length fields start here, but we allow direct access to indkey */
 	int2vector	indkey BKI_FORCE_NOT_NULL;	/* column numbers of indexed cols,
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 48c3f570fa..18fd3e7e9e 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -169,6 +169,7 @@ typedef struct IndexInfo
 	uint16	   *ii_UniqueStrats;	/* array with one entry per column */
 	Datum	   *ii_OpclassOptions;	/* array with one entry per column */
 	bool		ii_Unique;
+	bool		ii_UniqueValid;
 	bool		ii_ReadyForInserts;
 	bool		ii_Concurrent;
 	bool		ii_BrokenHotChain;
diff --git a/src/include/nodes/makefuncs.h b/src/include/nodes/makefuncs.h
index 48a7ebfe45..827b4d795f 100644
--- a/src/include/nodes/makefuncs.h
+++ b/src/include/nodes/makefuncs.h
@@ -96,7 +96,7 @@ extern List *make_ands_implicit(Expr *clause);
 
 extern IndexInfo *makeIndexInfo(int numattrs, int numkeyattrs, Oid amoid,
 								List *expressions, List *predicates,
-								bool unique, bool isready, bool concurrent);
+								bool unique, bool uniquevalid, bool isready, bool concurrent);
 
 extern DefElem *makeDefElem(char *name, Node *arg, int location);
 extern DefElem *makeDefElemExtended(char *nameSpace, char *name, Node *arg,
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index dc2bb40926..3107694899 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1873,7 +1873,11 @@ typedef enum AlterTableType
 	AT_AddIdentity,				/* ADD IDENTITY */
 	AT_SetIdentity,				/* SET identity column options */
 	AT_DropIdentity,			/* DROP IDENTITY */
-	AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+	AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+	AT_SetIndexNotUnique,		/* ALTER INDEX SET NOT UNIQUE */
+	AT_SetIndexUnique,			/* ALTER INDEX SET UNIQUE */
+	AT_SetIndexUniqueNotValid,	/* ALTER INDEX SET UNIQUE NOT VALID */
+	AT_ValidateIndexUnique		/* ALTER INDEX VALIDATE UNIQUE */
 } AlterTableType;
 
 typedef struct ReplicaIdentityStmt
@@ -2806,6 +2810,7 @@ typedef struct IndexStmt
 	SubTransactionId oldFirstRelfilenodeSubid;	/* rd_firstRelfilenodeSubid of
 												 * oldNode */
 	bool		unique;			/* is index unique? */
+	bool		uniquevalid;	/* is index uniqueness valid? */
 	bool		primary;		/* is index a primary key? */
 	bool		isconstraint;	/* is it for a pkey/unique constraint? */
 	bool		deferrable;		/* is the constraint DEFERRABLE? */
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index cde2637798..eff6714fcd 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -853,6 +853,7 @@ struct IndexOptInfo
 
 	bool		predOK;			/* true if index predicate matches query */
 	bool		unique;			/* true if a unique index */
+	bool		uniquevalid;	/* true if uniqueness has been validated */
 	bool		immediate;		/* is uniqueness enforced immediately? */
 	bool		hypothetical;	/* true if index doesn't really exist */
 
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4622..ed7b2f1408 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -228,6 +228,11 @@ ALTER INDEX IF EXISTS __attmp_onek_unique1 RENAME TO onek_unique1;
 NOTICE:  relation "__attmp_onek_unique1" does not exist, skipping
 ALTER INDEX onek_unique1 RENAME TO attmp_onek_unique1;
 ALTER INDEX attmp_onek_unique1 RENAME TO onek_unique1;
+ALTER INDEX onek_unique1 SET NOT UNIQUE;
+ALTER INDEX onek_unique1 SET UNIQUE NOT VALID;
+ALTER INDEX onek_unique1 VALIDATE UNIQUE;
+ALTER INDEX onek_unique1 SET NOT UNIQUE;
+ALTER INDEX onek_unique1 SET UNIQUE;
 SET ROLE regress_alter_table_user1;
 ALTER INDEX onek_unique1 RENAME TO fail;  -- permission denied
 ERROR:  must be owner of index onek_unique1
diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql
index 4cc55d8525..f6ce1d695b 100644
--- a/src/test/regress/sql/alter_table.sql
+++ b/src/test/regress/sql/alter_table.sql
@@ -223,6 +223,12 @@ ALTER INDEX IF EXISTS __attmp_onek_unique1 RENAME TO onek_unique1;
 ALTER INDEX onek_unique1 RENAME TO attmp_onek_unique1;
 ALTER INDEX attmp_onek_unique1 RENAME TO onek_unique1;
 
+ALTER INDEX onek_unique1 SET NOT UNIQUE;
+ALTER INDEX onek_unique1 SET UNIQUE NOT VALID;
+ALTER INDEX onek_unique1 VALIDATE UNIQUE;
+ALTER INDEX onek_unique1 SET NOT UNIQUE;
+ALTER INDEX onek_unique1 SET UNIQUE;
+
 SET ROLE regress_alter_table_user1;
 ALTER INDEX onek_unique1 RENAME TO fail;  -- permission denied
 RESET ROLE;
#2David Fetter
david@fetter.org
In reply to: Simon Riggs (#1)
Re: NOT VALID for Unique Indexes

On Thu, Jan 14, 2021 at 04:22:17PM +0000, Simon Riggs wrote:

As you may be aware the NOT VALID qualifier currently only applies to
CHECK and FK constraints, but not yet to unique indexes. I have had
customer requests to change that.

This is a great feature!

Not exactly on point with this, but in a pretty closely related
context, is there some way we could give people the ability to declare
at their peril that a constraint is valid without incurring the full
scan that VALIDATE currently does? This is currently doable by
fiddling directly with the catalog, which operation is broadly more
dangerous and ill-advised.

Best,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

#3japin
japinli@hotmail.com
In reply to: Simon Riggs (#1)
Re: NOT VALID for Unique Indexes

On Fri, 15 Jan 2021 at 00:22, Simon Riggs <simon.riggs@enterprisedb.com> wrote:

As you may be aware the NOT VALID qualifier currently only applies to
CHECK and FK constraints, but not yet to unique indexes. I have had
customer requests to change that.

It's a reasonably common requirement to be able to change an index
to/from a unique index, i.e. Unique -> NonUnique or NonUnique to
Unique. Previously, it was easy enough to do that using a catalog
update, but with security concerns and the fact that the optimizer
uses the uniqueness to optimize queries means that there is a gap in
our support. We obviously need to scan the index to see if it actually
can be marked as unique.

In terms of locking we need to exclude writes while we add uniqueness,
so scanning the index to check it is unique would cause problems. So
we need to do the same thing as we do with other constraint types: add
the constraint NOT VALID in one transaction and then later validate it
in a separate transaction (if ever).

I present a WIP patch to show it's a small patch to change Uniqueness
for an index, with docs and tests.

ALTER INDEX SET [NOT] UNIQUE [NOT VALID]
ALTER INDEX VALIDATE UNIQUE

It doesn't do the index validation scan (yet), but I wanted to check
acceptability, syntax and requirements before I do that.

I can also add similar syntax for UNIQUE and PK constraints.

Thoughts please?

Great! I have some questions.

1. In the patch, you add a new attribute named "induniquevalid" in pg_index,
however, there is a "indisvalid" in pg_index, can we use "indisvalid"?

2. The foreign key and CHECK constraints are valid by using
ALTER TABLE .. ADD table_constraint [ NOT VALID ]
ALTER TABLE .. VALIDATE CONSTRAINT constraint_name

Should we implement unique index valid/not valid same as foreign key and
CHECK constraints?

3. If we use the syntax to valid/not valid the unique, should we support
other constraints, such as foreign key and CHECK constraints?

--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.

#4Simon Riggs
simon.riggs@enterprisedb.com
In reply to: David Fetter (#2)
Re: NOT VALID for Unique Indexes

On Mon, Jan 18, 2021 at 12:34 AM David Fetter <david@fetter.org> wrote:

On Thu, Jan 14, 2021 at 04:22:17PM +0000, Simon Riggs wrote:

As you may be aware the NOT VALID qualifier currently only applies to
CHECK and FK constraints, but not yet to unique indexes. I have had
customer requests to change that.

This is a great feature!

Not exactly on point with this, but in a pretty closely related
context, is there some way we could give people the ability to declare
at their peril that a constraint is valid without incurring the full
scan that VALIDATE currently does? This is currently doable by
fiddling directly with the catalog, which operation is broadly more
dangerous and ill-advised.

That is what NOT VALID allows, but it can't be relied on for optimization.

--
Simon Riggs http://www.EnterpriseDB.com/

#5Simon Riggs
simon.riggs@enterprisedb.com
In reply to: japin (#3)
Re: NOT VALID for Unique Indexes

On Mon, Jan 18, 2021 at 11:19 PM japin <japinli@hotmail.com> wrote:

On Fri, 15 Jan 2021 at 00:22, Simon Riggs <simon.riggs@enterprisedb.com> wrote:

As you may be aware the NOT VALID qualifier currently only applies to
CHECK and FK constraints, but not yet to unique indexes. I have had
customer requests to change that.

It's a reasonably common requirement to be able to change an index
to/from a unique index, i.e. Unique -> NonUnique or NonUnique to
Unique. Previously, it was easy enough to do that using a catalog
update, but with security concerns and the fact that the optimizer
uses the uniqueness to optimize queries means that there is a gap in
our support. We obviously need to scan the index to see if it actually
can be marked as unique.

In terms of locking we need to exclude writes while we add uniqueness,
so scanning the index to check it is unique would cause problems. So
we need to do the same thing as we do with other constraint types: add
the constraint NOT VALID in one transaction and then later validate it
in a separate transaction (if ever).

I present a WIP patch to show it's a small patch to change Uniqueness
for an index, with docs and tests.

ALTER INDEX SET [NOT] UNIQUE [NOT VALID]
ALTER INDEX VALIDATE UNIQUE

It doesn't do the index validation scan (yet), but I wanted to check
acceptability, syntax and requirements before I do that.

I can also add similar syntax for UNIQUE and PK constraints.

Thoughts please?

Great! I have some questions.

1. In the patch, you add a new attribute named "induniquevalid" in pg_index,
however, there is a "indisvalid" in pg_index, can we use "indisvalid"?

indisvalid already has defined meaning related to creating indexes
concurrently, so I was forced to create another column with a similar
name.

Thanks for reviewing the code in detail.

2. The foreign key and CHECK constraints are valid by using
ALTER TABLE .. ADD table_constraint [ NOT VALID ]
ALTER TABLE .. VALIDATE CONSTRAINT constraint_name

Should we implement unique index valid/not valid same as foreign key and
CHECK constraints?

Yes, that is possible. (I wrote the NOT VALID patch for FKs, so was
aware of that).

The syntax I presented was for ALTER INDEX. Not all UNIQUE indexes are
constraints, so it is important to add the option on ALTER INDEX.
Adding the ALTER TABLE syntax can be done later.

3. If we use the syntax to valid/not valid the unique, should we support
other constraints, such as foreign key and CHECK constraints?

I'm sorry, I don't understand that question. FKs and CHECK constrants
are already supported, as you point out above.

I won't be able to finish this patch in time for this next CF, but
thanks for your interest, I will complete for PG15 later this year.

--
Simon Riggs http://www.EnterpriseDB.com/

#6japin
japinli@hotmail.com
In reply to: Simon Riggs (#5)
Re: NOT VALID for Unique Indexes

On Fri, 26 Feb 2021 at 17:36, Simon Riggs <simon.riggs@enterprisedb.com> wrote:

On Mon, Jan 18, 2021 at 11:19 PM japin <japinli@hotmail.com> wrote:

On Fri, 15 Jan 2021 at 00:22, Simon Riggs <simon.riggs@enterprisedb.com> wrote:

As you may be aware the NOT VALID qualifier currently only applies to
CHECK and FK constraints, but not yet to unique indexes. I have had
customer requests to change that.

It's a reasonably common requirement to be able to change an index
to/from a unique index, i.e. Unique -> NonUnique or NonUnique to
Unique. Previously, it was easy enough to do that using a catalog
update, but with security concerns and the fact that the optimizer
uses the uniqueness to optimize queries means that there is a gap in
our support. We obviously need to scan the index to see if it actually
can be marked as unique.

In terms of locking we need to exclude writes while we add uniqueness,
so scanning the index to check it is unique would cause problems. So
we need to do the same thing as we do with other constraint types: add
the constraint NOT VALID in one transaction and then later validate it
in a separate transaction (if ever).

I present a WIP patch to show it's a small patch to change Uniqueness
for an index, with docs and tests.

ALTER INDEX SET [NOT] UNIQUE [NOT VALID]
ALTER INDEX VALIDATE UNIQUE

It doesn't do the index validation scan (yet), but I wanted to check
acceptability, syntax and requirements before I do that.

I can also add similar syntax for UNIQUE and PK constraints.

Thoughts please?

Great! I have some questions.

1. In the patch, you add a new attribute named "induniquevalid" in pg_index,
however, there is a "indisvalid" in pg_index, can we use "indisvalid"?

indisvalid already has defined meaning related to creating indexes
concurrently, so I was forced to create another column with a similar
name.

The doc of indisvalid says [1]https://www.postgresql.org/docs/devel/catalog-pg-index.html:

If true, the index is currently valid for queries. False means the index
is possibly incomplete: it must still be modified by INSERT/UPDATE operations,
but it cannot safely be used for queries. If it is unique, the uniqueness
property is not guaranteed true either.

So I think we can use it instead of create a new column. Does induniquevalid
have any other special meaning?

Thanks for reviewing the code in detail.

2. The foreign key and CHECK constraints are valid by using
ALTER TABLE .. ADD table_constraint [ NOT VALID ]
ALTER TABLE .. VALIDATE CONSTRAINT constraint_name

Should we implement unique index valid/not valid same as foreign key and
CHECK constraints?

Yes, that is possible. (I wrote the NOT VALID patch for FKs, so was
aware of that).

The syntax I presented was for ALTER INDEX. Not all UNIQUE indexes are
constraints, so it is important to add the option on ALTER INDEX.
Adding the ALTER TABLE syntax can be done later.

3. If we use the syntax to valid/not valid the unique, should we support
other constraints, such as foreign key and CHECK constraints?

I'm sorry, I don't understand that question. FKs and CHECK constrants
are already supported, as you point out above.

I'm sorry, I mixed the indexes and constraints.

[1]: https://www.postgresql.org/docs/devel/catalog-pg-index.html

--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.

#7Daniel Gustafsson
daniel@yesql.se
In reply to: Simon Riggs (#5)
Re: NOT VALID for Unique Indexes

On 26 Feb 2021, at 10:36, Simon Riggs <simon.riggs@enterprisedb.com> wrote:

I won't be able to finish this patch in time for this next CF, but
thanks for your interest, I will complete for PG15 later this year.

This patch no longer applies to HEAD, will there be an updated version for this
CF?

--
Daniel Gustafsson https://vmware.com/