ALTER TYPE 2: skip already-provable no-work rewrites

Started by Noah Mischabout 15 years ago43 messages
#1Noah Misch
noah@leadboat.com
1 attachment(s)

This patch removes ALTER TYPE rewrites in cases we can already prove valid. I
add a function GetCoerceExemptions() that walks an Expr according to rules
discussed in the design thread, simplified slightly pending additions in the
next patch. See the comment at that function for a refresher. I use it to
populate two new bools to AlteredTableInfo, "new_bits" and "mayerror".
"new_bits" is a superset of "new_changedoids", so I subsume that. I change
ATRewriteTable to act on those and support the notion of evaluating the
transformation expressions when we're not rewriting the table.

As unintended fallout, it's no longer an error to add oids or a column with a
default value to a table whose rowtype is used in columns elsewhere. This seems
best. Defaults on the origin table do not even apply to new inserts into such a
column, and the rowtype does not gain an OID column via its table.

This helps on conversions like varchar(X)->text, xml->text, and conversions
between domains and their base types.

Attachments:

at2-skip-nowork.patchtext/plain; charset=us-asciiDownload
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 71,76 ****
--- 71,77 ----
  #include "storage/smgr.h"
  #include "utils/acl.h"
  #include "utils/builtins.h"
+ #include "utils/datum.h"
  #include "utils/fmgroids.h"
  #include "utils/inval.h"
  #include "utils/lsyscache.h"
***************
*** 140,147 **** typedef struct AlteredTableInfo
  	/* Information saved by Phases 1/2 for Phase 3: */
  	List	   *constraints;	/* List of NewConstraint */
  	List	   *newvals;		/* List of NewColumnValue */
  	bool		new_notnull;	/* T if we added new NOT NULL constraints */
- 	bool		new_changeoids; /* T if we added/dropped the OID column */
  	Oid			newTableSpace;	/* new tablespace; 0 means no change */
  	/* Objects to rebuild after completing ALTER TYPE operations */
  	List	   *changedConstraintOids;	/* OIDs of constraints to rebuild */
--- 141,149 ----
  	/* Information saved by Phases 1/2 for Phase 3: */
  	List	   *constraints;	/* List of NewConstraint */
  	List	   *newvals;		/* List of NewColumnValue */
+ 	bool		new_bits;		/* Could stored tuple or OID bits change? */
+ 	bool		mayerror;		/* Could the newval expressions throw errors? */
  	bool		new_notnull;	/* T if we added new NOT NULL constraints */
  	Oid			newTableSpace;	/* new tablespace; 0 means no change */
  	/* Objects to rebuild after completing ALTER TYPE operations */
  	List	   *changedConstraintOids;	/* OIDs of constraints to rebuild */
***************
*** 3184,3190 **** ATRewriteTables(List **wqueue, LOCKMODE lockmode)
  		 * We only need to rewrite the table if at least one column needs to
  		 * be recomputed, or we are adding/removing the OID column.
  		 */
! 		if (tab->newvals != NIL || tab->new_changeoids)
  		{
  			/* Build a temporary relation and copy data */
  			Relation	OldHeap;
--- 3186,3192 ----
  		 * We only need to rewrite the table if at least one column needs to
  		 * be recomputed, or we are adding/removing the OID column.
  		 */
! 		if (tab->new_bits)
  		{
  			/* Build a temporary relation and copy data */
  			Relation	OldHeap;
***************
*** 3250,3256 **** ATRewriteTables(List **wqueue, LOCKMODE lockmode)
  			 * Test the current data within the table against new constraints
  			 * generated by ALTER TABLE commands, but don't rebuild data.
  			 */
! 			if (tab->constraints != NIL || tab->new_notnull)
  				ATRewriteTable(tab, InvalidOid, lockmode);
  
  			/*
--- 3252,3258 ----
  			 * Test the current data within the table against new constraints
  			 * generated by ALTER TABLE commands, but don't rebuild data.
  			 */
! 			if (tab->mayerror || tab->constraints != NIL || tab->new_notnull)
  				ATRewriteTable(tab, InvalidOid, lockmode);
  
  			/*
***************
*** 3310,3316 **** ATRewriteTables(List **wqueue, LOCKMODE lockmode)
  /*
   * ATRewriteTable: scan or rewrite one table
   *
!  * OIDNewHeap is InvalidOid if we don't need to rewrite
   */
  static void
  ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
--- 3312,3319 ----
  /*
   * ATRewriteTable: scan or rewrite one table
   *
!  * OIDNewHeap is InvalidOid if we don't need to rewrite.  If the only new
!  * constraints are foreign key constraints, we do nothing here.
   */
  static void
  ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
***************
*** 3319,3325 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  	Relation	newrel;
  	TupleDesc	oldTupDesc;
  	TupleDesc	newTupDesc;
! 	bool		needscan = false;
  	List	   *notnull_attrs;
  	int			i;
  	ListCell   *l;
--- 3322,3328 ----
  	Relation	newrel;
  	TupleDesc	oldTupDesc;
  	TupleDesc	newTupDesc;
! 	bool		needscan = tab->mayerror;
  	List	   *notnull_attrs;
  	int			i;
  	ListCell   *l;
***************
*** 3365,3383 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  	}
  
  	/*
- 	 * If we need to rewrite the table, the operation has to be propagated to
- 	 * tables that use this table's rowtype as a column type.
- 	 *
- 	 * (Eventually this will probably become true for scans as well, but at
- 	 * the moment a composite type does not enforce any constraints, so it's
- 	 * not necessary/appropriate to enforce them just during ALTER.)
- 	 */
- 	if (newrel)
- 		find_composite_type_dependencies(oldrel->rd_rel->reltype,
- 										 RelationGetRelationName(oldrel),
- 										 NULL);
- 
- 	/*
  	 * Generate the constraint and default execution states
  	 */
  
--- 3368,3373 ----
***************
*** 3430,3435 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
--- 3420,3426 ----
  			needscan = true;
  	}
  
+ 	/* newrel && !needscan arises when only adding OIDs. */
  	if (newrel || needscan)
  	{
  		ExprContext *econtext;
***************
*** 3494,3511 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  
  		while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
  		{
! 			if (newrel)
  			{
  				Oid			tupOid = InvalidOid;
  
! 				/* Extract data from old tuple */
! 				heap_deform_tuple(tuple, oldTupDesc, values, isnull);
! 				if (oldTupDesc->tdhasoid)
! 					tupOid = HeapTupleGetOid(tuple);
! 
! 				/* Set dropped attributes to null in new tuple */
! 				foreach(lc, dropped_attrs)
! 					isnull[lfirst_int(lc)] = true;
  
  				/*
  				 * Process supplied expressions to replace selected columns.
--- 3485,3515 ----
  
  		while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
  		{
! 			/*
! 			 * If we're rewriting or verifying, compute new tuple values using
! 			 * each transformation expression.  When rewriting, also form a new
! 			 * physical tuple.  In Assert-enabled builds, check for false
! 			 * negatives of tab->new_bits by comparing the data.
! 			 */
! 			if (newrel || tab->mayerror)
  			{
  				Oid			tupOid = InvalidOid;
  
! 				if (newrel
! #ifdef USE_ASSERT_CHECKING
! 					|| assert_enabled
! #endif
! 					)
! 				{
! 					/* Extract data from old tuple */
! 					heap_deform_tuple(tuple, oldTupDesc, values, isnull);
! 					if (oldTupDesc->tdhasoid)
! 						tupOid = HeapTupleGetOid(tuple);
! 
! 					/* Set dropped attributes to null in new tuple */
! 					foreach(lc, dropped_attrs)
! 						isnull[lfirst_int(lc)] = true;
! 				}
  
  				/*
  				 * Process supplied expressions to replace selected columns.
***************
*** 3522,3538 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  														  econtext,
  													 &isnull[ex->attnum - 1],
  														  NULL);
  				}
  
! 				/*
! 				 * Form the new tuple. Note that we don't explicitly pfree it,
! 				 * since the per-tuple memory context will be reset shortly.
! 				 */
! 				tuple = heap_form_tuple(newTupDesc, values, isnull);
  
! 				/* Preserve OID, if any */
! 				if (newTupDesc->tdhasoid)
! 					HeapTupleSetOid(tuple, tupOid);
  			}
  
  			/* Now check any constraints on the possibly-changed tuple */
--- 3526,3569 ----
  														  econtext,
  													 &isnull[ex->attnum - 1],
  														  NULL);
+ 
+ #ifdef USE_ASSERT_CHECKING
+ 					if (assert_enabled)
+ 					{
+ 						Datum		oldval = values[ex->attnum - 1];
+ 						bool		oldisnull = isnull[ex->attnum - 1];
+ 						Form_pg_attribute f = newTupDesc->attrs[ex->attnum - 1];
+ 
+ 						if (f->attbyval && f->attlen == -1)
+ 							oldval = PointerGetDatum(PG_DETOAST_DATUM(oldval));
+ 
+ 						/*
+ 						 * We don't detect the gross error of new_bits == true when
+ 						 * the typlen actually changed.  attbyval could differ in
+ 						 * theory, but we assume it does not.
+ 						 */
+ 						Assert(tab->new_bits ||
+ 							   (isnull[ex->attnum - 1] == oldisnull
+ 								&& (oldisnull ||
+ 									datumIsEqual(oldval,
+ 												 values[ex->attnum - 1],
+ 												 f->attbyval, f->attlen))));
+ 					}
+ #endif
  				}
  
! 				if (newrel)
! 				{
! 					/*
! 					 * Form the new tuple. Note that we don't explicitly pfree it,
! 					 * since the per-tuple memory context will be reset shortly.
! 					 */
! 					tuple = heap_form_tuple(newTupDesc, values, isnull);
  
! 					/* Preserve OID, if any */
! 					if (newTupDesc->tdhasoid)
! 						HeapTupleSetOid(tuple, tupOid);
! 				}
  			}
  
  			/* Now check any constraints on the possibly-changed tuple */
***************
*** 4283,4288 **** ATExecAddColumn(AlteredTableInfo *tab, Relation rel,
--- 4314,4321 ----
  			newval->expr = defval;
  
  			tab->newvals = lappend(tab->newvals, newval);
+ 			tab->new_bits = true;
+ 			tab->mayerror = true;
  		}
  
  		/*
***************
*** 4299,4305 **** ATExecAddColumn(AlteredTableInfo *tab, Relation rel,
  	 * table to fix that.
  	 */
  	if (isOid)
! 		tab->new_changeoids = true;
  
  	/*
  	 * Add needed dependency entries for the new column.
--- 4332,4338 ----
  	 * table to fix that.
  	 */
  	if (isOid)
! 		tab->new_bits = true;
  
  	/*
  	 * Add needed dependency entries for the new column.
***************
*** 4970,4976 **** ATExecDropColumn(List **wqueue, Relation rel, const char *colName,
  		tab = ATGetQueueEntry(wqueue, rel);
  
  		/* Tell Phase 3 to physically remove the OID column */
! 		tab->new_changeoids = true;
  	}
  }
  
--- 5003,5009 ----
  		tab = ATGetQueueEntry(wqueue, rel);
  
  		/* Tell Phase 3 to physically remove the OID column */
! 		tab->new_bits = true;
  	}
  }
  
***************
*** 4994,5000 **** ATExecAddIndex(AlteredTableInfo *tab, Relation rel,
  	/* suppress schema rights check when rebuilding existing index */
  	check_rights = !is_rebuild;
  	/* skip index build if phase 3 will have to rewrite table anyway */
! 	skip_build = (tab->newvals != NIL);
  	/* suppress notices when rebuilding existing index */
  	quiet = is_rebuild;
  
--- 5027,5033 ----
  	/* suppress schema rights check when rebuilding existing index */
  	check_rights = !is_rebuild;
  	/* skip index build if phase 3 will have to rewrite table anyway */
! 	skip_build = tab->new_bits;
  	/* suppress notices when rebuilding existing index */
  	quiet = is_rebuild;
  
***************
*** 6293,6298 **** ATPrepAlterColumnType(List **wqueue,
--- 6326,6333 ----
  
  	if (tab->relkind == RELKIND_RELATION)
  	{
+ 		CoerceExemptions exempt;
+ 
  		/*
  		 * Set up an expression to transform the old data value to the new type.
  		 * If a USING option was given, transform and use that expression, else
***************
*** 6353,6358 **** ATPrepAlterColumnType(List **wqueue,
--- 6388,6394 ----
  					(errcode(ERRCODE_DATATYPE_MISMATCH),
  					 errmsg("column \"%s\" cannot be cast to type %s",
  							colName, format_type_be(targettype))));
+ 		exempt = GetCoerceExemptions(transform, 1, attnum);
  
  		/*
  		 * Add a work queue item to make ATRewriteTable update the column
***************
*** 6363,6368 **** ATPrepAlterColumnType(List **wqueue,
--- 6399,6423 ----
  		newval->expr = (Expr *) transform;
  
  		tab->newvals = lappend(tab->newvals, newval);
+ 		if (!(exempt & COERCE_EXEMPT_NOCHANGE))
+ 			tab->new_bits = true;
+ 		if (!(exempt & COERCE_EXEMPT_NOERROR))
+ 			tab->mayerror = true;
+ 
+ 		/*
+ 		 * If we need to rewrite the table to update tuple values or scan
+ 		 * it to verify tuple values, tables using this table's rowtype as
+ 		 * a column type would need the same treatment.
+ 		 */
+ 		find_composite_type_dependencies(rel->rd_rel->reltype,
+ 										 RelationGetRelationName(rel),
+ 										 NULL);
+ 	}
+ 	else if (tab->relkind == RELKIND_COMPOSITE_TYPE)
+ 	{
+ 		find_composite_type_dependencies(rel->rd_rel->reltype,
+ 										 NULL,
+ 										 RelationGetRelationName(rel));
  	}
  	else if (tab->relkind == RELKIND_FOREIGN_TABLE)
  	{
***************
*** 6372,6388 **** ATPrepAlterColumnType(List **wqueue,
  					 errmsg("ALTER TYPE USING is not supported on foreign tables")));
  	}
  
- 	if (tab->relkind == RELKIND_COMPOSITE_TYPE)
- 	{
- 		/*
- 		 * For composite types, do this check now.  Tables will check
- 		 * it later when the table is being rewritten.
- 		 */
- 		find_composite_type_dependencies(rel->rd_rel->reltype,
- 										 NULL,
- 										 RelationGetRelationName(rel));
- 	}
- 
  	ReleaseSysCache(tuple);
  
  	/*
--- 6427,6432 ----
*** a/src/backend/parser/parse_coerce.c
--- b/src/backend/parser/parse_coerce.c
***************
*** 19,24 ****
--- 19,25 ----
  #include "catalog/pg_inherits_fn.h"
  #include "catalog/pg_proc.h"
  #include "catalog/pg_type.h"
+ #include "commands/typecmds.h"
  #include "nodes/makefuncs.h"
  #include "nodes/nodeFuncs.h"
  #include "parser/parse_coerce.h"
***************
*** 1805,1810 **** IsBinaryCoercible(Oid srctype, Oid targettype)
--- 1806,1870 ----
  }
  
  
+ /* GetCoerceExemptions()
+  *		Assess invariants of a coercion expression.
+  *
+  * Various common expressions arising from type coercion are subject to
+  * optimizations.  For example, a simple varchar -> text cast will never change
+  * the underlying data (always-noop) and never yield an error (never-error).  A
+  * varchar(8) -> varchar(4) will never change the data, but it may yield an
+  * error.  Given a varno and varattno denoting "the" source datum, determine
+  * which invariants hold for an expression by walking it per these rules:
+  *
+  *	1. A Var with the varno/varattno in question has both invariants.
+  *	2. A RelabelType node inherits the invariants of its sole argument.
+  *	3. A CoerceToDomain node inherits any always-noop invariant from its sole
+  *		argument.  When GetDomainConstraints() == NIL, it also inherits
+  *		never-error.  Otherwise, never-error becomes false.
+  *	4. All other nodes have neither invariant.
+  *
+  * Returns a bit string that may contain the following bits:
+  *	COERCE_EXEMPT_NOCHANGE: expression result will always have the same binary
+  *				representation as a Var expression having the given varno and
+  *				varattno
+  *	COERCE_EXEMPT_NOERROR: expression will never throw an error
+  */
+ CoerceExemptions
+ GetCoerceExemptions(Node *expr,
+ 					Index varno, AttrNumber varattno)
+ {
+ 	CoerceExemptions ret = COERCE_EXEMPT_NOCHANGE | COERCE_EXEMPT_NOERROR;
+ 
+ 	Assert(expr != NULL);
+ 
+ 	for (;;)
+ 	{
+ 		if (IsA(expr, Var)
+ 			&& ((Var *) expr)->varno == varno
+ 			&& ((Var *) expr)->varattno == varattno)
+ 		{
+ 			return ret;
+ 		}
+ 		if (IsA(expr, RelabelType))
+ 		{
+ 			expr = (Node *) ((RelabelType *) expr)->arg;
+ 		}
+ 		else if (IsA(expr, CoerceToDomain))
+ 		{
+ 			CoerceToDomain *d = (CoerceToDomain *) expr;
+ 
+ 			if (GetDomainConstraints(d->resulttype) != NIL)
+ 				ret &= ~COERCE_EXEMPT_NOERROR;
+ 			expr = (Node *) d->arg;
+ 		}
+ 		else
+ 		{
+ 			return 0;
+ 		}
+ 	}
+ }
+ 
+ 
  /*
   * find_coercion_pathway
   *		Look for a coercion pathway between two types.
*** a/src/include/parser/parse_coerce.h
--- b/src/include/parser/parse_coerce.h
***************
*** 30,37 **** typedef enum CoercionPathType
--- 30,44 ----
  	COERCION_PATH_COERCEVIAIO	/* need a CoerceViaIO node */
  } CoercionPathType;
  
+ /* Bits in the return value of GetCoerceExemptions. */
+ typedef int CoerceExemptions;
+ 
+ #define COERCE_EXEMPT_NOCHANGE	0x1		/* expression never changes storage */
+ #define COERCE_EXEMPT_NOERROR	0x2		/* expression never throws an error */
  
  extern bool IsBinaryCoercible(Oid srctype, Oid targettype);
+ extern CoerceExemptions GetCoerceExemptions(Node *expr,
+ 					Index varno, AttrNumber varattno);
  extern bool IsPreferredType(TYPCATEGORY category, Oid type);
  extern TYPCATEGORY TypeCategory(Oid type);
  
*** a/src/test/regress/expected/alter_table.out
--- b/src/test/regress/expected/alter_table.out
***************
*** 1479,1489 **** alter table tab1 alter column b type varchar; -- fails
  ERROR:  cannot alter table "tab1" because column "tab2"."y" uses its rowtype
  alter table tab1 add check (b <> 'foo');
  alter table tab1 add c int not null;
! alter table tab1 add d int not null default 1; -- fails
! ERROR:  cannot alter table "tab1" because column "tab2"."y" uses its rowtype
  alter table tab1 drop a;
! alter table tab1 set with oids;	-- fails
! ERROR:  cannot alter table "tab1" because column "tab2"."y" uses its rowtype
  --
  -- Deeper alter-column-type tests
  --
--- 1479,1487 ----
  ERROR:  cannot alter table "tab1" because column "tab2"."y" uses its rowtype
  alter table tab1 add check (b <> 'foo');
  alter table tab1 add c int not null;
! alter table tab1 add d int not null default 1;
  alter table tab1 drop a;
! alter table tab1 set with oids;
  --
  -- Deeper alter-column-type tests
  --
***************
*** 1754,1798 **** FROM pg_trigger WHERE tgrelid = 't'::regclass ORDER BY tgname;
  -- though mostly not stated here.
  -- Constraint failures induced by a no-work type change.
  ALTER TABLE t ALTER constraint0 TYPE trickint;						-- verify-e
! DEBUG:  Rewriting table "t"
  ERROR:  check constraint "t_constraint0_check" is violated by some row
  ALTER TABLE t ALTER constraint1 TYPE trickint;						-- noop-e
- DEBUG:  Rewriting table "t"
- DEBUG:  Rebuilding index "t_constraint4_key"
- DEBUG:  Rebuilding index "t_integral_key"
- DEBUG:  Rebuilding index "t_rational_key"
- DEBUG:  Rebuilding index "t_daytimetz_key"
- DEBUG:  Rebuilding index "t_daytime_key"
- DEBUG:  Rebuilding index "t_stamptz_key"
- DEBUG:  Rebuilding index "t_stamp_key"
- DEBUG:  Rebuilding index "t_timegap_key"
- DEBUG:  Rebuilding index "t_bits_key"
- DEBUG:  Rebuilding index "t_network_key"
- DEBUG:  Rebuilding index "t_string_idx"
- DEBUG:  Rebuilding index "t_string_idx1"
- DEBUG:  Rebuilding index "t_strarr_idx"
- DEBUG:  Rebuilding index "t_square_idx"
- DEBUG:  Rebuilding index "t_expr_idx"
  DEBUG:  Rebuilding index "t_touchy_f_idx"
  ERROR:  could not create unique index "t_touchy_f_idx"
  DETAIL:  Key (touchy_f(constraint1))=(100) is duplicated.
  ALTER TABLE t ALTER constraint2 TYPE trickint;						-- noop-e
- DEBUG:  Rewriting table "t"
- DEBUG:  Rebuilding index "t_constraint4_key"
- DEBUG:  Rebuilding index "t_integral_key"
- DEBUG:  Rebuilding index "t_rational_key"
- DEBUG:  Rebuilding index "t_daytimetz_key"
- DEBUG:  Rebuilding index "t_daytime_key"
- DEBUG:  Rebuilding index "t_stamptz_key"
- DEBUG:  Rebuilding index "t_stamp_key"
- DEBUG:  Rebuilding index "t_timegap_key"
- DEBUG:  Rebuilding index "t_bits_key"
- DEBUG:  Rebuilding index "t_network_key"
- DEBUG:  Rebuilding index "t_string_idx"
- DEBUG:  Rebuilding index "t_string_idx1"
- DEBUG:  Rebuilding index "t_strarr_idx"
- DEBUG:  Rebuilding index "t_square_idx"
- DEBUG:  Rebuilding index "t_touchy_f_idx"
  DEBUG:  Rebuilding index "t_expr_idx"
  ERROR:  could not create unique index "t_expr_idx"
  DETAIL:  Key ((1))=(1) is duplicated.
--- 1752,1764 ----
  -- though mostly not stated here.
  -- Constraint failures induced by a no-work type change.
  ALTER TABLE t ALTER constraint0 TYPE trickint;						-- verify-e
! DEBUG:  Verifying table "t"
  ERROR:  check constraint "t_constraint0_check" is violated by some row
  ALTER TABLE t ALTER constraint1 TYPE trickint;						-- noop-e
  DEBUG:  Rebuilding index "t_touchy_f_idx"
  ERROR:  could not create unique index "t_touchy_f_idx"
  DETAIL:  Key (touchy_f(constraint1))=(100) is duplicated.
  ALTER TABLE t ALTER constraint2 TYPE trickint;						-- noop-e
  DEBUG:  Rebuilding index "t_expr_idx"
  ERROR:  could not create unique index "t_expr_idx"
  DETAIL:  Key ((1))=(1) is duplicated.
***************
*** 1956,2013 **** DEBUG:  Rebuilding index "t_constraint4_key"
  DEBUG:  Validating foreign key constraint "child_keycol_fkey"
  -- Type-specific tests.
  ALTER TABLE t ALTER integral TYPE abstime USING integral::abstime;	-- noop
- DEBUG:  Rewriting table "t"
- DEBUG:  Rebuilding index "t_rational_key"
- DEBUG:  Rebuilding index "t_daytimetz_key"
- DEBUG:  Rebuilding index "t_daytime_key"
- DEBUG:  Rebuilding index "t_stamptz_key"
- DEBUG:  Rebuilding index "t_stamp_key"
- DEBUG:  Rebuilding index "t_timegap_key"
- DEBUG:  Rebuilding index "t_bits_key"
- DEBUG:  Rebuilding index "t_network_key"
- DEBUG:  Rebuilding index "t_string_idx"
- DEBUG:  Rebuilding index "t_string_idx1"
- DEBUG:  Rebuilding index "t_strarr_idx"
- DEBUG:  Rebuilding index "t_square_idx"
- DEBUG:  Rebuilding index "t_touchy_f_idx"
- DEBUG:  Rebuilding index "t_expr_idx"
- DEBUG:  Rebuilding index "t_constraint4_key"
  DEBUG:  Rebuilding index "t_integral_key"
  ALTER TABLE t ALTER integral TYPE oid USING integral::int4;			-- noop
- DEBUG:  Rewriting table "t"
- DEBUG:  Rebuilding index "t_rational_key"
- DEBUG:  Rebuilding index "t_daytimetz_key"
- DEBUG:  Rebuilding index "t_daytime_key"
- DEBUG:  Rebuilding index "t_stamptz_key"
- DEBUG:  Rebuilding index "t_stamp_key"
- DEBUG:  Rebuilding index "t_timegap_key"
- DEBUG:  Rebuilding index "t_bits_key"
- DEBUG:  Rebuilding index "t_network_key"
- DEBUG:  Rebuilding index "t_string_idx"
- DEBUG:  Rebuilding index "t_string_idx1"
- DEBUG:  Rebuilding index "t_strarr_idx"
- DEBUG:  Rebuilding index "t_square_idx"
- DEBUG:  Rebuilding index "t_touchy_f_idx"
- DEBUG:  Rebuilding index "t_expr_idx"
- DEBUG:  Rebuilding index "t_constraint4_key"
  DEBUG:  Rebuilding index "t_integral_key"
  ALTER TABLE t ALTER integral TYPE regtype;							-- noop
- DEBUG:  Rewriting table "t"
- DEBUG:  Rebuilding index "t_rational_key"
- DEBUG:  Rebuilding index "t_daytimetz_key"
- DEBUG:  Rebuilding index "t_daytime_key"
- DEBUG:  Rebuilding index "t_stamptz_key"
- DEBUG:  Rebuilding index "t_stamp_key"
- DEBUG:  Rebuilding index "t_timegap_key"
- DEBUG:  Rebuilding index "t_bits_key"
- DEBUG:  Rebuilding index "t_network_key"
- DEBUG:  Rebuilding index "t_string_idx"
- DEBUG:  Rebuilding index "t_string_idx1"
- DEBUG:  Rebuilding index "t_strarr_idx"
- DEBUG:  Rebuilding index "t_square_idx"
- DEBUG:  Rebuilding index "t_touchy_f_idx"
- DEBUG:  Rebuilding index "t_expr_idx"
- DEBUG:  Rebuilding index "t_constraint4_key"
  DEBUG:  Rebuilding index "t_integral_key"
  ALTER TABLE t ALTER integral TYPE int8;								-- rewrite
  DEBUG:  Rewriting table "t"
--- 1922,1931 ----
***************
*** 2118,2139 **** DEBUG:  Rebuilding index "t_constraint4_key"
  DEBUG:  Rebuilding index "t_integral_key"
  DEBUG:  Rebuilding index "t_rational_key"
  ALTER TABLE t ALTER rational TYPE numeric;							-- noop
- DEBUG:  Rewriting table "t"
- DEBUG:  Rebuilding index "t_daytimetz_key"
- DEBUG:  Rebuilding index "t_daytime_key"
- DEBUG:  Rebuilding index "t_stamptz_key"
- DEBUG:  Rebuilding index "t_stamp_key"
- DEBUG:  Rebuilding index "t_timegap_key"
- DEBUG:  Rebuilding index "t_bits_key"
- DEBUG:  Rebuilding index "t_network_key"
- DEBUG:  Rebuilding index "t_string_idx"
- DEBUG:  Rebuilding index "t_string_idx1"
- DEBUG:  Rebuilding index "t_strarr_idx"
- DEBUG:  Rebuilding index "t_square_idx"
- DEBUG:  Rebuilding index "t_touchy_f_idx"
- DEBUG:  Rebuilding index "t_expr_idx"
- DEBUG:  Rebuilding index "t_constraint4_key"
- DEBUG:  Rebuilding index "t_integral_key"
  DEBUG:  Rebuilding index "t_rational_key"
  ALTER TABLE t ALTER rational TYPE numeric(5,4);						-- verify-e
  DEBUG:  Rewriting table "t"
--- 2036,2041 ----
***************
*** 2183,2242 **** ALTER TABLE t ALTER string TYPE shortdom;							-- rewrite-e
  DEBUG:  Rewriting table "t"
  ERROR:  value too long for type character varying(1)
  ALTER TABLE t ALTER string TYPE checkdom;							-- verify
- DEBUG:  Rewriting table "t"
- DEBUG:  Rebuilding index "t_daytimetz_key"
- DEBUG:  Rebuilding index "t_daytime_key"
- DEBUG:  Rebuilding index "t_stamptz_key"
- DEBUG:  Rebuilding index "t_stamp_key"
- DEBUG:  Rebuilding index "t_timegap_key"
- DEBUG:  Rebuilding index "t_bits_key"
- DEBUG:  Rebuilding index "t_network_key"
- DEBUG:  Rebuilding index "t_strarr_idx"
- DEBUG:  Rebuilding index "t_square_idx"
- DEBUG:  Rebuilding index "t_touchy_f_idx"
- DEBUG:  Rebuilding index "t_expr_idx"
- DEBUG:  Rebuilding index "t_constraint4_key"
- DEBUG:  Rebuilding index "t_integral_key"
- DEBUG:  Rebuilding index "t_rational_key"
  DEBUG:  Rebuilding index "t_string_idx1"
  DEBUG:  Rebuilding index "t_string_idx"
  ALTER TABLE t ALTER string TYPE faildom;							-- verify-e
! DEBUG:  Rewriting table "t"
  ERROR:  value for domain faildom violates check constraint "faildom_check"
  ALTER TABLE t ALTER string TYPE loosedom;							-- noop
- DEBUG:  Rewriting table "t"
- DEBUG:  Rebuilding index "t_daytimetz_key"
- DEBUG:  Rebuilding index "t_daytime_key"
- DEBUG:  Rebuilding index "t_stamptz_key"
- DEBUG:  Rebuilding index "t_stamp_key"
- DEBUG:  Rebuilding index "t_timegap_key"
- DEBUG:  Rebuilding index "t_bits_key"
- DEBUG:  Rebuilding index "t_network_key"
- DEBUG:  Rebuilding index "t_strarr_idx"
- DEBUG:  Rebuilding index "t_square_idx"
- DEBUG:  Rebuilding index "t_touchy_f_idx"
- DEBUG:  Rebuilding index "t_expr_idx"
- DEBUG:  Rebuilding index "t_constraint4_key"
- DEBUG:  Rebuilding index "t_integral_key"
- DEBUG:  Rebuilding index "t_rational_key"
  DEBUG:  Rebuilding index "t_string_idx"
  DEBUG:  Rebuilding index "t_string_idx1"
  ALTER TABLE t ALTER string TYPE text;								-- noop
- DEBUG:  Rewriting table "t"
- DEBUG:  Rebuilding index "t_daytimetz_key"
- DEBUG:  Rebuilding index "t_daytime_key"
- DEBUG:  Rebuilding index "t_stamptz_key"
- DEBUG:  Rebuilding index "t_stamp_key"
- DEBUG:  Rebuilding index "t_timegap_key"
- DEBUG:  Rebuilding index "t_bits_key"
- DEBUG:  Rebuilding index "t_network_key"
- DEBUG:  Rebuilding index "t_strarr_idx"
- DEBUG:  Rebuilding index "t_square_idx"
- DEBUG:  Rebuilding index "t_touchy_f_idx"
- DEBUG:  Rebuilding index "t_expr_idx"
- DEBUG:  Rebuilding index "t_constraint4_key"
- DEBUG:  Rebuilding index "t_integral_key"
- DEBUG:  Rebuilding index "t_rational_key"
  DEBUG:  Rebuilding index "t_string_idx1"
  DEBUG:  Rebuilding index "t_string_idx"
  ALTER TABLE t ALTER string TYPE varchar(20);						-- rewrite-v
--- 2085,2102 ----
  DEBUG:  Rewriting table "t"
  ERROR:  value too long for type character varying(1)
  ALTER TABLE t ALTER string TYPE checkdom;							-- verify
  DEBUG:  Rebuilding index "t_string_idx1"
  DEBUG:  Rebuilding index "t_string_idx"
+ DEBUG:  Verifying table "t"
  ALTER TABLE t ALTER string TYPE faildom;							-- verify-e
! DEBUG:  Rebuilding index "t_string_idx"
! DEBUG:  Rebuilding index "t_string_idx1"
! DEBUG:  Verifying table "t"
  ERROR:  value for domain faildom violates check constraint "faildom_check"
  ALTER TABLE t ALTER string TYPE loosedom;							-- noop
  DEBUG:  Rebuilding index "t_string_idx"
  DEBUG:  Rebuilding index "t_string_idx1"
  ALTER TABLE t ALTER string TYPE text;								-- noop
  DEBUG:  Rebuilding index "t_string_idx1"
  DEBUG:  Rebuilding index "t_string_idx"
  ALTER TABLE t ALTER string TYPE varchar(20);						-- rewrite-v
***************
*** 2851,2872 **** DEBUG:  Rebuilding index "t_stamp_key"
  DEBUG:  Rebuilding index "t_timegap_key"
  DEBUG:  Rebuilding index "t_bits_key"
  ALTER TABLE t ALTER network TYPE inet;								-- noop
- DEBUG:  Rewriting table "t"
- DEBUG:  Rebuilding index "t_strarr_idx"
- DEBUG:  Rebuilding index "t_square_idx"
- DEBUG:  Rebuilding index "t_touchy_f_idx"
- DEBUG:  Rebuilding index "t_expr_idx"
- DEBUG:  Rebuilding index "t_constraint4_key"
- DEBUG:  Rebuilding index "t_integral_key"
- DEBUG:  Rebuilding index "t_rational_key"
- DEBUG:  Rebuilding index "t_string_idx1"
- DEBUG:  Rebuilding index "t_string_idx"
- DEBUG:  Rebuilding index "t_daytimetz_key"
- DEBUG:  Rebuilding index "t_daytime_key"
- DEBUG:  Rebuilding index "t_stamptz_key"
- DEBUG:  Rebuilding index "t_stamp_key"
- DEBUG:  Rebuilding index "t_timegap_key"
- DEBUG:  Rebuilding index "t_bits_key"
  DEBUG:  Rebuilding index "t_network_key"
  ALTER TABLE t ALTER network TYPE cidr;								-- rewrite-v
  DEBUG:  Rewriting table "t"
--- 2711,2716 ----
***************
*** 2887,2909 **** DEBUG:  Rebuilding index "t_timegap_key"
  DEBUG:  Rebuilding index "t_bits_key"
  DEBUG:  Rebuilding index "t_network_key"
  ALTER TABLE t ALTER document TYPE text;								-- noop
- DEBUG:  Rewriting table "t"
- DEBUG:  Rebuilding index "t_strarr_idx"
- DEBUG:  Rebuilding index "t_square_idx"
- DEBUG:  Rebuilding index "t_touchy_f_idx"
- DEBUG:  Rebuilding index "t_expr_idx"
- DEBUG:  Rebuilding index "t_constraint4_key"
- DEBUG:  Rebuilding index "t_integral_key"
- DEBUG:  Rebuilding index "t_rational_key"
- DEBUG:  Rebuilding index "t_string_idx1"
- DEBUG:  Rebuilding index "t_string_idx"
- DEBUG:  Rebuilding index "t_daytimetz_key"
- DEBUG:  Rebuilding index "t_daytime_key"
- DEBUG:  Rebuilding index "t_stamptz_key"
- DEBUG:  Rebuilding index "t_stamp_key"
- DEBUG:  Rebuilding index "t_timegap_key"
- DEBUG:  Rebuilding index "t_bits_key"
- DEBUG:  Rebuilding index "t_network_key"
  ALTER TABLE t ALTER document TYPE xml USING document::xml;			-- verify
  DEBUG:  Rewriting table "t"
  DEBUG:  Rebuilding index "t_strarr_idx"
--- 2731,2736 ----
*** a/src/test/regress/expected/rowtypes.out
--- b/src/test/regress/expected/rowtypes.out
***************
*** 82,91 **** select * from people;
   (Joe,Blow) | 01-10-1984
  (1 row)
  
! -- at the moment this will not work due to ALTER TABLE inadequacy:
  alter table fullname add column suffix text default '';
! ERROR:  cannot alter table "fullname" because column "people"."fn" uses its rowtype
! -- but this should work:
  alter table fullname add column suffix text default null;
  select * from people;
       fn      |     bd     
--- 82,91 ----
   (Joe,Blow) | 01-10-1984
  (1 row)
  
! -- accepted, but the default does not propagate
  alter table fullname add column suffix text default '';
! alter table fullname drop column suffix;
! -- same effect
  alter table fullname add column suffix text default null;
  select * from people;
       fn      |     bd     
*** a/src/test/regress/sql/alter_table.sql
--- b/src/test/regress/sql/alter_table.sql
***************
*** 1096,1104 **** create table tab2 (x int, y tab1);
  alter table tab1 alter column b type varchar; -- fails
  alter table tab1 add check (b <> 'foo');
  alter table tab1 add c int not null;
! alter table tab1 add d int not null default 1; -- fails
  alter table tab1 drop a;
! alter table tab1 set with oids;	-- fails
  
  --
  -- Deeper alter-column-type tests
--- 1096,1104 ----
  alter table tab1 alter column b type varchar; -- fails
  alter table tab1 add check (b <> 'foo');
  alter table tab1 add c int not null;
! alter table tab1 add d int not null default 1;
  alter table tab1 drop a;
! alter table tab1 set with oids;
  
  --
  -- Deeper alter-column-type tests
*** a/src/test/regress/sql/rowtypes.sql
--- b/src/test/regress/sql/rowtypes.sql
***************
*** 45,54 **** insert into people values ('(Joe,Blow)', '1984-01-10');
  
  select * from people;
  
! -- at the moment this will not work due to ALTER TABLE inadequacy:
  alter table fullname add column suffix text default '';
  
! -- but this should work:
  alter table fullname add column suffix text default null;
  
  select * from people;
--- 45,55 ----
  
  select * from people;
  
! -- accepted, but the default does not propagate
  alter table fullname add column suffix text default '';
+ alter table fullname drop column suffix;
  
! -- same effect
  alter table fullname add column suffix text default null;
  
  select * from people;
#2Robert Haas
robertmhaas@gmail.com
In reply to: Noah Misch (#1)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Sun, Jan 9, 2011 at 5:01 PM, Noah Misch <noah@leadboat.com> wrote:

This patch removes ALTER TYPE rewrites in cases we can already prove valid.  I
add a function GetCoerceExemptions() that walks an Expr according to rules
discussed in the design thread, simplified slightly pending additions in the
next patch.  See the comment at that function for a refresher.  I use it to
populate two new bools to AlteredTableInfo, "new_bits" and "mayerror".
"new_bits" is a superset of "new_changedoids", so I subsume that.  I change
ATRewriteTable to act on those and support the notion of evaluating the
transformation expressions when we're not rewriting the table.

This helps on conversions like varchar(X)->text, xml->text, and conversions
between domains and their base types.

This certainly looks like a worthwhile thing to do, and it doesn't
seem to need a lot of code, which is great. But I confess I'm not
confident I really understand what this patch is changing or why it's
changing it. I think the problem is partly that the terminology used
is not very consistent:

+ 		if (!(exempt & COERCE_EXEMPT_NOCHANGE))
+ 			tab->new_bits = true;
+ 		if (!(exempt & COERCE_EXEMPT_NOERROR))
+ 			tab->mayerror = true;

These are the same two bits of status that are elsewhere called
always-noop and never-error. Or maybe not quite the same... but
close. A related problem is that I think only three of the four
combinations are actually interesting: if there are new bits... that
is, if !COERCE_EXEMPT_NOCHANGE... i.e. not always-noop, then the state
of the other bit is irrelevant. I think maybe this ought to just be
rephrased as an enum with three elements, describing the operation to
be performed: rewrite, check, nothing.

As unintended fallout, it's no longer an error to add oids or a column with a
default value to a table whose rowtype is used in columns elsewhere. This seems
best. Defaults on the origin table do not even apply to new inserts into such a
column, and the rowtype does not gain an OID column via its table.

I think this should be split into two patches (we can discuss both on
this thread, no need to start any new ones), one that implements just
the above improvement and another that accomplishes the main purpose
of the patch. Patches that do two or three or four things are quite a
bit harder to understand than patches that just do one thing.

On a related note, it is very helpful to avoid introducing unrelated
changes into a patch. Comment updates should reflect changes you
made, rather than editorialization on what's already there. There is
some value to the latter, but it makes it harder to understand what
the patch is doing.

Also, you need to update the ALTER TABLE documentation. The whole
notes section needs to be gone over, but the following part in
particular seems problematic, since we're proposing to break this:

# The fact that ALTER TYPE requires rewriting the whole table is
sometimes an advantage, because the rewriting process
# eliminates any dead space in the table. For example, to reclaim the
space occupied by a dropped column immediately, the
# fastest way is:
#
# ALTER TABLE table ALTER COLUMN anycol TYPE anytype;

I'm not sure what we can recommend here instead.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#3Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#2)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

Robert Haas <robertmhaas@gmail.com> writes:

On Sun, Jan 9, 2011 at 5:01 PM, Noah Misch <noah@leadboat.com> wrote:

As unintended fallout, it's no longer an error to add oids or a column with a
default value to a table whose rowtype is used in columns elsewhere. This seems
best. Defaults on the origin table do not even apply to new inserts into such a
column, and the rowtype does not gain an OID column via its table.

I think this should be split into two patches (we can discuss both on
this thread, no need to start any new ones), one that implements just
the above improvement and another that accomplishes the main purpose
of the patch.

I haven't been paying much attention to this thread, but I happened to
read the above, and quite frankly it scares the cr*p out of me. I don't
believe that Noah even begins to be qualified to understand the
implications of adding/removing an OID column, and I clearly remember
the non-obvious bugs we've had in the past from that. Before this goes
in I want to see a convincing explanation (not an unsupported assertion)
why this isn't going to re-introduce those old bugs.

I'm also pretty unclear on why it's a good idea to let uses of a rowtype
be different from the table on which it's allegedly based. To the
extent that the current behavior allows that, isn't that a bug rather
than a feature we should extend?

# The fact that ALTER TYPE requires rewriting the whole table is
sometimes an advantage, because the rewriting process
# eliminates any dead space in the table.

I'm not sure what we can recommend here instead.

New-style VACUUM FULL. I don't think that a patch that makes it harder
to use ALTER TABLE this way is a problem in itself, now that we've got
that.

regards, tom lane

#4Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#3)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Sun, Jan 23, 2011 at 12:06 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Robert Haas <robertmhaas@gmail.com> writes:

On Sun, Jan 9, 2011 at 5:01 PM, Noah Misch <noah@leadboat.com> wrote:

As unintended fallout, it's no longer an error to add oids or a column with a
default value to a table whose rowtype is used in columns elsewhere.  This seems
best.  Defaults on the origin table do not even apply to new inserts into such a
column, and the rowtype does not gain an OID column via its table.

I think this should be split into two patches (we can discuss both on
this thread, no need to start any new ones), one that implements just
the above improvement and another that accomplishes the main purpose
of the patch.

I haven't been paying much attention to this thread, but I happened to
read the above, and quite frankly it scares the cr*p out of me.  I don't
believe that Noah even begins to be qualified to understand the
implications of adding/removing an OID column, and I clearly remember
the non-obvious bugs we've had in the past from that.  Before this goes
in I want to see a convincing explanation (not an unsupported assertion)
why this isn't going to re-introduce those old bugs.

Because all of our old bugs now have regression tests that will break
if we reintroduce them?

I guess that probably falls into the category of "wishful thinking".

I'm also pretty unclear on why it's a good idea to let uses of a rowtype
be different from the table on which it's allegedly based.  To the
extent that the current behavior allows that, isn't that a bug rather
than a feature we should extend?

It's not clear to me what it would mean for OIDs or default values to
propagate themselves to the table's row type. Check constraints,
foreign keys, unique constraints, etc. don't either. In fact, not
even the NOT NULL property flows through. On the surface, preventing
these details from interfering with ALTER TABLE commands that can't
actually break anything seems like removing an annoying implementation
restriction, but I guess one could make the argument that we actually
intend to make those flow through some day. But if so, this is the
first I'm hearing of it.

# The fact that ALTER TYPE requires rewriting the whole table is
sometimes an advantage, because the rewriting process
# eliminates any dead space in the table.

I'm not sure what we can recommend here instead.

New-style VACUUM FULL.  I don't think that a patch that makes it harder
to use ALTER TABLE this way is a problem in itself, now that we've got
that.

Cool. That'll reclaim space from dropped columns and stuff?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#5Noah Misch
noah@leadboat.com
In reply to: Tom Lane (#3)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Sun, Jan 23, 2011 at 12:06:43PM -0500, Tom Lane wrote:

Robert Haas <robertmhaas@gmail.com> writes:

On Sun, Jan 9, 2011 at 5:01 PM, Noah Misch <noah@leadboat.com> wrote:

As unintended fallout, it's no longer an error to add oids or a column with a
default value to a table whose rowtype is used in columns elsewhere. This seems
best. Defaults on the origin table do not even apply to new inserts into such a
column, and the rowtype does not gain an OID column via its table.

I think this should be split into two patches (we can discuss both on
this thread, no need to start any new ones), one that implements just
the above improvement and another that accomplishes the main purpose
of the patch.

I haven't been paying much attention to this thread, but I happened to
read the above, and quite frankly it scares the cr*p out of me. I don't
believe that Noah even begins to be qualified to understand the
implications of adding/removing an OID column, and I clearly remember
the non-obvious bugs we've had in the past from that. Before this goes
in I want to see a convincing explanation (not an unsupported assertion)
why this isn't going to re-introduce those old bugs.

Turns out that we do set HEAP_HASOID and allocate space for an OID in the
composite-type datums. We don't actually assign an OID, and I can't see any way
to read it from the composite. It seems most consistent with the verdict of
commit 6d1e361 to continue rejecting these cases. Thanks for the heads-up.

I'm also pretty unclear on why it's a good idea to let uses of a rowtype
be different from the table on which it's allegedly based. To the
extent that the current behavior allows that, isn't that a bug rather
than a feature we should extend?

From the perspective of defining the behavior afresh, I'd agree. I haven't seen
any stirrings of actually implementing this, though. Like Robert, I'm doubting
there's a user benefit from rejecting these cases in preparation for the day
that they would actually require it.

nm

#6Noah Misch
noah@leadboat.com
In reply to: Robert Haas (#2)
5 attachment(s)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

Robert,

Thanks for the review.

On Sat, Jan 22, 2011 at 11:28:48PM -0500, Robert Haas wrote:

This certainly looks like a worthwhile thing to do, and it doesn't
seem to need a lot of code, which is great. But I confess I'm not
confident I really understand what this patch is changing or why it's
changing it.

I think the problem is partly that the terminology used
is not very consistent:

+ 		if (!(exempt & COERCE_EXEMPT_NOCHANGE))
+ 			tab->new_bits = true;
+ 		if (!(exempt & COERCE_EXEMPT_NOERROR))
+ 			tab->mayerror = true;

These are the same two bits of status that are elsewhere called
always-noop and never-error. Or maybe not quite the same... but
close. A related problem is that I think only three of the four
combinations are actually interesting: if there are new bits... that
is, if !COERCE_EXEMPT_NOCHANGE... i.e. not always-noop, then the state
of the other bit is irrelevant. I think maybe this ought to just be
rephrased as an enum with three elements, describing the operation to
be performed: rewrite, check, nothing.

I've fixed the GetCoerceExemptions header comments to follow the #define'd
names. You are correct that only three of the four possibilities are distinct
for ALTER TABLE purposes. I've adopted the enum in tablecmds.c.

As unintended fallout, it's no longer an error to add oids or a column with a
default value to a table whose rowtype is used in columns elsewhere. This seems
best. Defaults on the origin table do not even apply to new inserts into such a
column, and the rowtype does not gain an OID column via its table.

I think this should be split into two patches (we can discuss both on
this thread, no need to start any new ones), one that implements just
the above improvement and another that accomplishes the main purpose
of the patch. Patches that do two or three or four things are quite a
bit harder to understand than patches that just do one thing.

Sounds good; done.

Also, you need to update the ALTER TABLE documentation. The whole
notes section needs to be gone over, but the following part in
particular seems problematic, since we're proposing to break this:

Done.

I'm attaching four patches:

* at1.1-default-composite.patch
Remove the error when the user adds a column with a default value to a table
whose rowtype is used in a column elsewhere.
* at1.2-doc-set-data-type.patch
The documentation used "ALTER TYPE" when it meant "SET DATA TYPE", a subform of
"ALTER TABLE" or "ALTER FOREIGN TABLE". Fixes just that.
* at1.3-tablecmds-enum.patch
Implements your suggestion of using an enum to represent the choice between
rewriting, scanning, and doing nothing. No functional changes. Most of this
patch is re-indentation, so I'm also attaching "at1.3w-tablecmds-enum.patch",
the same change under "git diff -w". I reflowed the comment blocks that became
too wide, but I did not reflow the ones that now have more width available.
* at2v2-skip-nowork.patch
The remainder of the original patch, with the updates noted above.

nm

Attachments:

at1.1-default-composite.patchtext/plain; charset=us-asciiDownload
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 77ea9df..78262d8 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 3366,3384 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  	}
  
  	/*
- 	 * If we need to rewrite the table, the operation has to be propagated to
- 	 * tables that use this table's rowtype as a column type.
- 	 *
- 	 * (Eventually this will probably become true for scans as well, but at
- 	 * the moment a composite type does not enforce any constraints, so it's
- 	 * not necessary/appropriate to enforce them just during ALTER.)
- 	 */
- 	if (newrel)
- 		find_composite_type_dependencies(oldrel->rd_rel->reltype,
- 										 RelationGetRelationName(oldrel),
- 										 NULL);
- 
- 	/*
  	 * Generate the constraint and default execution states
  	 */
  
--- 3366,3371 ----
***************
*** 4300,4307 **** ATExecAddColumn(AlteredTableInfo *tab, Relation rel,
--- 4287,4301 ----
  	 * table to fix that.
  	 */
  	if (isOid)
+ 	{
  		tab->new_changeoids = true;
  
+ 		/* See comments in ATExecDropColumn. */
+ 		find_composite_type_dependencies(rel->rd_rel->reltype,
+ 										 RelationGetRelationName(rel),
+ 										 NULL);
+ 	}
+ 
  	/*
  	 * Add needed dependency entries for the new column.
  	 */
***************
*** 4942,4948 **** ATExecDropColumn(List **wqueue, Relation rel, const char *colName,
  
  	/*
  	 * If we dropped the OID column, must adjust pg_class.relhasoids and tell
! 	 * Phase 3 to physically get rid of the column.
  	 */
  	if (attnum == ObjectIdAttributeNumber)
  	{
--- 4936,4944 ----
  
  	/*
  	 * If we dropped the OID column, must adjust pg_class.relhasoids and tell
! 	 * Phase 3 to physically get rid of the column.  We formerly left the column
! 	 * in place physically, and this caused subtle problems:
! 	 * http://archives.postgresql.org/message-id/20594.1234109627@sss.pgh.pa.us
  	 */
  	if (attnum == ObjectIdAttributeNumber)
  	{
***************
*** 4972,4977 **** ATExecDropColumn(List **wqueue, Relation rel, const char *colName,
--- 4968,4982 ----
  
  		/* Tell Phase 3 to physically remove the OID column */
  		tab->new_changeoids = true;
+ 
+ 		/*
+ 		 * We would also need to rewrite all tables using this table's rowtype
+ 		 * as a column type.  The OID column is not visible to or populated for
+ 		 * composite type users, but we do set HEAP_HASOID and reserve space.
+ 		 */
+ 		find_composite_type_dependencies(rel->rd_rel->reltype,
+ 										 RelationGetRelationName(rel),
+ 										 NULL);
  	}
  }
  
***************
*** 6364,6369 **** ATPrepAlterColumnType(List **wqueue,
--- 6369,6388 ----
  		newval->expr = (Expr *) transform;
  
  		tab->newvals = lappend(tab->newvals, newval);
+ 
+ 		/*
+ 		 * If we need to rewrite or scan this table, tables using its rowtype as
+ 		 * a column type would need the same treatment.
+ 		 */
+ 		find_composite_type_dependencies(rel->rd_rel->reltype,
+ 										 RelationGetRelationName(rel),
+ 										 NULL);
+ 	}
+ 	else if (tab->relkind == RELKIND_COMPOSITE_TYPE)
+ 	{
+ 		find_composite_type_dependencies(rel->rd_rel->reltype,
+ 										 NULL,
+ 										 RelationGetRelationName(rel));
  	}
  	else if (tab->relkind == RELKIND_FOREIGN_TABLE)
  	{
***************
*** 6373,6389 **** ATPrepAlterColumnType(List **wqueue,
  					 errmsg("ALTER TYPE USING is not supported on foreign tables")));
  	}
  
- 	if (tab->relkind == RELKIND_COMPOSITE_TYPE)
- 	{
- 		/*
- 		 * For composite types, do this check now.  Tables will check
- 		 * it later when the table is being rewritten.
- 		 */
- 		find_composite_type_dependencies(rel->rd_rel->reltype,
- 										 NULL,
- 										 RelationGetRelationName(rel));
- 	}
- 
  	ReleaseSysCache(tuple);
  
  	/*
--- 6392,6397 ----
diff --git a/src/test/regress/expected/index 6fc9db8..f2cfc07 100644
*** a/src/test/regress/expected/alter_table.out
--- b/src/test/regress/expected/alter_table.out
***************
*** 1483,1490 **** alter table tab1 alter column b type varchar; -- fails
  ERROR:  cannot alter table "tab1" because column "tab2"."y" uses its rowtype
  alter table tab1 add check (b <> 'foo');
  alter table tab1 add c int not null;
! alter table tab1 add d int not null default 1; -- fails
! ERROR:  cannot alter table "tab1" because column "tab2"."y" uses its rowtype
  alter table tab1 drop a;
  alter table tab1 set with oids;	-- fails
  ERROR:  cannot alter table "tab1" because column "tab2"."y" uses its rowtype
--- 1483,1489 ----
  ERROR:  cannot alter table "tab1" because column "tab2"."y" uses its rowtype
  alter table tab1 add check (b <> 'foo');
  alter table tab1 add c int not null;
! alter table tab1 add d int not null default 1;
  alter table tab1 drop a;
  alter table tab1 set with oids;	-- fails
  ERROR:  cannot alter table "tab1" because column "tab2"."y" uses its rowtype
diff --git a/src/test/regress/expected/rowtypes.index e5cd714..707ca38 100644
*** a/src/test/regress/expected/rowtypes.out
--- b/src/test/regress/expected/rowtypes.out
***************
*** 82,91 **** select * from people;
   (Joe,Blow) | 01-10-1984
  (1 row)
  
! -- at the moment this will not work due to ALTER TABLE inadequacy:
  alter table fullname add column suffix text default '';
! ERROR:  cannot alter table "fullname" because column "people"."fn" uses its rowtype
! -- but this should work:
  alter table fullname add column suffix text default null;
  select * from people;
       fn      |     bd     
--- 82,91 ----
   (Joe,Blow) | 01-10-1984
  (1 row)
  
! -- accepted, but the default does not propagate
  alter table fullname add column suffix text default '';
! alter table fullname drop column suffix;
! -- same effect
  alter table fullname add column suffix text default null;
  select * from people;
       fn      |     bd     
diff --git a/src/test/regress/sql/alter_tableindex ed857cf..1818010 100644
*** a/src/test/regress/sql/alter_table.sql
--- b/src/test/regress/sql/alter_table.sql
***************
*** 1098,1104 **** create table tab2 (x int, y tab1);
  alter table tab1 alter column b type varchar; -- fails
  alter table tab1 add check (b <> 'foo');
  alter table tab1 add c int not null;
! alter table tab1 add d int not null default 1; -- fails
  alter table tab1 drop a;
  alter table tab1 set with oids;	-- fails
  
--- 1098,1104 ----
  alter table tab1 alter column b type varchar; -- fails
  alter table tab1 add check (b <> 'foo');
  alter table tab1 add c int not null;
! alter table tab1 add d int not null default 1;
  alter table tab1 drop a;
  alter table tab1 set with oids;	-- fails
  
diff --git a/src/test/regress/sql/rowtypes.index 9041df1..839cd7c 100644
*** a/src/test/regress/sql/rowtypes.sql
--- b/src/test/regress/sql/rowtypes.sql
***************
*** 45,54 **** insert into people values ('(Joe,Blow)', '1984-01-10');
  
  select * from people;
  
! -- at the moment this will not work due to ALTER TABLE inadequacy:
  alter table fullname add column suffix text default '';
  
! -- but this should work:
  alter table fullname add column suffix text default null;
  
  select * from people;
--- 45,55 ----
  
  select * from people;
  
! -- accepted, but the default does not propagate
  alter table fullname add column suffix text default '';
+ alter table fullname drop column suffix;
  
! -- same effect
  alter table fullname add column suffix text default null;
  
  select * from people;
at1.2-doc-set-data-type.patchtext/plain; charset=us-asciiDownload
diff --git a/doc/src/sgml/ref/alter_foreign_table.sgml b/doc/src/sgml/ref/alter_foreign_table.sgml
index 9d14b19..c2ebdac 100644
*** a/doc/src/sgml/ref/alter_foreign_table.sgml
--- b/doc/src/sgml/ref/alter_foreign_table.sgml
***************
*** 264,270 **** ALTER FOREIGN TABLE <replaceable class="PARAMETER">name</replaceable>
      added or removed with <literal>ADD COLUMN</literal> or
      <literal>DROP COLUMN</literal>, a system <literal>oid</> column is added
      or removed, a <literal>CHECK</> or <literal>NOT NULL</> constraint is
!     added, or column type is changed with <literal>ALTER TYPE</>.  It is the
      user's responsibility to ensure that the table definition matches the
      remote side.
     </para>
--- 264,270 ----
      added or removed with <literal>ADD COLUMN</literal> or
      <literal>DROP COLUMN</literal>, a system <literal>oid</> column is added
      or removed, a <literal>CHECK</> or <literal>NOT NULL</> constraint is
!     added, or column type is changed with <literal>SET DATA TYPE</>.  It is the
      user's responsibility to ensure that the table definition matches the
      remote side.
     </para>
diff --git a/doc/src/sgml/ref/alter_table.sgml bindex 17a1d34..5c94a9f 100644
*** a/doc/src/sgml/ref/alter_table.sgml
--- b/doc/src/sgml/ref/alter_table.sgml
***************
*** 719,725 **** ALTER TABLE <replaceable class="PARAMETER">name</replaceable>
     </para>
  
     <para>
!     The fact that <literal>ALTER TYPE</> requires rewriting the whole table
      is sometimes an advantage, because the rewriting process eliminates
      any dead space in the table.  For example, to reclaim the space occupied
      by a dropped column immediately, the fastest way is:
--- 719,725 ----
     </para>
  
     <para>
!     The fact that <literal>SET DATA TYPE</> requires rewriting the whole table
      is sometimes an advantage, because the rewriting process eliminates
      any dead space in the table.  For example, to reclaim the space occupied
      by a dropped column immediately, the fastest way is:
***************
*** 734,748 **** ALTER TABLE table ALTER COLUMN anycol TYPE anytype;
     </para>
  
     <para>
!     The <literal>USING</literal> option of <literal>ALTER TYPE</> can actually
      specify any expression involving the old values of the row; that is, it
      can refer to other columns as well as the one being converted.  This allows
!     very general conversions to be done with the <literal>ALTER TYPE</>
      syntax.  Because of this flexibility, the <literal>USING</literal>
      expression is not applied to the column's default value (if any); the
      result might not be a constant expression as required for a default.
      This means that when there is no implicit or assignment cast from old to
!     new type, <literal>ALTER TYPE</> might fail to convert the default even
      though a <literal>USING</literal> clause is supplied.  In such cases,
      drop the default with <literal>DROP DEFAULT</>, perform the <literal>ALTER
      TYPE</>, and then use <literal>SET DEFAULT</> to add a suitable new
--- 734,748 ----
     </para>
  
     <para>
!     The <literal>USING</literal> option of <literal>SET DATA TYPE</> can actually
      specify any expression involving the old values of the row; that is, it
      can refer to other columns as well as the one being converted.  This allows
!     very general conversions to be done with the <literal>SET DATA TYPE</>
      syntax.  Because of this flexibility, the <literal>USING</literal>
      expression is not applied to the column's default value (if any); the
      result might not be a constant expression as required for a default.
      This means that when there is no implicit or assignment cast from old to
!     new type, <literal>SET DATA TYPE</> might fail to convert the default even
      though a <literal>USING</literal> clause is supplied.  In such cases,
      drop the default with <literal>DROP DEFAULT</>, perform the <literal>ALTER
      TYPE</>, and then use <literal>SET DEFAULT</> to add a suitable new
at1.3-tablecmds-enum.patchtext/plain; charset=us-asciiDownload
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 78262d8..63e2538 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 129,134 **** static List *on_commits = NIL;
--- 129,137 ----
  #define AT_PASS_MISC			8		/* other stuff */
  #define AT_NUM_PASSES			9
  
+ /* Level of effort required in Phase 3 (ATRewriteTables). */
+ typedef enum { WORK_NONE = 0, WORK_SCAN = 1, WORK_REWRITE = 2 } WorkLevel;
+ 
  typedef struct AlteredTableInfo
  {
  	/* Information saved before any work commences: */
***************
*** 141,147 **** typedef struct AlteredTableInfo
  	List	   *constraints;	/* List of NewConstraint */
  	List	   *newvals;		/* List of NewColumnValue */
  	bool		new_notnull;	/* T if we added new NOT NULL constraints */
! 	bool		new_changeoids; /* T if we added/dropped the OID column */
  	Oid			newTableSpace;	/* new tablespace; 0 means no change */
  	/* Objects to rebuild after completing ALTER TYPE operations */
  	List	   *changedConstraintOids;	/* OIDs of constraints to rebuild */
--- 144,150 ----
  	List	   *constraints;	/* List of NewConstraint */
  	List	   *newvals;		/* List of NewColumnValue */
  	bool		new_notnull;	/* T if we added new NOT NULL constraints */
! 	WorkLevel	worklevel;		/* How much work shall we do on the heap? */
  	Oid			newTableSpace;	/* new tablespace; 0 means no change */
  	/* Objects to rebuild after completing ALTER TYPE operations */
  	List	   *changedConstraintOids;	/* OIDs of constraints to rebuild */
***************
*** 3180,3265 **** ATRewriteTables(List **wqueue, LOCKMODE lockmode)
  		if (tab->relkind == RELKIND_FOREIGN_TABLE)
  			continue;
  
! 		/*
! 		 * We only need to rewrite the table if at least one column needs to
! 		 * be recomputed, or we are adding/removing the OID column.
! 		 */
! 		if (tab->newvals != NIL || tab->new_changeoids)
  		{
! 			/* Build a temporary relation and copy data */
! 			Relation	OldHeap;
! 			Oid			OIDNewHeap;
! 			Oid			NewTableSpace;
  
! 			OldHeap = heap_open(tab->relid, NoLock);
  
! 			/*
! 			 * We don't support rewriting of system catalogs; there are too
! 			 * many corner cases and too little benefit.  In particular this
! 			 * is certainly not going to work for mapped catalogs.
! 			 */
! 			if (IsSystemRelation(OldHeap))
! 				ereport(ERROR,
! 						(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 						 errmsg("cannot rewrite system relation \"%s\"",
! 								RelationGetRelationName(OldHeap))));
  
! 			/*
! 			 * Don't allow rewrite on temp tables of other backends ... their
! 			 * local buffer manager is not going to cope.
! 			 */
! 			if (RELATION_IS_OTHER_TEMP(OldHeap))
! 				ereport(ERROR,
! 						(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 				errmsg("cannot rewrite temporary tables of other sessions")));
  
! 			/*
! 			 * Select destination tablespace (same as original unless user
! 			 * requested a change)
! 			 */
! 			if (tab->newTableSpace)
! 				NewTableSpace = tab->newTableSpace;
! 			else
! 				NewTableSpace = OldHeap->rd_rel->reltablespace;
  
! 			heap_close(OldHeap, NoLock);
  
! 			/* Create transient table that will receive the modified data */
! 			OIDNewHeap = make_new_heap(tab->relid, NewTableSpace);
  
! 			/*
! 			 * Copy the heap data into the new table with the desired
! 			 * modifications, and test the current data within the table
! 			 * against new constraints generated by ALTER TABLE commands.
! 			 */
! 			ATRewriteTable(tab, OIDNewHeap, lockmode);
  
! 			/*
! 			 * Swap the physical files of the old and new heaps, then rebuild
! 			 * indexes and discard the old heap.  We can use RecentXmin for
! 			 * the table's new relfrozenxid because we rewrote all the tuples
! 			 * in ATRewriteTable, so no older Xid remains in the table.  Also,
! 			 * we never try to swap toast tables by content, since we have no
! 			 * interest in letting this code work on system catalogs.
! 			 */
! 			finish_heap_swap(tab->relid, OIDNewHeap,
! 							 false, false, true, RecentXmin);
! 		}
! 		else
! 		{
! 			/*
! 			 * Test the current data within the table against new constraints
! 			 * generated by ALTER TABLE commands, but don't rebuild data.
! 			 */
! 			if (tab->constraints != NIL || tab->new_notnull)
  				ATRewriteTable(tab, InvalidOid, lockmode);
  
! 			/*
! 			 * If we had SET TABLESPACE but no reason to reconstruct tuples,
! 			 * just do a block-by-block copy.
! 			 */
! 			if (tab->newTableSpace)
! 				ATExecSetTableSpace(tab->relid, tab->newTableSpace, lockmode);
  		}
  	}
  
--- 3183,3271 ----
  		if (tab->relkind == RELKIND_FOREIGN_TABLE)
  			continue;
  
! 		/* New NOT NULL constraints always require a scan. */
! 		if (tab->new_notnull)
! 			tab->worklevel = Max(tab->worklevel, WORK_SCAN);
! 
! 		switch (tab->worklevel)
  		{
! 			case WORK_REWRITE:
! 				{
! 					/* Build a temporary relation and copy data */
! 					Relation	OldHeap;
! 					Oid			OIDNewHeap;
! 					Oid			NewTableSpace;
  
! 					OldHeap = heap_open(tab->relid, NoLock);
  
! 					/*
! 					 * We don't support rewriting of system catalogs; there are
! 					 * too many corner cases and too little benefit.  In
! 					 * particular, it would fail for mapped catalogs.
! 					 */
! 					if (IsSystemRelation(OldHeap))
! 						ereport(ERROR,
! 								(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 								 errmsg("cannot rewrite system relation \"%s\"",
! 										RelationGetRelationName(OldHeap))));
  
! 					/*
! 					 * Don't allow rewrite on temp tables of other backends
! 					 * ... their local buffer manager is not going to cope.
! 					 */
! 					if (RELATION_IS_OTHER_TEMP(OldHeap))
! 						ereport(ERROR,
! 								(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 								 errmsg("cannot rewrite temporary tables of other sessions")));
  
! 					/*
! 					 * Select destination tablespace (same as original unless
! 					 * user requested a change)
! 					 */
! 					if (tab->newTableSpace)
! 						NewTableSpace = tab->newTableSpace;
! 					else
! 						NewTableSpace = OldHeap->rd_rel->reltablespace;
  
! 					heap_close(OldHeap, NoLock);
  
! 					/* Create transient table for the modified data */
! 					OIDNewHeap = make_new_heap(tab->relid, NewTableSpace);
  
! 					/*
! 					 * Copy the heap data into the new table with the desired
! 					 * modifications, and test the current data within the table
! 					 * against new constraints.
! 					 */
! 					ATRewriteTable(tab, OIDNewHeap, lockmode);
  
! 					/*
! 					 * Swap the physical files of the old and new heaps, then
! 					 * rebuild indexes and discard the old heap.  We can use
! 					 * RecentXmin for the table's new relfrozenxid because we
! 					 * rewrote all the tuples in ATRewriteTable, so no older Xid
! 					 * remains in the table.  Also, we never try to swap toast
! 					 * tables by content, since we have no interest in letting
! 					 * this code work on system catalogs.
! 					 */
! 					finish_heap_swap(tab->relid, OIDNewHeap,
! 									 false, false, true, RecentXmin);
! 				}
! 				break;
! 
! 			case WORK_SCAN:
! 				/* Test the current table data. */
  				ATRewriteTable(tab, InvalidOid, lockmode);
+ 				break;
  
! 			case WORK_NONE:
! 				/*
! 				 * If we had SET TABLESPACE but no reason to reconstruct tuples,
! 				 * just do a block-by-block copy.
! 				 */
! 				if (tab->newTableSpace)
! 					ATExecSetTableSpace(tab->relid, tab->newTableSpace, lockmode);
! 				break;
  		}
  	}
  
***************
*** 3320,3326 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  	Relation	newrel;
  	TupleDesc	oldTupDesc;
  	TupleDesc	newTupDesc;
- 	bool		needscan = false;
  	List	   *notnull_attrs;
  	int			i;
  	ListCell   *l;
--- 3326,3331 ----
***************
*** 3328,3333 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
--- 3333,3348 ----
  	CommandId	mycid;
  	BulkInsertState bistate;
  	int			hi_options;
+ 	ExprContext *econtext;
+ 	Datum	   *values;
+ 	bool	   *isnull;
+ 	TupleTableSlot *oldslot;
+ 	TupleTableSlot *newslot;
+ 	HeapScanDesc scan;
+ 	HeapTuple	tuple;
+ 	MemoryContext oldCxt;
+ 	List	   *dropped_attrs = NIL;
+ 	ListCell   *lc;
  
  	/*
  	 * Open the relation(s).  We have surely already locked the existing
***************
*** 3379,3385 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  		switch (con->contype)
  		{
  			case CONSTR_CHECK:
- 				needscan = true;
  				con->qualstate = (List *)
  					ExecPrepareExpr((Expr *) con->qual, estate);
  				break;
--- 3394,3399 ----
***************
*** 3414,3580 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  				!newTupDesc->attrs[i]->attisdropped)
  				notnull_attrs = lappend_int(notnull_attrs, i);
  		}
- 		if (notnull_attrs)
- 			needscan = true;
  	}
  
! 	if (newrel || needscan)
! 	{
! 		ExprContext *econtext;
! 		Datum	   *values;
! 		bool	   *isnull;
! 		TupleTableSlot *oldslot;
! 		TupleTableSlot *newslot;
! 		HeapScanDesc scan;
! 		HeapTuple	tuple;
! 		MemoryContext oldCxt;
! 		List	   *dropped_attrs = NIL;
! 		ListCell   *lc;
! 
! 		if (newrel)
! 			ereport(DEBUG1,
! 					(errmsg("Rewriting table \"%s\"",
! 							RelationGetRelationName(oldrel))));
! 		else
! 			ereport(DEBUG1,
! 					(errmsg("Verifying table \"%s\"",
! 							RelationGetRelationName(oldrel))));
  
! 		econtext = GetPerTupleExprContext(estate);
  
! 		/*
! 		 * Make tuple slots for old and new tuples.  Note that even when the
! 		 * tuples are the same, the tupDescs might not be (consider ADD COLUMN
! 		 * without a default).
! 		 */
! 		oldslot = MakeSingleTupleTableSlot(oldTupDesc);
! 		newslot = MakeSingleTupleTableSlot(newTupDesc);
  
! 		/* Preallocate values/isnull arrays */
! 		i = Max(newTupDesc->natts, oldTupDesc->natts);
! 		values = (Datum *) palloc(i * sizeof(Datum));
! 		isnull = (bool *) palloc(i * sizeof(bool));
! 		memset(values, 0, i * sizeof(Datum));
! 		memset(isnull, true, i * sizeof(bool));
  
! 		/*
! 		 * Any attributes that are dropped according to the new tuple
! 		 * descriptor can be set to NULL. We precompute the list of dropped
! 		 * attributes to avoid needing to do so in the per-tuple loop.
! 		 */
! 		for (i = 0; i < newTupDesc->natts; i++)
! 		{
! 			if (newTupDesc->attrs[i]->attisdropped)
! 				dropped_attrs = lappend_int(dropped_attrs, i);
! 		}
  
! 		/*
! 		 * Scan through the rows, generating a new row if needed and then
! 		 * checking all the constraints.
! 		 */
! 		scan = heap_beginscan(oldrel, SnapshotNow, 0, NULL);
  
! 		/*
! 		 * Switch to per-tuple memory context and reset it for each tuple
! 		 * produced, so we don't leak memory.
! 		 */
! 		oldCxt = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
  
! 		while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
  		{
! 			if (newrel)
! 			{
! 				Oid			tupOid = InvalidOid;
  
! 				/* Extract data from old tuple */
! 				heap_deform_tuple(tuple, oldTupDesc, values, isnull);
! 				if (oldTupDesc->tdhasoid)
! 					tupOid = HeapTupleGetOid(tuple);
  
! 				/* Set dropped attributes to null in new tuple */
! 				foreach(lc, dropped_attrs)
! 					isnull[lfirst_int(lc)] = true;
  
! 				/*
! 				 * Process supplied expressions to replace selected columns.
! 				 * Expression inputs come from the old tuple.
! 				 */
! 				ExecStoreTuple(tuple, oldslot, InvalidBuffer, false);
! 				econtext->ecxt_scantuple = oldslot;
  
! 				foreach(l, tab->newvals)
! 				{
! 					NewColumnValue *ex = lfirst(l);
  
! 					values[ex->attnum - 1] = ExecEvalExpr(ex->exprstate,
! 														  econtext,
! 													 &isnull[ex->attnum - 1],
! 														  NULL);
! 				}
  
! 				/*
! 				 * Form the new tuple. Note that we don't explicitly pfree it,
! 				 * since the per-tuple memory context will be reset shortly.
! 				 */
! 				tuple = heap_form_tuple(newTupDesc, values, isnull);
  
! 				/* Preserve OID, if any */
! 				if (newTupDesc->tdhasoid)
! 					HeapTupleSetOid(tuple, tupOid);
! 			}
  
! 			/* Now check any constraints on the possibly-changed tuple */
! 			ExecStoreTuple(tuple, newslot, InvalidBuffer, false);
! 			econtext->ecxt_scantuple = newslot;
  
! 			foreach(l, notnull_attrs)
! 			{
! 				int			attn = lfirst_int(l);
  
! 				if (heap_attisnull(tuple, attn + 1))
! 					ereport(ERROR,
! 							(errcode(ERRCODE_NOT_NULL_VIOLATION),
! 							 errmsg("column \"%s\" contains null values",
  								NameStr(newTupDesc->attrs[attn]->attname))));
! 			}
  
! 			foreach(l, tab->constraints)
! 			{
! 				NewConstraint *con = lfirst(l);
  
! 				switch (con->contype)
! 				{
! 					case CONSTR_CHECK:
! 						if (!ExecQual(con->qualstate, econtext, true))
! 							ereport(ERROR,
! 									(errcode(ERRCODE_CHECK_VIOLATION),
! 									 errmsg("check constraint \"%s\" is violated by some row",
! 											con->name)));
! 						break;
! 					case CONSTR_FOREIGN:
! 						/* Nothing to do here */
! 						break;
! 					default:
! 						elog(ERROR, "unrecognized constraint type: %d",
! 							 (int) con->contype);
! 				}
  			}
  
! 			/* Write the tuple out to the new relation */
! 			if (newrel)
! 				heap_insert(newrel, tuple, mycid, hi_options, bistate);
  
! 			ResetExprContext(econtext);
  
! 			CHECK_FOR_INTERRUPTS();
! 		}
  
! 		MemoryContextSwitchTo(oldCxt);
! 		heap_endscan(scan);
  
! 		ExecDropSingleTupleTableSlot(oldslot);
! 		ExecDropSingleTupleTableSlot(newslot);
! 	}
  
  	FreeExecutorState(estate);
  
--- 3428,3578 ----
  				!newTupDesc->attrs[i]->attisdropped)
  				notnull_attrs = lappend_int(notnull_attrs, i);
  		}
  	}
  
! 	if (newrel)
! 		ereport(DEBUG1,
! 				(errmsg("Rewriting table \"%s\"",
! 						RelationGetRelationName(oldrel))));
! 	else
! 		ereport(DEBUG1,
! 				(errmsg("Verifying table \"%s\"",
! 						RelationGetRelationName(oldrel))));
  
! 	econtext = GetPerTupleExprContext(estate);
  
! 	/*
! 	 * Make tuple slots for old and new tuples.  Note that even when the
! 	 * tuples are the same, the tupDescs might not be (consider ADD COLUMN
! 	 * without a default).
! 	 */
! 	oldslot = MakeSingleTupleTableSlot(oldTupDesc);
! 	newslot = MakeSingleTupleTableSlot(newTupDesc);
  
! 	/* Preallocate values/isnull arrays */
! 	i = Max(newTupDesc->natts, oldTupDesc->natts);
! 	values = (Datum *) palloc(i * sizeof(Datum));
! 	isnull = (bool *) palloc(i * sizeof(bool));
! 	memset(values, 0, i * sizeof(Datum));
! 	memset(isnull, true, i * sizeof(bool));
  
! 	/*
! 	 * Any attributes that are dropped according to the new tuple
! 	 * descriptor can be set to NULL. We precompute the list of dropped
! 	 * attributes to avoid needing to do so in the per-tuple loop.
! 	 */
! 	for (i = 0; i < newTupDesc->natts; i++)
! 	{
! 		if (newTupDesc->attrs[i]->attisdropped)
! 			dropped_attrs = lappend_int(dropped_attrs, i);
! 	}
  
! 	/*
! 	 * Scan through the rows, generating a new row if needed and then
! 	 * checking all the constraints.
! 	 */
! 	scan = heap_beginscan(oldrel, SnapshotNow, 0, NULL);
  
! 	/*
! 	 * Switch to per-tuple memory context and reset it for each tuple
! 	 * produced, so we don't leak memory.
! 	 */
! 	oldCxt = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
  
! 	while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
! 	{
! 		if (newrel)
  		{
! 			Oid			tupOid = InvalidOid;
  
! 			/* Extract data from old tuple */
! 			heap_deform_tuple(tuple, oldTupDesc, values, isnull);
! 			if (oldTupDesc->tdhasoid)
! 				tupOid = HeapTupleGetOid(tuple);
  
! 			/* Set dropped attributes to null in new tuple */
! 			foreach(lc, dropped_attrs)
! 				isnull[lfirst_int(lc)] = true;
  
! 			/*
! 			 * Process supplied expressions to replace selected columns.
! 			 * Expression inputs come from the old tuple.
! 			 */
! 			ExecStoreTuple(tuple, oldslot, InvalidBuffer, false);
! 			econtext->ecxt_scantuple = oldslot;
  
! 			foreach(l, tab->newvals)
! 			{
! 				NewColumnValue *ex = lfirst(l);
  
! 				values[ex->attnum - 1] = ExecEvalExpr(ex->exprstate,
! 													  econtext,
! 													  &isnull[ex->attnum - 1],
! 													  NULL);
! 			}
  
! 			/*
! 			 * Form the new tuple. Note that we don't explicitly pfree it,
! 			 * since the per-tuple memory context will be reset shortly.
! 			 */
! 			tuple = heap_form_tuple(newTupDesc, values, isnull);
  
! 			/* Preserve OID, if any */
! 			if (newTupDesc->tdhasoid)
! 				HeapTupleSetOid(tuple, tupOid);
! 		}
  
! 		/* Now check any constraints on the possibly-changed tuple */
! 		ExecStoreTuple(tuple, newslot, InvalidBuffer, false);
! 		econtext->ecxt_scantuple = newslot;
  
! 		foreach(l, notnull_attrs)
! 		{
! 			int			attn = lfirst_int(l);
  
! 			if (heap_attisnull(tuple, attn + 1))
! 				ereport(ERROR,
! 						(errcode(ERRCODE_NOT_NULL_VIOLATION),
! 						 errmsg("column \"%s\" contains null values",
  								NameStr(newTupDesc->attrs[attn]->attname))));
! 		}
  
! 		foreach(l, tab->constraints)
! 		{
! 			NewConstraint *con = lfirst(l);
  
! 			switch (con->contype)
! 			{
! 			case CONSTR_CHECK:
! 				if (!ExecQual(con->qualstate, econtext, true))
! 					ereport(ERROR,
! 							(errcode(ERRCODE_CHECK_VIOLATION),
! 							 errmsg("check constraint \"%s\" is violated by some row",
! 									con->name)));
! 				break;
! 			case CONSTR_FOREIGN:
! 				/* Nothing to do here */
! 				break;
! 			default:
! 				elog(ERROR, "unrecognized constraint type: %d",
! 					 (int) con->contype);
  			}
+ 		}
  
! 		/* Write the tuple out to the new relation */
! 		if (newrel)
! 			heap_insert(newrel, tuple, mycid, hi_options, bistate);
  
! 		ResetExprContext(econtext);
  
! 		CHECK_FOR_INTERRUPTS();
! 	}
  
! 	MemoryContextSwitchTo(oldCxt);
! 	heap_endscan(scan);
  
! 	ExecDropSingleTupleTableSlot(oldslot);
! 	ExecDropSingleTupleTableSlot(newslot);
  
  	FreeExecutorState(estate);
  
***************
*** 4271,4276 **** ATExecAddColumn(AlteredTableInfo *tab, Relation rel,
--- 4269,4275 ----
  			newval->expr = defval;
  
  			tab->newvals = lappend(tab->newvals, newval);
+ 			tab->worklevel = WORK_REWRITE;
  		}
  
  		/*
***************
*** 4288,4294 **** ATExecAddColumn(AlteredTableInfo *tab, Relation rel,
  	 */
  	if (isOid)
  	{
! 		tab->new_changeoids = true;
  
  		/* See comments in ATExecDropColumn. */
  		find_composite_type_dependencies(rel->rd_rel->reltype,
--- 4287,4293 ----
  	 */
  	if (isOid)
  	{
! 		tab->worklevel = WORK_REWRITE;
  
  		/* See comments in ATExecDropColumn. */
  		find_composite_type_dependencies(rel->rd_rel->reltype,
***************
*** 4967,4973 **** ATExecDropColumn(List **wqueue, Relation rel, const char *colName,
  		tab = ATGetQueueEntry(wqueue, rel);
  
  		/* Tell Phase 3 to physically remove the OID column */
! 		tab->new_changeoids = true;
  
  		/*
  		 * We would also need to rewrite all tables using this table's rowtype
--- 4966,4972 ----
  		tab = ATGetQueueEntry(wqueue, rel);
  
  		/* Tell Phase 3 to physically remove the OID column */
! 		tab->worklevel = WORK_REWRITE;
  
  		/*
  		 * We would also need to rewrite all tables using this table's rowtype
***************
*** 5000,5006 **** ATExecAddIndex(AlteredTableInfo *tab, Relation rel,
  	/* suppress schema rights check when rebuilding existing index */
  	check_rights = !is_rebuild;
  	/* skip index build if phase 3 will have to rewrite table anyway */
! 	skip_build = (tab->newvals != NIL);
  	/* suppress notices when rebuilding existing index */
  	quiet = is_rebuild;
  
--- 4999,5005 ----
  	/* suppress schema rights check when rebuilding existing index */
  	check_rights = !is_rebuild;
  	/* skip index build if phase 3 will have to rewrite table anyway */
! 	skip_build = (tab->worklevel == WORK_REWRITE);
  	/* suppress notices when rebuilding existing index */
  	quiet = is_rebuild;
  
***************
*** 5139,5144 **** ATAddCheckConstraint(List **wqueue, AlteredTableInfo *tab, Relation rel,
--- 5138,5144 ----
  		newcon->qual = (Node *) make_ands_implicit((Expr *) ccon->expr);
  
  		tab->constraints = lappend(tab->constraints, newcon);
+ 		tab->worklevel = Max(tab->worklevel, WORK_SCAN);
  
  		/* Save the actually assigned name if it was defaulted */
  		if (constr->conname == NULL)
***************
*** 5485,5490 **** ATAddForeignKeyConstraint(AlteredTableInfo *tab, Relation rel,
--- 5485,5494 ----
  		newcon->qual = (Node *) fkconstraint;
  
  		tab->constraints = lappend(tab->constraints, newcon);
+ 		/*
+ 		 * No need to set tab->worklevel; foreign key validation is a distinct
+ 		 * aspect of Phase 3.
+ 		 */
  	}
  
  	/*
***************
*** 6369,6374 **** ATPrepAlterColumnType(List **wqueue,
--- 6373,6379 ----
  		newval->expr = (Expr *) transform;
  
  		tab->newvals = lappend(tab->newvals, newval);
+ 		tab->worklevel = WORK_REWRITE;
  
  		/*
  		 * If we need to rewrite or scan this table, tables using its rowtype as
at1.3w-tablecmds-enum.patchtext/plain; charset=us-asciiDownload
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 78262d8..63e2538 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 129,134 **** static List *on_commits = NIL;
--- 129,137 ----
  #define AT_PASS_MISC			8		/* other stuff */
  #define AT_NUM_PASSES			9
  
+ /* Level of effort required in Phase 3 (ATRewriteTables). */
+ typedef enum { WORK_NONE = 0, WORK_SCAN = 1, WORK_REWRITE = 2 } WorkLevel;
+ 
  typedef struct AlteredTableInfo
  {
  	/* Information saved before any work commences: */
***************
*** 141,147 **** typedef struct AlteredTableInfo
  	List	   *constraints;	/* List of NewConstraint */
  	List	   *newvals;		/* List of NewColumnValue */
  	bool		new_notnull;	/* T if we added new NOT NULL constraints */
! 	bool		new_changeoids; /* T if we added/dropped the OID column */
  	Oid			newTableSpace;	/* new tablespace; 0 means no change */
  	/* Objects to rebuild after completing ALTER TYPE operations */
  	List	   *changedConstraintOids;	/* OIDs of constraints to rebuild */
--- 144,150 ----
  	List	   *constraints;	/* List of NewConstraint */
  	List	   *newvals;		/* List of NewColumnValue */
  	bool		new_notnull;	/* T if we added new NOT NULL constraints */
! 	WorkLevel	worklevel;		/* How much work shall we do on the heap? */
  	Oid			newTableSpace;	/* new tablespace; 0 means no change */
  	/* Objects to rebuild after completing ALTER TYPE operations */
  	List	   *changedConstraintOids;	/* OIDs of constraints to rebuild */
***************
*** 3180,3190 **** ATRewriteTables(List **wqueue, LOCKMODE lockmode)
  		if (tab->relkind == RELKIND_FOREIGN_TABLE)
  			continue;
  
! 		/*
! 		 * We only need to rewrite the table if at least one column needs to
! 		 * be recomputed, or we are adding/removing the OID column.
! 		 */
! 		if (tab->newvals != NIL || tab->new_changeoids)
  		{
  			/* Build a temporary relation and copy data */
  			Relation	OldHeap;
--- 3183,3195 ----
  		if (tab->relkind == RELKIND_FOREIGN_TABLE)
  			continue;
  
! 		/* New NOT NULL constraints always require a scan. */
! 		if (tab->new_notnull)
! 			tab->worklevel = Max(tab->worklevel, WORK_SCAN);
! 
! 		switch (tab->worklevel)
! 		{
! 			case WORK_REWRITE:
  		{
  			/* Build a temporary relation and copy data */
  			Relation	OldHeap;
***************
*** 3194,3202 **** ATRewriteTables(List **wqueue, LOCKMODE lockmode)
  			OldHeap = heap_open(tab->relid, NoLock);
  
  			/*
! 			 * We don't support rewriting of system catalogs; there are too
! 			 * many corner cases and too little benefit.  In particular this
! 			 * is certainly not going to work for mapped catalogs.
  			 */
  			if (IsSystemRelation(OldHeap))
  				ereport(ERROR,
--- 3199,3207 ----
  			OldHeap = heap_open(tab->relid, NoLock);
  
  			/*
! 					 * We don't support rewriting of system catalogs; there are
! 					 * too many corner cases and too little benefit.  In
! 					 * particular, it would fail for mapped catalogs.
  			 */
  			if (IsSystemRelation(OldHeap))
  				ereport(ERROR,
***************
*** 3205,3212 **** ATRewriteTables(List **wqueue, LOCKMODE lockmode)
  								RelationGetRelationName(OldHeap))));
  
  			/*
! 			 * Don't allow rewrite on temp tables of other backends ... their
! 			 * local buffer manager is not going to cope.
  			 */
  			if (RELATION_IS_OTHER_TEMP(OldHeap))
  				ereport(ERROR,
--- 3210,3217 ----
  								RelationGetRelationName(OldHeap))));
  
  			/*
! 					 * Don't allow rewrite on temp tables of other backends
! 					 * ... their local buffer manager is not going to cope.
  			 */
  			if (RELATION_IS_OTHER_TEMP(OldHeap))
  				ereport(ERROR,
***************
*** 3214,3221 **** ATRewriteTables(List **wqueue, LOCKMODE lockmode)
  				errmsg("cannot rewrite temporary tables of other sessions")));
  
  			/*
! 			 * Select destination tablespace (same as original unless user
! 			 * requested a change)
  			 */
  			if (tab->newTableSpace)
  				NewTableSpace = tab->newTableSpace;
--- 3219,3226 ----
  				errmsg("cannot rewrite temporary tables of other sessions")));
  
  			/*
! 					 * Select destination tablespace (same as original unless
! 					 * user requested a change)
  			 */
  			if (tab->newTableSpace)
  				NewTableSpace = tab->newTableSpace;
***************
*** 3224,3265 **** ATRewriteTables(List **wqueue, LOCKMODE lockmode)
  
  			heap_close(OldHeap, NoLock);
  
! 			/* Create transient table that will receive the modified data */
  			OIDNewHeap = make_new_heap(tab->relid, NewTableSpace);
  
  			/*
  			 * Copy the heap data into the new table with the desired
  			 * modifications, and test the current data within the table
! 			 * against new constraints generated by ALTER TABLE commands.
  			 */
  			ATRewriteTable(tab, OIDNewHeap, lockmode);
  
  			/*
! 			 * Swap the physical files of the old and new heaps, then rebuild
! 			 * indexes and discard the old heap.  We can use RecentXmin for
! 			 * the table's new relfrozenxid because we rewrote all the tuples
! 			 * in ATRewriteTable, so no older Xid remains in the table.  Also,
! 			 * we never try to swap toast tables by content, since we have no
! 			 * interest in letting this code work on system catalogs.
  			 */
  			finish_heap_swap(tab->relid, OIDNewHeap,
  							 false, false, true, RecentXmin);
  		}
! 		else
! 		{
! 			/*
! 			 * Test the current data within the table against new constraints
! 			 * generated by ALTER TABLE commands, but don't rebuild data.
! 			 */
! 			if (tab->constraints != NIL || tab->new_notnull)
  				ATRewriteTable(tab, InvalidOid, lockmode);
  
  			/*
  			 * If we had SET TABLESPACE but no reason to reconstruct tuples,
  			 * just do a block-by-block copy.
  			 */
  			if (tab->newTableSpace)
  				ATExecSetTableSpace(tab->relid, tab->newTableSpace, lockmode);
  		}
  	}
  
--- 3229,3271 ----
  
  			heap_close(OldHeap, NoLock);
  
! 					/* Create transient table for the modified data */
  			OIDNewHeap = make_new_heap(tab->relid, NewTableSpace);
  
  			/*
  			 * Copy the heap data into the new table with the desired
  			 * modifications, and test the current data within the table
! 					 * against new constraints.
  			 */
  			ATRewriteTable(tab, OIDNewHeap, lockmode);
  
  			/*
! 					 * Swap the physical files of the old and new heaps, then
! 					 * rebuild indexes and discard the old heap.  We can use
! 					 * RecentXmin for the table's new relfrozenxid because we
! 					 * rewrote all the tuples in ATRewriteTable, so no older Xid
! 					 * remains in the table.  Also, we never try to swap toast
! 					 * tables by content, since we have no interest in letting
! 					 * this code work on system catalogs.
  			 */
  			finish_heap_swap(tab->relid, OIDNewHeap,
  							 false, false, true, RecentXmin);
  		}
! 				break;
! 
! 			case WORK_SCAN:
! 				/* Test the current table data. */
  				ATRewriteTable(tab, InvalidOid, lockmode);
+ 				break;
  
+ 			case WORK_NONE:
  			/*
  			 * If we had SET TABLESPACE but no reason to reconstruct tuples,
  			 * just do a block-by-block copy.
  			 */
  			if (tab->newTableSpace)
  				ATExecSetTableSpace(tab->relid, tab->newTableSpace, lockmode);
+ 				break;
  		}
  	}
  
***************
*** 3320,3326 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  	Relation	newrel;
  	TupleDesc	oldTupDesc;
  	TupleDesc	newTupDesc;
- 	bool		needscan = false;
  	List	   *notnull_attrs;
  	int			i;
  	ListCell   *l;
--- 3326,3331 ----
***************
*** 3328,3333 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
--- 3333,3348 ----
  	CommandId	mycid;
  	BulkInsertState bistate;
  	int			hi_options;
+ 	ExprContext *econtext;
+ 	Datum	   *values;
+ 	bool	   *isnull;
+ 	TupleTableSlot *oldslot;
+ 	TupleTableSlot *newslot;
+ 	HeapScanDesc scan;
+ 	HeapTuple	tuple;
+ 	MemoryContext oldCxt;
+ 	List	   *dropped_attrs = NIL;
+ 	ListCell   *lc;
  
  	/*
  	 * Open the relation(s).  We have surely already locked the existing
***************
*** 3379,3385 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  		switch (con->contype)
  		{
  			case CONSTR_CHECK:
- 				needscan = true;
  				con->qualstate = (List *)
  					ExecPrepareExpr((Expr *) con->qual, estate);
  				break;
--- 3394,3399 ----
***************
*** 3414,3436 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  				!newTupDesc->attrs[i]->attisdropped)
  				notnull_attrs = lappend_int(notnull_attrs, i);
  		}
- 		if (notnull_attrs)
- 			needscan = true;
  	}
  
- 	if (newrel || needscan)
- 	{
- 		ExprContext *econtext;
- 		Datum	   *values;
- 		bool	   *isnull;
- 		TupleTableSlot *oldslot;
- 		TupleTableSlot *newslot;
- 		HeapScanDesc scan;
- 		HeapTuple	tuple;
- 		MemoryContext oldCxt;
- 		List	   *dropped_attrs = NIL;
- 		ListCell   *lc;
- 
  		if (newrel)
  			ereport(DEBUG1,
  					(errmsg("Rewriting table \"%s\"",
--- 3428,3435 ----
***************
*** 3574,3580 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  
  		ExecDropSingleTupleTableSlot(oldslot);
  		ExecDropSingleTupleTableSlot(newslot);
- 	}
  
  	FreeExecutorState(estate);
  
--- 3573,3578 ----
***************
*** 4271,4276 **** ATExecAddColumn(AlteredTableInfo *tab, Relation rel,
--- 4269,4275 ----
  			newval->expr = defval;
  
  			tab->newvals = lappend(tab->newvals, newval);
+ 			tab->worklevel = WORK_REWRITE;
  		}
  
  		/*
***************
*** 4288,4294 **** ATExecAddColumn(AlteredTableInfo *tab, Relation rel,
  	 */
  	if (isOid)
  	{
! 		tab->new_changeoids = true;
  
  		/* See comments in ATExecDropColumn. */
  		find_composite_type_dependencies(rel->rd_rel->reltype,
--- 4287,4293 ----
  	 */
  	if (isOid)
  	{
! 		tab->worklevel = WORK_REWRITE;
  
  		/* See comments in ATExecDropColumn. */
  		find_composite_type_dependencies(rel->rd_rel->reltype,
***************
*** 4967,4973 **** ATExecDropColumn(List **wqueue, Relation rel, const char *colName,
  		tab = ATGetQueueEntry(wqueue, rel);
  
  		/* Tell Phase 3 to physically remove the OID column */
! 		tab->new_changeoids = true;
  
  		/*
  		 * We would also need to rewrite all tables using this table's rowtype
--- 4966,4972 ----
  		tab = ATGetQueueEntry(wqueue, rel);
  
  		/* Tell Phase 3 to physically remove the OID column */
! 		tab->worklevel = WORK_REWRITE;
  
  		/*
  		 * We would also need to rewrite all tables using this table's rowtype
***************
*** 5000,5006 **** ATExecAddIndex(AlteredTableInfo *tab, Relation rel,
  	/* suppress schema rights check when rebuilding existing index */
  	check_rights = !is_rebuild;
  	/* skip index build if phase 3 will have to rewrite table anyway */
! 	skip_build = (tab->newvals != NIL);
  	/* suppress notices when rebuilding existing index */
  	quiet = is_rebuild;
  
--- 4999,5005 ----
  	/* suppress schema rights check when rebuilding existing index */
  	check_rights = !is_rebuild;
  	/* skip index build if phase 3 will have to rewrite table anyway */
! 	skip_build = (tab->worklevel == WORK_REWRITE);
  	/* suppress notices when rebuilding existing index */
  	quiet = is_rebuild;
  
***************
*** 5139,5144 **** ATAddCheckConstraint(List **wqueue, AlteredTableInfo *tab, Relation rel,
--- 5138,5144 ----
  		newcon->qual = (Node *) make_ands_implicit((Expr *) ccon->expr);
  
  		tab->constraints = lappend(tab->constraints, newcon);
+ 		tab->worklevel = Max(tab->worklevel, WORK_SCAN);
  
  		/* Save the actually assigned name if it was defaulted */
  		if (constr->conname == NULL)
***************
*** 5485,5490 **** ATAddForeignKeyConstraint(AlteredTableInfo *tab, Relation rel,
--- 5485,5494 ----
  		newcon->qual = (Node *) fkconstraint;
  
  		tab->constraints = lappend(tab->constraints, newcon);
+ 		/*
+ 		 * No need to set tab->worklevel; foreign key validation is a distinct
+ 		 * aspect of Phase 3.
+ 		 */
  	}
  
  	/*
***************
*** 6369,6374 **** ATPrepAlterColumnType(List **wqueue,
--- 6373,6379 ----
  		newval->expr = (Expr *) transform;
  
  		tab->newvals = lappend(tab->newvals, newval);
+ 		tab->worklevel = WORK_REWRITE;
  
  		/*
  		 * If we need to rewrite or scan this table, tables using its rowtype as
at2v2-skip-nowork.patchtext/plain; charset=us-asciiDownload
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 5c94a9f..1f6cafd 100644
*** a/doc/src/sgml/ref/alter_table.sgml
--- b/doc/src/sgml/ref/alter_table.sgml
***************
*** 345,353 **** ALTER TABLE <replaceable class="PARAMETER">name</replaceable>
        for details on the available parameters.  Note that the table contents
        will not be modified immediately by this command; depending on the
        parameter you might need to rewrite the table to get the desired effects.
!       That can be done with <xref linkend="SQL-CLUSTER">
!       or one of the forms of <command>ALTER
!       TABLE</> that forces a table rewrite.
       </para>
  
       <note>
--- 345,353 ----
        for details on the available parameters.  Note that the table contents
        will not be modified immediately by this command; depending on the
        parameter you might need to rewrite the table to get the desired effects.
!       That can be done with <link linkend="SQL-VACUUM">VACUUM
!       FULL</>, <xref linkend="SQL-CLUSTER"> or one of the forms
!       of <command>ALTER TABLE</> that forces a table rewrite.
       </para>
  
       <note>
***************
*** 688,701 **** ALTER TABLE <replaceable class="PARAMETER">name</replaceable>
     </para>
  
     <para>
!     Adding a column with a non-null default or changing the type of an
!     existing column will require the entire table and indexes to be rewritten.
!     This might take a significant amount of time for a large table; and it will
!     temporarily require double the disk space.  Adding or removing a system
      <literal>oid</> column likewise requires rewriting the entire table.
     </para>
  
     <para>
      Adding a <literal>CHECK</> or <literal>NOT NULL</> constraint requires
      scanning the table to verify that existing rows meet the constraint.
     </para>
--- 688,712 ----
     </para>
  
     <para>
!     Adding a column with a non-null default will rewrite the entire table and
!     all indexes.  Changing the type of an existing column will do the same
!     unless a binary-coercible cast implements the type conversion.  Refer to
!     <xref linkend="SQL-CREATECAST"> for further information.  A rewrite might
!     take a significant amount of time for a large table, and it will temporarily
!     require double the disk space.  Adding or removing a system
      <literal>oid</> column likewise requires rewriting the entire table.
     </para>
  
     <para>
+     Similar to the behavior of <link linkend="SQL-VACUUM">VACUUM FULL</>, the
+     rewriting process eliminates any dead space in the table.  Prior
+     to <productname>PostgreSQL</> 9.0, <literal>SET DATA TYPE</>
+     outpaced <literal>VACUUM FULL</> at this task, so it was useful even absent
+     the need for a column type change.  This speed advantage no longer holds,
+     and <literal>SET DATA TYPE</> may not even rewrite the table.
+    </para>
+ 
+    <para>
      Adding a <literal>CHECK</> or <literal>NOT NULL</> constraint requires
      scanning the table to verify that existing rows meet the constraint.
     </para>
***************
*** 719,739 **** ALTER TABLE <replaceable class="PARAMETER">name</replaceable>
     </para>
  
     <para>
-     The fact that <literal>SET DATA TYPE</> requires rewriting the whole table
-     is sometimes an advantage, because the rewriting process eliminates
-     any dead space in the table.  For example, to reclaim the space occupied
-     by a dropped column immediately, the fastest way is:
- <programlisting>
- ALTER TABLE table ALTER COLUMN anycol TYPE anytype;
- </programlisting>
-     where <literal>anycol</> is any remaining table column and
-     <literal>anytype</> is the same type that column already has.
-     This results in no semantically-visible change in the table,
-     but the command forces rewriting, which gets rid of no-longer-useful
-     data.
-    </para>
- 
-    <para>
      The <literal>USING</literal> option of <literal>SET DATA TYPE</> can actually
      specify any expression involving the old values of the row; that is, it
      can refer to other columns as well as the one being converted.  This allows
--- 730,735 ----
diff --git a/src/backend/commands/tablecindex 63e2538..d9cb4a2 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 71,76 ****
--- 71,77 ----
  #include "storage/smgr.h"
  #include "utils/acl.h"
  #include "utils/builtins.h"
+ #include "utils/datum.h"
  #include "utils/fmgroids.h"
  #include "utils/inval.h"
  #include "utils/lsyscache.h"
***************
*** 3481,3498 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  
  	while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
  	{
! 		if (newrel)
  		{
  			Oid			tupOid = InvalidOid;
  
! 			/* Extract data from old tuple */
! 			heap_deform_tuple(tuple, oldTupDesc, values, isnull);
! 			if (oldTupDesc->tdhasoid)
! 				tupOid = HeapTupleGetOid(tuple);
! 
! 			/* Set dropped attributes to null in new tuple */
! 			foreach(lc, dropped_attrs)
! 				isnull[lfirst_int(lc)] = true;
  
  			/*
  			 * Process supplied expressions to replace selected columns.
--- 3482,3512 ----
  
  	while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
  	{
! 		/*
! 		 * If we're rewriting or verifying, compute new tuple values using each
! 		 * transformation expression.  When rewriting, also form a new physical
! 		 * tuple.  In Assert-enabled builds, check for cases that should have
! 		 * been WORK_REWRITE by comparing the data.
! 		 */
! 		if (newrel || tab->newvals != NIL)
  		{
  			Oid			tupOid = InvalidOid;
  
! 			if (newrel
! #ifdef USE_ASSERT_CHECKING
! 				|| assert_enabled
! #endif
! 				)
! 			{
! 				/* Extract data from old tuple */
! 				heap_deform_tuple(tuple, oldTupDesc, values, isnull);
! 				if (oldTupDesc->tdhasoid)
! 					tupOid = HeapTupleGetOid(tuple);
! 
! 				/* Set dropped attributes to null in new tuple */
! 				foreach(lc, dropped_attrs)
! 					isnull[lfirst_int(lc)] = true;
! 			}
  
  			/*
  			 * Process supplied expressions to replace selected columns.
***************
*** 3509,3525 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  													  econtext,
  													  &isnull[ex->attnum - 1],
  													  NULL);
  			}
  
! 			/*
! 			 * Form the new tuple. Note that we don't explicitly pfree it,
! 			 * since the per-tuple memory context will be reset shortly.
! 			 */
! 			tuple = heap_form_tuple(newTupDesc, values, isnull);
  
! 			/* Preserve OID, if any */
! 			if (newTupDesc->tdhasoid)
! 				HeapTupleSetOid(tuple, tupOid);
  		}
  
  		/* Now check any constraints on the possibly-changed tuple */
--- 3523,3566 ----
  													  econtext,
  													  &isnull[ex->attnum - 1],
  													  NULL);
+ 
+ #ifdef USE_ASSERT_CHECKING
+ 				if (assert_enabled)
+ 				{
+ 					Datum		oldval = values[ex->attnum - 1];
+ 					bool		oldisnull = isnull[ex->attnum - 1];
+ 					Form_pg_attribute f = newTupDesc->attrs[ex->attnum - 1];
+ 
+ 					if (f->attbyval && f->attlen == -1)
+ 						oldval = PointerGetDatum(PG_DETOAST_DATUM(oldval));
+ 
+ 					/*
+ 					 * We don't detect the gross error of !newrel when the
+ 					 * typlen actually changed.  attbyval could differ in
+ 					 * theory, but we assume it does not.
+ 					 */
+ 					Assert(newrel ||
+ 						   (isnull[ex->attnum - 1] == oldisnull
+ 							&& (oldisnull ||
+ 								datumIsEqual(oldval,
+ 											 values[ex->attnum - 1],
+ 											 f->attbyval, f->attlen))));
+ 				}
+ #endif
  			}
  
! 			if (newrel)
! 			{
! 				/*
! 				 * Form the new tuple. Note that we don't explicitly pfree it,
! 				 * since the per-tuple memory context will be reset shortly.
! 				 */
! 				tuple = heap_form_tuple(newTupDesc, values, isnull);
  
! 				/* Preserve OID, if any */
! 				if (newTupDesc->tdhasoid)
! 					HeapTupleSetOid(tuple, tupOid);
! 			}
  		}
  
  		/* Now check any constraints on the possibly-changed tuple */
***************
*** 6303,6308 **** ATPrepAlterColumnType(List **wqueue,
--- 6344,6352 ----
  
  	if (tab->relkind == RELKIND_RELATION)
  	{
+ 		CoerceExemptions exempt;
+ 		WorkLevel	worklevel;
+ 
  		/*
  		 * Set up an expression to transform the old data value to the new type.
  		 * If a USING option was given, transform and use that expression, else
***************
*** 6363,6368 **** ATPrepAlterColumnType(List **wqueue,
--- 6407,6413 ----
  					(errcode(ERRCODE_DATATYPE_MISMATCH),
  					 errmsg("column \"%s\" cannot be cast to type %s",
  							colName, format_type_be(targettype))));
+ 		exempt = GetCoerceExemptions(transform, 1, attnum);
  
  		/*
  		 * Add a work queue item to make ATRewriteTable update the column
***************
*** 6373,6387 **** ATPrepAlterColumnType(List **wqueue,
  		newval->expr = (Expr *) transform;
  
  		tab->newvals = lappend(tab->newvals, newval);
! 		tab->worklevel = WORK_REWRITE;
  
  		/*
  		 * If we need to rewrite or scan this table, tables using its rowtype as
  		 * a column type would need the same treatment.
  		 */
! 		find_composite_type_dependencies(rel->rd_rel->reltype,
! 										 RelationGetRelationName(rel),
! 										 NULL);
  	}
  	else if (tab->relkind == RELKIND_COMPOSITE_TYPE)
  	{
--- 6418,6440 ----
  		newval->expr = (Expr *) transform;
  
  		tab->newvals = lappend(tab->newvals, newval);
! 		if (!(exempt & COERCE_EXEMPT_NOCHANGE))
! 			worklevel = WORK_REWRITE;
! 		else if (!(exempt & COERCE_EXEMPT_NOERROR))
! 			worklevel = WORK_SCAN;
! 		else					/* both bits set */
! 			worklevel = WORK_NONE;
  
  		/*
  		 * If we need to rewrite or scan this table, tables using its rowtype as
  		 * a column type would need the same treatment.
  		 */
! 		if (worklevel != WORK_NONE)
! 			find_composite_type_dependencies(rel->rd_rel->reltype,
! 											 RelationGetRelationName(rel),
! 											 NULL);
! 
! 		tab->worklevel = Max(tab->worklevel, worklevel);
  	}
  	else if (tab->relkind == RELKIND_COMPOSITE_TYPE)
  	{
diff --git a/src/backend/parser/parse_cindex 5b0dc14..b27e5d2 100644
*** a/src/backend/parser/parse_coerce.c
--- b/src/backend/parser/parse_coerce.c
***************
*** 19,24 ****
--- 19,25 ----
  #include "catalog/pg_inherits_fn.h"
  #include "catalog/pg_proc.h"
  #include "catalog/pg_type.h"
+ #include "commands/typecmds.h"
  #include "nodes/makefuncs.h"
  #include "nodes/nodeFuncs.h"
  #include "parser/parse_coerce.h"
***************
*** 1805,1810 **** IsBinaryCoercible(Oid srctype, Oid targettype)
--- 1806,1871 ----
  }
  
  
+ /* GetCoerceExemptions()
+  *		Assess invariants of a coercion expression.
+  *
+  * Various common expressions arising from type coercion are subject to
+  * optimizations.  For example, a simple varchar -> text cast will never change
+  * the underlying data (COERCE_EXEMPT_NOCHANGE) and never yield an error
+  * (COERCE_EXEMPT_NOERROR).  A varchar(8) -> varchar(4) will never change the
+  * data, but it may yield an error.  Given a varno and varattno denoting "the"
+  * source datum, determine which invariants hold for an expression by walking it
+  * per these rules:
+  *
+  *	1. A Var with the varno/varattno in question has both invariants.
+  *	2. A RelabelType node inherits the invariants of its sole argument.
+  *	3. A CoerceToDomain node inherits any COERCE_EXEMPT_NOCHANGE invariant from
+  *		its sole argument.  When GetDomainConstraints() == NIL, it also inherits
+  *		COERCE_EXEMPT_NOERROR.  Otherwise, COERCE_EXEMPT_NOERROR becomes false.
+  *	4. All other nodes have neither invariant.
+  *
+  * Returns a bit string that may contain the following bits:
+  *	COERCE_EXEMPT_NOCHANGE: expression result will always have the same binary
+  *				representation as a Var expression having the given varno and
+  *				varattno
+  *	COERCE_EXEMPT_NOERROR: expression will never throw an error
+  */
+ CoerceExemptions
+ GetCoerceExemptions(Node *expr,
+ 					Index varno, AttrNumber varattno)
+ {
+ 	CoerceExemptions ret = COERCE_EXEMPT_NOCHANGE | COERCE_EXEMPT_NOERROR;
+ 
+ 	Assert(expr != NULL);
+ 
+ 	for (;;)
+ 	{
+ 		if (IsA(expr, Var)
+ 			&& ((Var *) expr)->varno == varno
+ 			&& ((Var *) expr)->varattno == varattno)
+ 		{
+ 			return ret;
+ 		}
+ 		if (IsA(expr, RelabelType))
+ 		{
+ 			expr = (Node *) ((RelabelType *) expr)->arg;
+ 		}
+ 		else if (IsA(expr, CoerceToDomain))
+ 		{
+ 			CoerceToDomain *d = (CoerceToDomain *) expr;
+ 
+ 			if (GetDomainConstraints(d->resulttype) != NIL)
+ 				ret &= ~COERCE_EXEMPT_NOERROR;
+ 			expr = (Node *) d->arg;
+ 		}
+ 		else
+ 		{
+ 			return 0;
+ 		}
+ 	}
+ }
+ 
+ 
  /*
   * find_coercion_pathway
   *		Look for a coercion pathway between two types.
diff --git a/src/include/parser/parse_coindex ceaff2f..4303acf 100644
*** a/src/include/parser/parse_coerce.h
--- b/src/include/parser/parse_coerce.h
***************
*** 30,37 **** typedef enum CoercionPathType
--- 30,44 ----
  	COERCION_PATH_COERCEVIAIO	/* need a CoerceViaIO node */
  } CoercionPathType;
  
+ /* Bits in the return value of GetCoerceExemptions. */
+ typedef int CoerceExemptions;
+ 
+ #define COERCE_EXEMPT_NOCHANGE	0x1		/* expression never changes storage */
+ #define COERCE_EXEMPT_NOERROR	0x2		/* expression never throws an error */
  
  extern bool IsBinaryCoercible(Oid srctype, Oid targettype);
+ extern CoerceExemptions GetCoerceExemptions(Node *expr,
+ 					Index varno, AttrNumber varattno);
  extern bool IsPreferredType(TYPCATEGORY category, Oid type);
  extern TYPCATEGORY TypeCategory(Oid type);
  
diff --git a/src/test/regress/expected/aindex f2cfc07..dd6365c 100644
*** a/src/test/regress/expected/alter_table.out
--- b/src/test/regress/expected/alter_table.out
***************
*** 1479,1485 **** drop table another;
  -- table's row type
  create table tab1 (a int, b text);
  create table tab2 (x int, y tab1);
! alter table tab1 alter column b type varchar; -- fails
  ERROR:  cannot alter table "tab1" because column "tab2"."y" uses its rowtype
  alter table tab1 add check (b <> 'foo');
  alter table tab1 add c int not null;
--- 1479,1486 ----
  -- table's row type
  create table tab1 (a int, b text);
  create table tab2 (x int, y tab1);
! alter table tab1 alter column b type varchar;
! alter table tab1 alter column b type varchar(1); -- fails
  ERROR:  cannot alter table "tab1" because column "tab2"."y" uses its rowtype
  alter table tab1 add check (b <> 'foo');
  alter table tab1 add c int not null;
***************
*** 1757,1801 **** FROM pg_trigger WHERE tgrelid = 't'::regclass ORDER BY tgname;
  -- though mostly not stated here.
  -- Constraint failures induced by a no-work type change.
  ALTER TABLE t ALTER constraint0 TYPE trickint;						-- verify-e
! DEBUG:  Rewriting table "t"
  ERROR:  check constraint "t_constraint0_check" is violated by some row
  ALTER TABLE t ALTER constraint1 TYPE trickint;						-- noop-e
- DEBUG:  Rewriting table "t"
- DEBUG:  Rebuilding index "t_constraint4_key"
- DEBUG:  Rebuilding index "t_integral_key"
- DEBUG:  Rebuilding index "t_rational_key"
- DEBUG:  Rebuilding index "t_daytimetz_key"
- DEBUG:  Rebuilding index "t_daytime_key"
- DEBUG:  Rebuilding index "t_stamptz_key"
- DEBUG:  Rebuilding index "t_stamp_key"
- DEBUG:  Rebuilding index "t_timegap_key"
- DEBUG:  Rebuilding index "t_bits_key"
- DEBUG:  Rebuilding index "t_network_key"
- DEBUG:  Rebuilding index "t_string_idx"
- DEBUG:  Rebuilding index "t_string_idx1"
- DEBUG:  Rebuilding index "t_strarr_idx"
- DEBUG:  Rebuilding index "t_square_idx"
- DEBUG:  Rebuilding index "t_expr_idx"
  DEBUG:  Rebuilding index "t_touchy_f_idx"
  ERROR:  could not create unique index "t_touchy_f_idx"
  DETAIL:  Key (touchy_f(constraint1))=(100) is duplicated.
  ALTER TABLE t ALTER constraint2 TYPE trickint;						-- noop-e
- DEBUG:  Rewriting table "t"
- DEBUG:  Rebuilding index "t_constraint4_key"
- DEBUG:  Rebuilding index "t_integral_key"
- DEBUG:  Rebuilding index "t_rational_key"
- DEBUG:  Rebuilding index "t_daytimetz_key"
- DEBUG:  Rebuilding index "t_daytime_key"
- DEBUG:  Rebuilding index "t_stamptz_key"
- DEBUG:  Rebuilding index "t_stamp_key"
- DEBUG:  Rebuilding index "t_timegap_key"
- DEBUG:  Rebuilding index "t_bits_key"
- DEBUG:  Rebuilding index "t_network_key"
- DEBUG:  Rebuilding index "t_string_idx"
- DEBUG:  Rebuilding index "t_string_idx1"
- DEBUG:  Rebuilding index "t_strarr_idx"
- DEBUG:  Rebuilding index "t_square_idx"
- DEBUG:  Rebuilding index "t_touchy_f_idx"
  DEBUG:  Rebuilding index "t_expr_idx"
  ERROR:  could not create unique index "t_expr_idx"
  DETAIL:  Key ((1))=(1) is duplicated.
--- 1758,1770 ----
  -- though mostly not stated here.
  -- Constraint failures induced by a no-work type change.
  ALTER TABLE t ALTER constraint0 TYPE trickint;						-- verify-e
! DEBUG:  Verifying table "t"
  ERROR:  check constraint "t_constraint0_check" is violated by some row
  ALTER TABLE t ALTER constraint1 TYPE trickint;						-- noop-e
  DEBUG:  Rebuilding index "t_touchy_f_idx"
  ERROR:  could not create unique index "t_touchy_f_idx"
  DETAIL:  Key (touchy_f(constraint1))=(100) is duplicated.
  ALTER TABLE t ALTER constraint2 TYPE trickint;						-- noop-e
  DEBUG:  Rebuilding index "t_expr_idx"
  ERROR:  could not create unique index "t_expr_idx"
  DETAIL:  Key ((1))=(1) is duplicated.
***************
*** 1959,2016 **** DEBUG:  Rebuilding index "t_constraint4_key"
  DEBUG:  Validating foreign key constraint "child_keycol_fkey"
  -- Type-specific tests.
  ALTER TABLE t ALTER integral TYPE abstime USING integral::abstime;	-- noop
- DEBUG:  Rewriting table "t"
- DEBUG:  Rebuilding index "t_rational_key"
- DEBUG:  Rebuilding index "t_daytimetz_key"
- DEBUG:  Rebuilding index "t_daytime_key"
- DEBUG:  Rebuilding index "t_stamptz_key"
- DEBUG:  Rebuilding index "t_stamp_key"
- DEBUG:  Rebuilding index "t_timegap_key"
- DEBUG:  Rebuilding index "t_bits_key"
- DEBUG:  Rebuilding index "t_network_key"
- DEBUG:  Rebuilding index "t_string_idx"
- DEBUG:  Rebuilding index "t_string_idx1"
- DEBUG:  Rebuilding index "t_strarr_idx"
- DEBUG:  Rebuilding index "t_square_idx"
- DEBUG:  Rebuilding index "t_touchy_f_idx"
- DEBUG:  Rebuilding index "t_expr_idx"
- DEBUG:  Rebuilding index "t_constraint4_key"
  DEBUG:  Rebuilding index "t_integral_key"
  ALTER TABLE t ALTER integral TYPE oid USING integral::int4;			-- noop
- DEBUG:  Rewriting table "t"
- DEBUG:  Rebuilding index "t_rational_key"
- DEBUG:  Rebuilding index "t_daytimetz_key"
- DEBUG:  Rebuilding index "t_daytime_key"
- DEBUG:  Rebuilding index "t_stamptz_key"
- DEBUG:  Rebuilding index "t_stamp_key"
- DEBUG:  Rebuilding index "t_timegap_key"
- DEBUG:  Rebuilding index "t_bits_key"
- DEBUG:  Rebuilding index "t_network_key"
- DEBUG:  Rebuilding index "t_string_idx"
- DEBUG:  Rebuilding index "t_string_idx1"
- DEBUG:  Rebuilding index "t_strarr_idx"
- DEBUG:  Rebuilding index "t_square_idx"
- DEBUG:  Rebuilding index "t_touchy_f_idx"
- DEBUG:  Rebuilding index "t_expr_idx"
- DEBUG:  Rebuilding index "t_constraint4_key"
  DEBUG:  Rebuilding index "t_integral_key"
  ALTER TABLE t ALTER integral TYPE regtype;							-- noop
- DEBUG:  Rewriting table "t"
- DEBUG:  Rebuilding index "t_rational_key"
- DEBUG:  Rebuilding index "t_daytimetz_key"
- DEBUG:  Rebuilding index "t_daytime_key"
- DEBUG:  Rebuilding index "t_stamptz_key"
- DEBUG:  Rebuilding index "t_stamp_key"
- DEBUG:  Rebuilding index "t_timegap_key"
- DEBUG:  Rebuilding index "t_bits_key"
- DEBUG:  Rebuilding index "t_network_key"
- DEBUG:  Rebuilding index "t_string_idx"
- DEBUG:  Rebuilding index "t_string_idx1"
- DEBUG:  Rebuilding index "t_strarr_idx"
- DEBUG:  Rebuilding index "t_square_idx"
- DEBUG:  Rebuilding index "t_touchy_f_idx"
- DEBUG:  Rebuilding index "t_expr_idx"
- DEBUG:  Rebuilding index "t_constraint4_key"
  DEBUG:  Rebuilding index "t_integral_key"
  ALTER TABLE t ALTER integral TYPE int8;								-- rewrite
  DEBUG:  Rewriting table "t"
--- 1928,1937 ----
***************
*** 2121,2142 **** DEBUG:  Rebuilding index "t_constraint4_key"
  DEBUG:  Rebuilding index "t_integral_key"
  DEBUG:  Rebuilding index "t_rational_key"
  ALTER TABLE t ALTER rational TYPE numeric;							-- noop
- DEBUG:  Rewriting table "t"
- DEBUG:  Rebuilding index "t_daytimetz_key"
- DEBUG:  Rebuilding index "t_daytime_key"
- DEBUG:  Rebuilding index "t_stamptz_key"
- DEBUG:  Rebuilding index "t_stamp_key"
- DEBUG:  Rebuilding index "t_timegap_key"
- DEBUG:  Rebuilding index "t_bits_key"
- DEBUG:  Rebuilding index "t_network_key"
- DEBUG:  Rebuilding index "t_string_idx"
- DEBUG:  Rebuilding index "t_string_idx1"
- DEBUG:  Rebuilding index "t_strarr_idx"
- DEBUG:  Rebuilding index "t_square_idx"
- DEBUG:  Rebuilding index "t_touchy_f_idx"
- DEBUG:  Rebuilding index "t_expr_idx"
- DEBUG:  Rebuilding index "t_constraint4_key"
- DEBUG:  Rebuilding index "t_integral_key"
  DEBUG:  Rebuilding index "t_rational_key"
  ALTER TABLE t ALTER rational TYPE numeric(5,4);						-- verify-e
  DEBUG:  Rewriting table "t"
--- 2042,2047 ----
***************
*** 2186,2245 **** ALTER TABLE t ALTER string TYPE shortdom;							-- rewrite-e
  DEBUG:  Rewriting table "t"
  ERROR:  value too long for type character varying(1)
  ALTER TABLE t ALTER string TYPE checkdom;							-- verify
- DEBUG:  Rewriting table "t"
- DEBUG:  Rebuilding index "t_daytimetz_key"
- DEBUG:  Rebuilding index "t_daytime_key"
- DEBUG:  Rebuilding index "t_stamptz_key"
- DEBUG:  Rebuilding index "t_stamp_key"
- DEBUG:  Rebuilding index "t_timegap_key"
- DEBUG:  Rebuilding index "t_bits_key"
- DEBUG:  Rebuilding index "t_network_key"
- DEBUG:  Rebuilding index "t_strarr_idx"
- DEBUG:  Rebuilding index "t_square_idx"
- DEBUG:  Rebuilding index "t_touchy_f_idx"
- DEBUG:  Rebuilding index "t_expr_idx"
- DEBUG:  Rebuilding index "t_constraint4_key"
- DEBUG:  Rebuilding index "t_integral_key"
- DEBUG:  Rebuilding index "t_rational_key"
  DEBUG:  Rebuilding index "t_string_idx1"
  DEBUG:  Rebuilding index "t_string_idx"
  ALTER TABLE t ALTER string TYPE faildom;							-- verify-e
! DEBUG:  Rewriting table "t"
  ERROR:  value for domain faildom violates check constraint "faildom_check"
  ALTER TABLE t ALTER string TYPE loosedom;							-- noop
- DEBUG:  Rewriting table "t"
- DEBUG:  Rebuilding index "t_daytimetz_key"
- DEBUG:  Rebuilding index "t_daytime_key"
- DEBUG:  Rebuilding index "t_stamptz_key"
- DEBUG:  Rebuilding index "t_stamp_key"
- DEBUG:  Rebuilding index "t_timegap_key"
- DEBUG:  Rebuilding index "t_bits_key"
- DEBUG:  Rebuilding index "t_network_key"
- DEBUG:  Rebuilding index "t_strarr_idx"
- DEBUG:  Rebuilding index "t_square_idx"
- DEBUG:  Rebuilding index "t_touchy_f_idx"
- DEBUG:  Rebuilding index "t_expr_idx"
- DEBUG:  Rebuilding index "t_constraint4_key"
- DEBUG:  Rebuilding index "t_integral_key"
- DEBUG:  Rebuilding index "t_rational_key"
  DEBUG:  Rebuilding index "t_string_idx"
  DEBUG:  Rebuilding index "t_string_idx1"
  ALTER TABLE t ALTER string TYPE text;								-- noop
- DEBUG:  Rewriting table "t"
- DEBUG:  Rebuilding index "t_daytimetz_key"
- DEBUG:  Rebuilding index "t_daytime_key"
- DEBUG:  Rebuilding index "t_stamptz_key"
- DEBUG:  Rebuilding index "t_stamp_key"
- DEBUG:  Rebuilding index "t_timegap_key"
- DEBUG:  Rebuilding index "t_bits_key"
- DEBUG:  Rebuilding index "t_network_key"
- DEBUG:  Rebuilding index "t_strarr_idx"
- DEBUG:  Rebuilding index "t_square_idx"
- DEBUG:  Rebuilding index "t_touchy_f_idx"
- DEBUG:  Rebuilding index "t_expr_idx"
- DEBUG:  Rebuilding index "t_constraint4_key"
- DEBUG:  Rebuilding index "t_integral_key"
- DEBUG:  Rebuilding index "t_rational_key"
  DEBUG:  Rebuilding index "t_string_idx1"
  DEBUG:  Rebuilding index "t_string_idx"
  ALTER TABLE t ALTER string TYPE varchar(20);						-- rewrite-v
--- 2091,2108 ----
  DEBUG:  Rewriting table "t"
  ERROR:  value too long for type character varying(1)
  ALTER TABLE t ALTER string TYPE checkdom;							-- verify
  DEBUG:  Rebuilding index "t_string_idx1"
  DEBUG:  Rebuilding index "t_string_idx"
+ DEBUG:  Verifying table "t"
  ALTER TABLE t ALTER string TYPE faildom;							-- verify-e
! DEBUG:  Rebuilding index "t_string_idx"
! DEBUG:  Rebuilding index "t_string_idx1"
! DEBUG:  Verifying table "t"
  ERROR:  value for domain faildom violates check constraint "faildom_check"
  ALTER TABLE t ALTER string TYPE loosedom;							-- noop
  DEBUG:  Rebuilding index "t_string_idx"
  DEBUG:  Rebuilding index "t_string_idx1"
  ALTER TABLE t ALTER string TYPE text;								-- noop
  DEBUG:  Rebuilding index "t_string_idx1"
  DEBUG:  Rebuilding index "t_string_idx"
  ALTER TABLE t ALTER string TYPE varchar(20);						-- rewrite-v
***************
*** 2854,2875 **** DEBUG:  Rebuilding index "t_stamp_key"
  DEBUG:  Rebuilding index "t_timegap_key"
  DEBUG:  Rebuilding index "t_bits_key"
  ALTER TABLE t ALTER network TYPE inet;								-- noop
- DEBUG:  Rewriting table "t"
- DEBUG:  Rebuilding index "t_strarr_idx"
- DEBUG:  Rebuilding index "t_square_idx"
- DEBUG:  Rebuilding index "t_touchy_f_idx"
- DEBUG:  Rebuilding index "t_expr_idx"
- DEBUG:  Rebuilding index "t_constraint4_key"
- DEBUG:  Rebuilding index "t_integral_key"
- DEBUG:  Rebuilding index "t_rational_key"
- DEBUG:  Rebuilding index "t_string_idx1"
- DEBUG:  Rebuilding index "t_string_idx"
- DEBUG:  Rebuilding index "t_daytimetz_key"
- DEBUG:  Rebuilding index "t_daytime_key"
- DEBUG:  Rebuilding index "t_stamptz_key"
- DEBUG:  Rebuilding index "t_stamp_key"
- DEBUG:  Rebuilding index "t_timegap_key"
- DEBUG:  Rebuilding index "t_bits_key"
  DEBUG:  Rebuilding index "t_network_key"
  ALTER TABLE t ALTER network TYPE cidr;								-- rewrite-v
  DEBUG:  Rewriting table "t"
--- 2717,2722 ----
***************
*** 2890,2912 **** DEBUG:  Rebuilding index "t_timegap_key"
  DEBUG:  Rebuilding index "t_bits_key"
  DEBUG:  Rebuilding index "t_network_key"
  ALTER TABLE t ALTER document TYPE text;								-- noop
- DEBUG:  Rewriting table "t"
- DEBUG:  Rebuilding index "t_strarr_idx"
- DEBUG:  Rebuilding index "t_square_idx"
- DEBUG:  Rebuilding index "t_touchy_f_idx"
- DEBUG:  Rebuilding index "t_expr_idx"
- DEBUG:  Rebuilding index "t_constraint4_key"
- DEBUG:  Rebuilding index "t_integral_key"
- DEBUG:  Rebuilding index "t_rational_key"
- DEBUG:  Rebuilding index "t_string_idx1"
- DEBUG:  Rebuilding index "t_string_idx"
- DEBUG:  Rebuilding index "t_daytimetz_key"
- DEBUG:  Rebuilding index "t_daytime_key"
- DEBUG:  Rebuilding index "t_stamptz_key"
- DEBUG:  Rebuilding index "t_stamp_key"
- DEBUG:  Rebuilding index "t_timegap_key"
- DEBUG:  Rebuilding index "t_bits_key"
- DEBUG:  Rebuilding index "t_network_key"
  ALTER TABLE t ALTER document TYPE xml USING document::xml;			-- verify
  DEBUG:  Rewriting table "t"
  DEBUG:  Rebuilding index "t_strarr_idx"
--- 2737,2742 ----
diff --git a/src/test/regress/sql/alter_table.sqindex 1818010..baa3935 100644
*** a/src/test/regress/sql/alter_table.sql
--- b/src/test/regress/sql/alter_table.sql
***************
*** 1095,1101 **** drop table another;
  -- table's row type
  create table tab1 (a int, b text);
  create table tab2 (x int, y tab1);
! alter table tab1 alter column b type varchar; -- fails
  alter table tab1 add check (b <> 'foo');
  alter table tab1 add c int not null;
  alter table tab1 add d int not null default 1;
--- 1095,1102 ----
  -- table's row type
  create table tab1 (a int, b text);
  create table tab2 (x int, y tab1);
! alter table tab1 alter column b type varchar;
! alter table tab1 alter column b type varchar(1); -- fails
  alter table tab1 add check (b <> 'foo');
  alter table tab1 add c int not null;
  alter table tab1 add d int not null default 1;
#7Robert Haas
robertmhaas@gmail.com
In reply to: Noah Misch (#6)
1 attachment(s)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Mon, Jan 24, 2011 at 7:10 PM, Noah Misch <noah@leadboat.com> wrote:

* at1.1-default-composite.patch
Remove the error when the user adds a column with a default value to a table
whose rowtype is used in a column elsewhere.

Can we fix this without moving the logic around quite so much? I'm
worried that could introduce bugs.

It strikes me that the root of the problem here is that this test is
subtly wrong:

if (newrel)
find_composite_type_dependencies(oldrel->rd_rel->reltype,

RelationGetRelationName(oldrel),

NULL);

So what this is saying is: If the user has performed an operation that
requires a rewrite, then we can't carry out that operation if the
rowtype is used elsewhere, because we wouldn't be able to propagate
the rewrite to those other objects. That's correct, unless the
operation in question is one which isn't supported by composite types
anyway. We trigger a rewrite if there is a has-OIDs change or if
tab->newvals contains any elements, which can happen if either there
is a type change or if a column with a default is added. So it seems
to me that we could fix this with something like the attached.
Thoughts?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Attachments:

defaults-are-not-so-evil.patchapplication/octet-stream; name=defaults-are-not-so-evil.patchDownload
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a9bb835..e5e81c5 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -142,6 +142,7 @@ typedef struct AlteredTableInfo
 	List	   *newvals;		/* List of NewColumnValue */
 	bool		new_notnull;	/* T if we added new NOT NULL constraints */
 	bool		new_changeoids; /* T if we added/dropped the OID column */
+	bool		new_changetypes; /* T if we changed column types */
 	Oid			newTableSpace;	/* new tablespace; 0 means no change */
 	/* Objects to rebuild after completing ALTER TYPE operations */
 	List	   *changedConstraintOids;	/* OIDs of constraints to rebuild */
@@ -3366,14 +3367,14 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
 	}
 
 	/*
-	 * If we need to rewrite the table, the operation has to be propagated to
-	 * tables that use this table's rowtype as a column type.
+	 * If we change column data types or add/remove OIDs, the operation has to
+	 * be propagated to tables that use this table's rowtype as a column type.
 	 *
 	 * (Eventually this will probably become true for scans as well, but at
 	 * the moment a composite type does not enforce any constraints, so it's
 	 * not necessary/appropriate to enforce them just during ALTER.)
 	 */
-	if (newrel)
+	if (tab->new_changetypes || tab->new_changeoids)
 		find_composite_type_dependencies(oldrel->rd_rel->reltype,
 										 RelationGetRelationName(oldrel),
 										 NULL);
@@ -6347,6 +6348,7 @@ ATPrepAlterColumnType(List **wqueue,
 		newval->expr = (Expr *) transform;
 
 		tab->newvals = lappend(tab->newvals, newval);
+		tab->new_changetypes = true;
 	}
 	else if (tab->relkind == RELKIND_FOREIGN_TABLE)
 	{
diff --git a/src/test/regress/expected/rowtypes.out b/src/test/regress/expected/rowtypes.out
index e5cd714..3d5f4d6 100644
--- a/src/test/regress/expected/rowtypes.out
+++ b/src/test/regress/expected/rowtypes.out
@@ -82,11 +82,14 @@ select * from people;
  (Joe,Blow) | 01-10-1984
 (1 row)
 
--- at the moment this will not work due to ALTER TABLE inadequacy:
+-- the default doesn't need to propagate through to the rowtypes, so this is OK
 alter table fullname add column suffix text default '';
-ERROR:  cannot alter table "fullname" because column "people"."fn" uses its rowtype
--- but this should work:
+alter table fullname drop column suffix;
+-- this one, without a default, is OK too
 alter table fullname add column suffix text default null;
+-- but this should fail, due to ALTER TABLE inadequacy
+alter table fullname alter column suffix set data type integer using null;
+ERROR:  cannot alter table "fullname" because column "people"."fn" uses its rowtype
 select * from people;
      fn      |     bd     
 -------------+------------
diff --git a/src/test/regress/sql/rowtypes.sql b/src/test/regress/sql/rowtypes.sql
index 9041df1..257213d 100644
--- a/src/test/regress/sql/rowtypes.sql
+++ b/src/test/regress/sql/rowtypes.sql
@@ -45,12 +45,16 @@ insert into people values ('(Joe,Blow)', '1984-01-10');
 
 select * from people;
 
--- at the moment this will not work due to ALTER TABLE inadequacy:
+-- the default doesn't need to propagate through to the rowtypes, so this is OK
 alter table fullname add column suffix text default '';
+alter table fullname drop column suffix;
 
--- but this should work:
+-- this one, without a default, is OK too
 alter table fullname add column suffix text default null;
 
+-- but this should fail, due to ALTER TABLE inadequacy
+alter table fullname alter column suffix set data type integer using null;
+
 select * from people;
 
 -- test insertion/updating of subfields
#8Robert Haas
robertmhaas@gmail.com
In reply to: Noah Misch (#6)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Mon, Jan 24, 2011 at 7:10 PM, Noah Misch <noah@leadboat.com> wrote:

* at1.2-doc-set-data-type.patch
The documentation used "ALTER TYPE" when it meant "SET DATA TYPE", a subform of
"ALTER TABLE" or "ALTER FOREIGN TABLE".  Fixes just that.

Committed this part. For reasons involving me being tired, I
initially thought that only the first part was correct, which is why I
did it as two commits. But it's obviously right, so now it's all
committed. I back-patched the ALTER TABLE part to 9.0.X so it'll show
up in the web site docs after the next minor release.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#9Noah Misch
noah@leadboat.com
In reply to: Robert Haas (#7)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Tue, Jan 25, 2011 at 06:40:08PM -0500, Robert Haas wrote:

On Mon, Jan 24, 2011 at 7:10 PM, Noah Misch <noah@leadboat.com> wrote:

* at1.1-default-composite.patch
Remove the error when the user adds a column with a default value to a table
whose rowtype is used in a column elsewhere.

Can we fix this without moving the logic around quite so much? I'm
worried that could introduce bugs.

It strikes me that the root of the problem here is that this test is
subtly wrong:

if (newrel)
find_composite_type_dependencies(oldrel->rd_rel->reltype,

RelationGetRelationName(oldrel),

NULL);

Correct.

So it seems
to me that we could fix this with something like the attached.
Thoughts?

I'm fine with this patch. A few notes based on its context in the larger
project:

--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c

@@ -3366,14 +3367,14 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
}

/*
-	 * If we need to rewrite the table, the operation has to be propagated to
-	 * tables that use this table's rowtype as a column type.
+	 * If we change column data types or add/remove OIDs, the operation has to
+	 * be propagated to tables that use this table's rowtype as a column type.
*
* (Eventually this will probably become true for scans as well, but at
* the moment a composite type does not enforce any constraints, so it's
* not necessary/appropriate to enforce them just during ALTER.)
*/
-	if (newrel)
+	if (tab->new_changetypes || tab->new_changeoids)

The next patch removed new_changeoids, so we would instead be keeping it with
this as the only place referencing it.

find_composite_type_dependencies(oldrel->rd_rel->reltype,
RelationGetRelationName(oldrel),
NULL);
@@ -6347,6 +6348,7 @@ ATPrepAlterColumnType(List **wqueue,
newval->expr = (Expr *) transform;

tab->newvals = lappend(tab->newvals, newval);
+ tab->new_changetypes = true;

The at2v2 patch would then morph to do something like:

if (worklevel != WORK_NONE)
tab->new_changetypes = true;

That weakens the name "new_changetypes" a bit.

#10Robert Haas
robertmhaas@gmail.com
In reply to: Noah Misch (#9)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Tue, Jan 25, 2011 at 10:22 PM, Noah Misch <noah@leadboat.com> wrote:

I'm fine with this patch.

OK, committed.

The next patch removed new_changeoids, so we would instead be keeping it with
this as the only place referencing it.

[...]

The at2v2 patch would then morph to do something like:

if (worklevel != WORK_NONE)
       tab->new_changetypes = true;

Well, I'm not too keen on either of those things. The second one,
especially, looks like the sense of the Boolean is clearly being
abused, so either the Boolean needs to be renamed or some other change
is required.

I'd also suggest that this big if-block you changed to a case
statement could just as well stay as an if-block. There are only
three cases, and we want to avoid rearranging things more than
necessary. It complicates both review and back-patching to no good
end.

I think you should collect up what's left of ALTER TABLE 0 and the
stuff on this thread, rebase it, and submit it as a single patch on
this thread that applies directly against the master branch. We may
decide to split it back up again in some other way, but I think the
current division isn't actually buying us much.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#11Noah Misch
noah@leadboat.com
In reply to: Robert Haas (#10)
1 attachment(s)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Wed, Jan 26, 2011 at 07:31:40AM -0500, Robert Haas wrote:

I'd also suggest that this big if-block you changed to a case
statement could just as well stay as an if-block. There are only
three cases, and we want to avoid rearranging things more than
necessary. It complicates both review and back-patching to no good
end.

Okay. I've also left out the large reindent in ATRewriteTable for now. Easy to
re-add it later if desired.

I think you should collect up what's left of ALTER TABLE 0 and the
stuff on this thread, rebase it, and submit it as a single patch on
this thread that applies directly against the master branch. We may
decide to split it back up again in some other way, but I think the
current division isn't actually buying us much.

Done as attached. This preserves compatibility with our current handling of
composite type dependencies. The rest you've seen before.

Thanks,
nm

Attachments:

at0,2v4-skip-nowork.patchtext/plain; charset=us-asciiDownload
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 1d52be6..4efb02e 100644
*** a/doc/src/sgml/ref/alter_table.sgml
--- b/doc/src/sgml/ref/alter_table.sgml
***************
*** 403,411 **** ALTER TABLE <replaceable class="PARAMETER">name</replaceable>
        for details on the available parameters.  Note that the table contents
        will not be modified immediately by this command; depending on the
        parameter you might need to rewrite the table to get the desired effects.
!       That can be done with <xref linkend="SQL-CLUSTER">
!       or one of the forms of <command>ALTER
!       TABLE</> that forces a table rewrite.
       </para>
  
       <note>
--- 403,411 ----
        for details on the available parameters.  Note that the table contents
        will not be modified immediately by this command; depending on the
        parameter you might need to rewrite the table to get the desired effects.
!       That can be done with <link linkend="SQL-VACUUM">VACUUM
!       FULL</>, <xref linkend="SQL-CLUSTER"> or one of the forms
!       of <command>ALTER TABLE</> that forces a table rewrite.
       </para>
  
       <note>
***************
*** 746,759 **** ALTER TABLE <replaceable class="PARAMETER">name</replaceable>
     </para>
  
     <para>
!     Adding a column with a non-null default or changing the type of an
!     existing column will require the entire table and indexes to be rewritten.
!     This might take a significant amount of time for a large table; and it will
!     temporarily require double the disk space.  Adding or removing a system
      <literal>oid</> column likewise requires rewriting the entire table.
     </para>
  
     <para>
      Adding a <literal>CHECK</> or <literal>NOT NULL</> constraint requires
      scanning the table to verify that existing rows meet the constraint.
     </para>
--- 746,770 ----
     </para>
  
     <para>
!     Adding a column with a non-null default will rewrite the entire table and
!     all indexes.  Changing the type of an existing column will do the same
!     unless a binary-coercible cast implements the type conversion.  Refer to
!     <xref linkend="SQL-CREATECAST"> for further information.  A rewrite might
!     take a significant amount of time for a large table, and it will temporarily
!     require double the disk space.  Adding or removing a system
      <literal>oid</> column likewise requires rewriting the entire table.
     </para>
  
     <para>
+     Similar to the behavior of <link linkend="SQL-VACUUM">VACUUM FULL</>, the
+     rewriting process eliminates any dead space in the table.  Prior
+     to <productname>PostgreSQL</> 9.0, <literal>SET DATA TYPE</>
+     outpaced <literal>VACUUM FULL</> at this task, so it was useful even absent
+     the need for a column type change.  This speed advantage no longer holds,
+     and <literal>SET DATA TYPE</> may not even rewrite the table.
+    </para>
+ 
+    <para>
      Adding a <literal>CHECK</> or <literal>NOT NULL</> constraint requires
      scanning the table to verify that existing rows meet the constraint.
     </para>
***************
*** 777,797 **** ALTER TABLE <replaceable class="PARAMETER">name</replaceable>
     </para>
  
     <para>
-     The fact that <literal>SET DATA TYPE</> requires rewriting the whole table
-     is sometimes an advantage, because the rewriting process eliminates
-     any dead space in the table.  For example, to reclaim the space occupied
-     by a dropped column immediately, the fastest way is:
- <programlisting>
- ALTER TABLE table ALTER COLUMN anycol TYPE anytype;
- </programlisting>
-     where <literal>anycol</> is any remaining table column and
-     <literal>anytype</> is the same type that column already has.
-     This results in no semantically-visible change in the table,
-     but the command forces rewriting, which gets rid of no-longer-useful
-     data.
-    </para>
- 
-    <para>
      The <literal>USING</literal> option of <literal>SET DATA TYPE</> can actually
      specify any expression involving the old values of the row; that is, it
      can refer to other columns as well as the one being converted.  This allows
--- 788,793 ----
diff --git a/src/backend/catalog/index.cindex 5254b65..411de09 100644
*** a/src/backend/catalog/index.c
--- b/src/backend/catalog/index.c
***************
*** 1673,1678 **** index_build(Relation heapRelation,
--- 1673,1688 ----
  	procedure = indexRelation->rd_am->ambuild;
  	Assert(RegProcedureIsValid(procedure));
  
+ 	if (indexInfo->ii_ToastForRelName != NULL)
+ 		ereport(DEBUG1,
+ 				(errmsg("building TOAST index for table \"%s\"",
+ 						indexInfo->ii_ToastForRelName)));
+ 	else
+ 		ereport(DEBUG1,
+ 				(errmsg("building index \"%s\" on table \"%s\"",
+ 						RelationGetRelationName(indexRelation),
+ 						RelationGetRelationName(heapRelation))));
+ 
  	/*
  	 * Switch to the table owner's userid, so that any index functions are run
  	 * as that user.  Also lock down security-restricted operations and
***************
*** 2663,2669 **** IndexGetRelation(Oid indexId)
   * reindex_index - This routine is used to recreate a single index
   */
  void
! reindex_index(Oid indexId, bool skip_constraint_checks)
  {
  	Relation	iRel,
  				heapRelation,
--- 2673,2680 ----
   * reindex_index - This routine is used to recreate a single index
   */
  void
! reindex_index(Oid indexId, const char *toastFor,
! 			  bool skip_constraint_checks)
  {
  	Relation	iRel,
  				heapRelation,
***************
*** 2721,2726 **** reindex_index(Oid indexId, bool skip_constraint_checks)
--- 2732,2740 ----
  			indexInfo->ii_ExclusionStrats = NULL;
  		}
  
+ 		/* Pass the name of relation this TOAST index serves, if any. */
+ 		indexInfo->ii_ToastForRelName = toastFor;
+ 
  		/* We'll build a new physical relation for the index */
  		RelationSetNewRelfilenode(iRel, InvalidTransactionId);
  
***************
*** 2780,2785 **** reindex_index(Oid indexId, bool skip_constraint_checks)
--- 2794,2802 ----
   * reindex_relation - This routine is used to recreate all indexes
   * of a relation (and optionally its toast relation too, if any).
   *
+  * If this is a TOAST relation, toastFor may bear the parent relation name,
+  * facilitating improved messages.
+  *
   * "flags" can include REINDEX_SUPPRESS_INDEX_USE and REINDEX_CHECK_CONSTRAINTS.
   *
   * If flags has REINDEX_SUPPRESS_INDEX_USE, the relation was just completely
***************
*** 2802,2808 **** reindex_index(Oid indexId, bool skip_constraint_checks)
   * CommandCounterIncrement will occur after each index rebuild.
   */
  bool
! reindex_relation(Oid relid, bool toast_too, int flags)
  {
  	Relation	rel;
  	Oid			toast_relid;
--- 2819,2826 ----
   * CommandCounterIncrement will occur after each index rebuild.
   */
  bool
! reindex_relation(Oid relid, const char *toastFor,
! 				 bool toast_too, int flags)
  {
  	Relation	rel;
  	Oid			toast_relid;
***************
*** 2879,2885 **** reindex_relation(Oid relid, bool toast_too, int flags)
  			if (is_pg_class)
  				RelationSetIndexList(rel, doneIndexes, InvalidOid);
  
! 			reindex_index(indexOid, !(flags & REINDEX_CHECK_CONSTRAINTS));
  
  			CommandCounterIncrement();
  
--- 2897,2903 ----
  			if (is_pg_class)
  				RelationSetIndexList(rel, doneIndexes, InvalidOid);
  
! 			reindex_index(indexOid, toastFor, !(flags & REINDEX_CHECK_CONSTRAINTS));
  
  			CommandCounterIncrement();
  
***************
*** 2902,2912 **** reindex_relation(Oid relid, bool toast_too, int flags)
  	if (is_pg_class)
  		RelationSetIndexList(rel, indexIds, ClassOidIndexId);
  
- 	/*
- 	 * Close rel, but continue to hold the lock.
- 	 */
- 	heap_close(rel, NoLock);
- 
  	result = (indexIds != NIL);
  
  	/*
--- 2920,2925 ----
***************
*** 2916,2922 **** reindex_relation(Oid relid, bool toast_too, int flags)
  	 */
  	Assert(!(toast_too && (flags & REINDEX_SUPPRESS_INDEX_USE)));
  	if (toast_too && OidIsValid(toast_relid))
! 		result |= reindex_relation(toast_relid, false, flags);
  
  	return result;
  }
--- 2929,2941 ----
  	 */
  	Assert(!(toast_too && (flags & REINDEX_SUPPRESS_INDEX_USE)));
  	if (toast_too && OidIsValid(toast_relid))
! 		result |= reindex_relation(toast_relid, RelationGetRelationName(rel),
! 								   false, flags);
! 
! 	/*
! 	 * Close rel, but continue to hold the lock.
! 	 */
! 	heap_close(rel, NoLock);
  
  	return result;
  }
diff --git a/src/backend/catalog/tindex c4be3a9..f60b7c1 100644
*** a/src/backend/catalog/toasting.c
--- b/src/backend/catalog/toasting.c
***************
*** 36,43 **** extern Oid	binary_upgrade_next_toast_pg_class_oid;
  
  Oid			binary_upgrade_next_toast_pg_type_oid = InvalidOid;
  
! static bool create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
! 				   Datum reloptions);
  static bool needs_toast_table(Relation rel);
  
  
--- 36,43 ----
  
  Oid			binary_upgrade_next_toast_pg_type_oid = InvalidOid;
  
! static bool create_toast_table(Relation rel, const char *finalRelName,
! 				   Oid toastOid, Oid toastIndexOid, Datum reloptions);
  static bool needs_toast_table(Relation rel);
  
  
***************
*** 46,51 **** static bool needs_toast_table(Relation rel);
--- 46,54 ----
   *		If the table needs a toast table, and doesn't already have one,
   *		then create a toast table for it.
   *
+  * make_new_heap fills finalRelName, so messages display the permanent table
+  * name, not the rewrite-temporary name.  Most callers should pass NULL.
+  *
   * reloptions for the toast table can be passed, too.  Pass (Datum) 0
   * for default reloptions.
   *
***************
*** 54,60 **** static bool needs_toast_table(Relation rel);
   * to end with CommandCounterIncrement if it makes any changes.
   */
  void
! AlterTableCreateToastTable(Oid relOid, Datum reloptions)
  {
  	Relation	rel;
  
--- 57,64 ----
   * to end with CommandCounterIncrement if it makes any changes.
   */
  void
! AlterTableCreateToastTable(Oid relOid, const char *finalRelName,
! 						   Datum reloptions)
  {
  	Relation	rel;
  
***************
*** 66,72 **** AlterTableCreateToastTable(Oid relOid, Datum reloptions)
  	rel = heap_open(relOid, AccessExclusiveLock);
  
  	/* create_toast_table does all the work */
! 	(void) create_toast_table(rel, InvalidOid, InvalidOid, reloptions);
  
  	heap_close(rel, NoLock);
  }
--- 70,77 ----
  	rel = heap_open(relOid, AccessExclusiveLock);
  
  	/* create_toast_table does all the work */
! 	(void) create_toast_table(rel, finalRelName,
! 							  InvalidOid, InvalidOid, reloptions);
  
  	heap_close(rel, NoLock);
  }
***************
*** 92,98 **** BootstrapToastTable(char *relName, Oid toastOid, Oid toastIndexOid)
  						relName)));
  
  	/* create_toast_table does all the work */
! 	if (!create_toast_table(rel, toastOid, toastIndexOid, (Datum) 0))
  		elog(ERROR, "\"%s\" does not require a toast table",
  			 relName);
  
--- 97,103 ----
  						relName)));
  
  	/* create_toast_table does all the work */
! 	if (!create_toast_table(rel, NULL, toastOid, toastIndexOid, (Datum) 0))
  		elog(ERROR, "\"%s\" does not require a toast table",
  			 relName);
  
***************
*** 104,114 **** BootstrapToastTable(char *relName, Oid toastOid, Oid toastIndexOid)
   * create_toast_table --- internal workhorse
   *
   * rel is already opened and exclusive-locked
   * toastOid and toastIndexOid are normally InvalidOid, but during
   * bootstrap they can be nonzero to specify hand-assigned OIDs
   */
  static bool
! create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid, Datum reloptions)
  {
  	Oid			relOid = RelationGetRelid(rel);
  	HeapTuple	reltup;
--- 109,121 ----
   * create_toast_table --- internal workhorse
   *
   * rel is already opened and exclusive-locked
+  * finalRelName is normally NULL; make_new_heap overrides it
   * toastOid and toastIndexOid are normally InvalidOid, but during
   * bootstrap they can be nonzero to specify hand-assigned OIDs
   */
  static bool
! create_toast_table(Relation rel, const char *finalRelName,
! 				   Oid toastOid, Oid toastIndexOid, Datum reloptions)
  {
  	Oid			relOid = RelationGetRelid(rel);
  	HeapTuple	reltup;
***************
*** 259,264 **** create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid, Datum reloptio
--- 266,273 ----
  	indexInfo->ii_ExclusionOps = NULL;
  	indexInfo->ii_ExclusionProcs = NULL;
  	indexInfo->ii_ExclusionStrats = NULL;
+ 	indexInfo->ii_ToastForRelName
+ 		= finalRelName != NULL ? finalRelName : RelationGetRelationName(rel);
  	indexInfo->ii_Unique = true;
  	indexInfo->ii_ReadyForInserts = true;
  	indexInfo->ii_Concurrent = false;
diff --git a/src/backend/commands/cluindex 59a4394..87af84f 100644
*** a/src/backend/commands/cluster.c
--- b/src/backend/commands/cluster.c
***************
*** 681,687 **** make_new_heap(Oid OIDOldHeap, Oid NewTableSpace)
  		if (isNull)
  			reloptions = (Datum) 0;
  
! 		AlterTableCreateToastTable(OIDNewHeap, reloptions);
  
  		ReleaseSysCache(tuple);
  	}
--- 681,688 ----
  		if (isNull)
  			reloptions = (Datum) 0;
  
! 		AlterTableCreateToastTable(OIDNewHeap, RelationGetRelationName(OldHeap),
! 								   reloptions);
  
  		ReleaseSysCache(tuple);
  	}
***************
*** 1400,1406 **** finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
  	reindex_flags = REINDEX_SUPPRESS_INDEX_USE;
  	if (check_constraints)
  		reindex_flags |= REINDEX_CHECK_CONSTRAINTS;
! 	reindex_relation(OIDOldHeap, false, reindex_flags);
  
  	/* Destroy new heap with old filenode */
  	object.classId = RelationRelationId;
--- 1401,1407 ----
  	reindex_flags = REINDEX_SUPPRESS_INDEX_USE;
  	if (check_constraints)
  		reindex_flags |= REINDEX_CHECK_CONSTRAINTS;
! 	reindex_relation(OIDOldHeap, NULL, false, reindex_flags);
  
  	/* Destroy new heap with old filenode */
  	object.classId = RelationRelationId;
diff --git a/src/backend/commands/indindex 94ed437..1dab757 100644
*** a/src/backend/commands/indexcmds.c
--- b/src/backend/commands/indexcmds.c
***************
*** 1482,1488 **** ReindexIndex(RangeVar *indexRelation)
  
  	ReleaseSysCache(tuple);
  
! 	reindex_index(indOid, false);
  }
  
  /*
--- 1482,1488 ----
  
  	ReleaseSysCache(tuple);
  
! 	reindex_index(indOid, NULL, false);
  }
  
  /*
***************
*** 1514,1520 **** ReindexTable(RangeVar *relation)
  
  	ReleaseSysCache(tuple);
  
! 	if (!reindex_relation(heapOid, true, 0))
  		ereport(NOTICE,
  				(errmsg("table \"%s\" has no indexes",
  						relation->relname)));
--- 1514,1520 ----
  
  	ReleaseSysCache(tuple);
  
! 	if (!reindex_relation(heapOid, NULL, true, 0))
  		ereport(NOTICE,
  				(errmsg("table \"%s\" has no indexes",
  						relation->relname)));
***************
*** 1627,1633 **** ReindexDatabase(const char *databaseName, bool do_system, bool do_user)
  		StartTransactionCommand();
  		/* functions in indexes may want a snapshot set */
  		PushActiveSnapshot(GetTransactionSnapshot());
! 		if (reindex_relation(relid, true, 0))
  			ereport(NOTICE,
  					(errmsg("table \"%s.%s\" was reindexed",
  							get_namespace_name(get_rel_namespace(relid)),
--- 1627,1633 ----
  		StartTransactionCommand();
  		/* functions in indexes may want a snapshot set */
  		PushActiveSnapshot(GetTransactionSnapshot());
! 		if (reindex_relation(relid, NULL, true, 0))
  			ereport(NOTICE,
  					(errmsg("table \"%s.%s\" was reindexed",
  							get_namespace_name(get_rel_namespace(relid)),
diff --git a/src/backend/commands/tableindex 1ecba02..67fdafc 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 71,76 ****
--- 71,77 ----
  #include "storage/smgr.h"
  #include "utils/acl.h"
  #include "utils/builtins.h"
+ #include "utils/datum.h"
  #include "utils/fmgroids.h"
  #include "utils/inval.h"
  #include "utils/lsyscache.h"
***************
*** 129,134 **** static List *on_commits = NIL;
--- 130,138 ----
  #define AT_PASS_MISC			8		/* other stuff */
  #define AT_NUM_PASSES			9
  
+ /* Level of effort required in Phase 3 (ATRewriteTables). */
+ typedef enum { WORK_NONE = 0, WORK_SCAN = 1, WORK_REWRITE = 2 } WorkLevel;
+ 
  typedef struct AlteredTableInfo
  {
  	/* Information saved before any work commences: */
***************
*** 142,147 **** typedef struct AlteredTableInfo
--- 146,152 ----
  	List	   *newvals;		/* List of NewColumnValue */
  	bool		new_notnull;	/* T if we added new NOT NULL constraints */
  	bool		new_changeoids; /* T if we added/dropped the OID column */
+ 	WorkLevel	worklevel;		/* How much work shall we do on the heap? */
  	Oid			newTableSpace;	/* new tablespace; 0 means no change */
  	/* Objects to rebuild after completing ALTER TYPE operations */
  	List	   *changedConstraintOids;	/* OIDs of constraints to rebuild */
***************
*** 1078,1084 **** ExecuteTruncate(TruncateStmt *stmt)
  			/*
  			 * Reconstruct the indexes to match, and we're done.
  			 */
! 			reindex_relation(heap_relid, true, 0);
  		}
  	}
  
--- 1083,1089 ----
  			/*
  			 * Reconstruct the indexes to match, and we're done.
  			 */
! 			reindex_relation(heap_relid, NULL, true, 0);
  		}
  	}
  
***************
*** 2990,2996 **** ATRewriteCatalogs(List **wqueue, LOCKMODE lockmode)
  			(tab->subcmds[AT_PASS_ADD_COL] ||
  			 tab->subcmds[AT_PASS_ALTER_TYPE] ||
  			 tab->subcmds[AT_PASS_COL_ATTRS]))
! 			AlterTableCreateToastTable(tab->relid, (Datum) 0);
  	}
  }
  
--- 2995,3001 ----
  			(tab->subcmds[AT_PASS_ADD_COL] ||
  			 tab->subcmds[AT_PASS_ALTER_TYPE] ||
  			 tab->subcmds[AT_PASS_COL_ATTRS]))
! 			AlterTableCreateToastTable(tab->relid, NULL, (Datum) 0);
  	}
  }
  
***************
*** 3193,3202 **** ATRewriteTables(List **wqueue, LOCKMODE lockmode)
  			continue;
  
  		/*
  		 * We only need to rewrite the table if at least one column needs to
  		 * be recomputed, or we are adding/removing the OID column.
  		 */
! 		if (tab->newvals != NIL || tab->new_changeoids)
  		{
  			/* Build a temporary relation and copy data */
  			Relation	OldHeap;
--- 3198,3232 ----
  			continue;
  
  		/*
+ 		 * Any operation has to be propagated to tables that use this table's
+ 		 * rowtype as a column type, but this is not yet implemented.  Reject
+ 		 * changes where this ommission would be particularly glaring: column
+ 		 * type changes, new columns with default values, and OIDs changes.
+ 		 *
+ 		 * (Eventually this will probably become true for scans as well, but at
+ 		 * the moment a composite type does not enforce any constraints, so it's
+ 		 * not necessary/appropriate to enforce them just during ALTER.)
+ 		 */
+ 		if (tab->newvals != NIL || tab->new_changeoids)
+ 		{
+ 			Relation	rel;
+ 
+ 			rel = heap_open(tab->relid, NoLock);
+ 			find_composite_type_dependencies(rel->rd_rel->reltype,
+ 											 RelationGetRelationName(rel),
+ 											 NULL);
+ 			heap_close(rel, NoLock);
+ 		}
+ 
+ 		/* New NOT NULL constraints always require a scan. */
+ 		if (tab->new_notnull)
+ 			tab->worklevel = Max(tab->worklevel, WORK_SCAN);
+ 
+ 		/*
  		 * We only need to rewrite the table if at least one column needs to
  		 * be recomputed, or we are adding/removing the OID column.
  		 */
! 		if (tab->worklevel == WORK_REWRITE)
  		{
  			/* Build a temporary relation and copy data */
  			Relation	OldHeap;
***************
*** 3263,3269 **** ATRewriteTables(List **wqueue, LOCKMODE lockmode)
  			 * Test the current data within the table against new constraints
  			 * generated by ALTER TABLE commands, but don't rebuild data.
  			 */
! 			if (tab->constraints != NIL || tab->new_notnull)
  				ATRewriteTable(tab, InvalidOid, lockmode);
  
  			/*
--- 3293,3299 ----
  			 * Test the current data within the table against new constraints
  			 * generated by ALTER TABLE commands, but don't rebuild data.
  			 */
! 			if (tab->worklevel == WORK_SCAN)
  				ATRewriteTable(tab, InvalidOid, lockmode);
  
  			/*
***************
*** 3332,3338 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  	Relation	newrel;
  	TupleDesc	oldTupDesc;
  	TupleDesc	newTupDesc;
- 	bool		needscan = false;
  	List	   *notnull_attrs;
  	int			i;
  	ListCell   *l;
--- 3362,3367 ----
***************
*** 3378,3396 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  	}
  
  	/*
- 	 * If we need to rewrite the table, the operation has to be propagated to
- 	 * tables that use this table's rowtype as a column type.
- 	 *
- 	 * (Eventually this will probably become true for scans as well, but at
- 	 * the moment a composite type does not enforce any constraints, so it's
- 	 * not necessary/appropriate to enforce them just during ALTER.)
- 	 */
- 	if (newrel)
- 		find_composite_type_dependencies(oldrel->rd_rel->reltype,
- 										 RelationGetRelationName(oldrel),
- 										 NULL);
- 
- 	/*
  	 * Generate the constraint and default execution states
  	 */
  
--- 3407,3412 ----
***************
*** 3404,3410 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  		switch (con->contype)
  		{
  			case CONSTR_CHECK:
- 				needscan = true;
  				con->qualstate = (List *)
  					ExecPrepareExpr((Expr *) con->qual, estate);
  				break;
--- 3420,3425 ----
***************
*** 3439,3450 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  				!newTupDesc->attrs[i]->attisdropped)
  				notnull_attrs = lappend_int(notnull_attrs, i);
  		}
- 		if (notnull_attrs)
- 			needscan = true;
  	}
  
! 	if (newrel || needscan)
! 	{
  		ExprContext *econtext;
  		Datum	   *values;
  		bool	   *isnull;
--- 3454,3462 ----
  				!newTupDesc->attrs[i]->attisdropped)
  				notnull_attrs = lappend_int(notnull_attrs, i);
  		}
  	}
  
! 	{							/* XXX reindent? */
  		ExprContext *econtext;
  		Datum	   *values;
  		bool	   *isnull;
***************
*** 3456,3461 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
--- 3468,3482 ----
  		List	   *dropped_attrs = NIL;
  		ListCell   *lc;
  
+ 		if (newrel)
+ 			ereport(DEBUG1,
+ 					(errmsg("rewriting table \"%s\"",
+ 							RelationGetRelationName(oldrel))));
+ 		else
+ 			ereport(DEBUG1,
+ 					(errmsg("verifying table \"%s\"",
+ 							RelationGetRelationName(oldrel))));
+ 
  		econtext = GetPerTupleExprContext(estate);
  
  		/*
***************
*** 3498,3515 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  
  		while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
  		{
! 			if (newrel)
  			{
  				Oid			tupOid = InvalidOid;
  
! 				/* Extract data from old tuple */
! 				heap_deform_tuple(tuple, oldTupDesc, values, isnull);
! 				if (oldTupDesc->tdhasoid)
! 					tupOid = HeapTupleGetOid(tuple);
! 
! 				/* Set dropped attributes to null in new tuple */
! 				foreach(lc, dropped_attrs)
! 					isnull[lfirst_int(lc)] = true;
  
  				/*
  				 * Process supplied expressions to replace selected columns.
--- 3519,3549 ----
  
  		while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
  		{
! 			/*
! 			 * If we're changing the TupleDesc, compute new tuple values using
! 			 * each transformation expression.  When rewriting, also form a new
! 			 * physical tuple.  In Assert-enabled builds, check for cases that
! 			 * should have been WORK_REWRITE by comparing the data.
! 			 */
! 			if (tab->newvals != NIL || tab->new_changeoids)
  			{
  				Oid			tupOid = InvalidOid;
  
! 				if (newrel
! #ifdef USE_ASSERT_CHECKING
! 					|| assert_enabled
! #endif
! 					)
! 				{
! 					/* Extract data from old tuple */
! 					heap_deform_tuple(tuple, oldTupDesc, values, isnull);
! 					if (oldTupDesc->tdhasoid)
! 						tupOid = HeapTupleGetOid(tuple);
! 
! 					/* Set dropped attributes to null in new tuple */
! 					foreach(lc, dropped_attrs)
! 						isnull[lfirst_int(lc)] = true;
! 				}
  
  				/*
  				 * Process supplied expressions to replace selected columns.
***************
*** 3526,3542 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  														  econtext,
  													 &isnull[ex->attnum - 1],
  														  NULL);
  				}
  
! 				/*
! 				 * Form the new tuple. Note that we don't explicitly pfree it,
! 				 * since the per-tuple memory context will be reset shortly.
! 				 */
! 				tuple = heap_form_tuple(newTupDesc, values, isnull);
  
! 				/* Preserve OID, if any */
! 				if (newTupDesc->tdhasoid)
! 					HeapTupleSetOid(tuple, tupOid);
  			}
  
  			/* Now check any constraints on the possibly-changed tuple */
--- 3560,3603 ----
  														  econtext,
  													 &isnull[ex->attnum - 1],
  														  NULL);
+ 
+ #ifdef USE_ASSERT_CHECKING
+ 					if (assert_enabled)
+ 					{
+ 						Datum		oldval = values[ex->attnum - 1];
+ 						bool		oldisnull = isnull[ex->attnum - 1];
+ 						Form_pg_attribute f = newTupDesc->attrs[ex->attnum - 1];
+ 
+ 						if (f->attbyval && f->attlen == -1)
+ 							oldval = PointerGetDatum(PG_DETOAST_DATUM(oldval));
+ 
+ 						/*
+ 						 * We don't detect the gross error of !newrel when the
+ 						 * typlen actually changed.  attbyval could differ in
+ 						 * theory, but we assume it does not.
+ 						 */
+ 						Assert(newrel ||
+ 							   (isnull[ex->attnum - 1] == oldisnull
+ 								&& (oldisnull ||
+ 									datumIsEqual(oldval,
+ 												 values[ex->attnum - 1],
+ 												 f->attbyval, f->attlen))));
+ 					}
+ #endif
  				}
  
! 				if (newrel)
! 				{
! 					/*
! 					 * Form the new tuple. Note that we don't explicitly pfree it,
! 					 * since the per-tuple memory context will be reset shortly.
! 					 */
! 					tuple = heap_form_tuple(newTupDesc, values, isnull);
  
! 					/* Preserve OID, if any */
! 					if (newTupDesc->tdhasoid)
! 						HeapTupleSetOid(tuple, tupOid);
! 				}
  			}
  
  			/* Now check any constraints on the possibly-changed tuple */
***************
*** 4283,4288 **** ATExecAddColumn(AlteredTableInfo *tab, Relation rel,
--- 4344,4350 ----
  			newval->expr = defval;
  
  			tab->newvals = lappend(tab->newvals, newval);
+ 			tab->worklevel = WORK_REWRITE;
  		}
  
  		/*
***************
*** 4299,4305 **** ATExecAddColumn(AlteredTableInfo *tab, Relation rel,
--- 4361,4370 ----
  	 * table to fix that.
  	 */
  	if (isOid)
+ 	{
  		tab->new_changeoids = true;
+ 		tab->worklevel = WORK_REWRITE;
+ 	}
  
  	/*
  	 * Add needed dependency entries for the new column.
***************
*** 4973,4978 **** ATExecDropColumn(List **wqueue, Relation rel, const char *colName,
--- 5038,5044 ----
  
  		/* Tell Phase 3 to physically remove the OID column */
  		tab->new_changeoids = true;
+ 		tab->worklevel = WORK_REWRITE;
  	}
  }
  
***************
*** 4996,5002 **** ATExecAddIndex(AlteredTableInfo *tab, Relation rel,
  	/* suppress schema rights check when rebuilding existing index */
  	check_rights = !is_rebuild;
  	/* skip index build if phase 3 will have to rewrite table anyway */
! 	skip_build = (tab->newvals != NIL);
  	/* suppress notices when rebuilding existing index */
  	quiet = is_rebuild;
  
--- 5062,5068 ----
  	/* suppress schema rights check when rebuilding existing index */
  	check_rights = !is_rebuild;
  	/* skip index build if phase 3 will have to rewrite table anyway */
! 	skip_build = (tab->worklevel == WORK_REWRITE);
  	/* suppress notices when rebuilding existing index */
  	quiet = is_rebuild;
  
***************
*** 5205,5210 **** ATAddCheckConstraint(List **wqueue, AlteredTableInfo *tab, Relation rel,
--- 5271,5277 ----
  		newcon->qual = (Node *) make_ands_implicit((Expr *) ccon->expr);
  
  		tab->constraints = lappend(tab->constraints, newcon);
+ 		tab->worklevel = Max(tab->worklevel, WORK_SCAN);
  
  		/* Save the actually assigned name if it was defaulted */
  		if (constr->conname == NULL)
***************
*** 5551,5556 **** ATAddForeignKeyConstraint(AlteredTableInfo *tab, Relation rel,
--- 5618,5627 ----
  		newcon->qual = (Node *) fkconstraint;
  
  		tab->constraints = lappend(tab->constraints, newcon);
+ 		/*
+ 		 * No need to set tab->worklevel; foreign key validation is a distinct
+ 		 * aspect of Phase 3.
+ 		 */
  	}
  
  	/*
***************
*** 5874,5879 **** validateForeignKeyConstraint(Constraint *fkconstraint,
--- 5945,5954 ----
  	HeapTuple	tuple;
  	Trigger		trig;
  
+ 	ereport(DEBUG1,
+ 			(errmsg("validating foreign key constraint \"%s\"",
+ 					fkconstraint->conname)));
+ 
  	/*
  	 * Build a trigger call structure; we'll need it either way.
  	 */
***************
*** 6361,6366 **** ATPrepAlterColumnType(List **wqueue,
--- 6436,6443 ----
  
  	if (tab->relkind == RELKIND_RELATION)
  	{
+ 		CoerceExemptions exempt;
+ 
  		/*
  		 * Set up an expression to transform the old data value to the new type.
  		 * If a USING option was given, transform and use that expression, else
***************
*** 6421,6426 **** ATPrepAlterColumnType(List **wqueue,
--- 6498,6504 ----
  					(errcode(ERRCODE_DATATYPE_MISMATCH),
  					 errmsg("column \"%s\" cannot be cast to type %s",
  							colName, format_type_be(targettype))));
+ 		exempt = GetCoerceExemptions(transform, 1, attnum);
  
  		/*
  		 * Add a work queue item to make ATRewriteTable update the column
***************
*** 6431,6436 **** ATPrepAlterColumnType(List **wqueue,
--- 6509,6519 ----
  		newval->expr = (Expr *) transform;
  
  		tab->newvals = lappend(tab->newvals, newval);
+ 		if (!(exempt & COERCE_EXEMPT_NOCHANGE))
+ 			tab->worklevel = WORK_REWRITE;
+ 		else if (!(exempt & COERCE_EXEMPT_NOERROR))
+ 			tab->worklevel = Max(tab->worklevel, WORK_SCAN);
+ 		/* else, both bits set: WORK_NONE */
  	}
  	else if (tab->relkind == RELKIND_FOREIGN_TABLE)
  	{
diff --git a/src/backend/executor/execMindex 600f7e0..56b10a3 100644
*** a/src/backend/executor/execMain.c
--- b/src/backend/executor/execMain.c
***************
*** 2307,2313 **** OpenIntoRel(QueryDesc *queryDesc)
  
  	(void) heap_reloptions(RELKIND_TOASTVALUE, reloptions, true);
  
! 	AlterTableCreateToastTable(intoRelationId, reloptions);
  
  	/*
  	 * And open the constructed table for writing.
--- 2307,2313 ----
  
  	(void) heap_reloptions(RELKIND_TOASTVALUE, reloptions, true);
  
! 	AlterTableCreateToastTable(intoRelationId, NULL, reloptions);
  
  	/*
  	 * And open the constructed table for writing.
diff --git a/src/backend/parser/parse_index 5b0dc14..b27e5d2 100644
*** a/src/backend/parser/parse_coerce.c
--- b/src/backend/parser/parse_coerce.c
***************
*** 19,24 ****
--- 19,25 ----
  #include "catalog/pg_inherits_fn.h"
  #include "catalog/pg_proc.h"
  #include "catalog/pg_type.h"
+ #include "commands/typecmds.h"
  #include "nodes/makefuncs.h"
  #include "nodes/nodeFuncs.h"
  #include "parser/parse_coerce.h"
***************
*** 1805,1810 **** IsBinaryCoercible(Oid srctype, Oid targettype)
--- 1806,1871 ----
  }
  
  
+ /* GetCoerceExemptions()
+  *		Assess invariants of a coercion expression.
+  *
+  * Various common expressions arising from type coercion are subject to
+  * optimizations.  For example, a simple varchar -> text cast will never change
+  * the underlying data (COERCE_EXEMPT_NOCHANGE) and never yield an error
+  * (COERCE_EXEMPT_NOERROR).  A varchar(8) -> varchar(4) will never change the
+  * data, but it may yield an error.  Given a varno and varattno denoting "the"
+  * source datum, determine which invariants hold for an expression by walking it
+  * per these rules:
+  *
+  *	1. A Var with the varno/varattno in question has both invariants.
+  *	2. A RelabelType node inherits the invariants of its sole argument.
+  *	3. A CoerceToDomain node inherits any COERCE_EXEMPT_NOCHANGE invariant from
+  *		its sole argument.  When GetDomainConstraints() == NIL, it also inherits
+  *		COERCE_EXEMPT_NOERROR.  Otherwise, COERCE_EXEMPT_NOERROR becomes false.
+  *	4. All other nodes have neither invariant.
+  *
+  * Returns a bit string that may contain the following bits:
+  *	COERCE_EXEMPT_NOCHANGE: expression result will always have the same binary
+  *				representation as a Var expression having the given varno and
+  *				varattno
+  *	COERCE_EXEMPT_NOERROR: expression will never throw an error
+  */
+ CoerceExemptions
+ GetCoerceExemptions(Node *expr,
+ 					Index varno, AttrNumber varattno)
+ {
+ 	CoerceExemptions ret = COERCE_EXEMPT_NOCHANGE | COERCE_EXEMPT_NOERROR;
+ 
+ 	Assert(expr != NULL);
+ 
+ 	for (;;)
+ 	{
+ 		if (IsA(expr, Var)
+ 			&& ((Var *) expr)->varno == varno
+ 			&& ((Var *) expr)->varattno == varattno)
+ 		{
+ 			return ret;
+ 		}
+ 		if (IsA(expr, RelabelType))
+ 		{
+ 			expr = (Node *) ((RelabelType *) expr)->arg;
+ 		}
+ 		else if (IsA(expr, CoerceToDomain))
+ 		{
+ 			CoerceToDomain *d = (CoerceToDomain *) expr;
+ 
+ 			if (GetDomainConstraints(d->resulttype) != NIL)
+ 				ret &= ~COERCE_EXEMPT_NOERROR;
+ 			expr = (Node *) d->arg;
+ 		}
+ 		else
+ 		{
+ 			return 0;
+ 		}
+ 	}
+ }
+ 
+ 
  /*
   * find_coercion_pathway
   *		Look for a coercion pathway between two types.
diff --git a/src/backend/tcop/utility.c index 9500037..e6416e4 100644
*** a/src/backend/tcop/utility.c
--- b/src/backend/tcop/utility.c
***************
*** 540,546 **** standard_ProcessUtility(Node *parsetree,
  						(void) heap_reloptions(RELKIND_TOASTVALUE, toast_options,
  											   true);
  
! 						AlterTableCreateToastTable(relOid, toast_options);
  					}
  					else if (IsA(stmt, CreateForeignTableStmt))
  					{
--- 540,546 ----
  						(void) heap_reloptions(RELKIND_TOASTVALUE, toast_options,
  											   true);
  
! 						AlterTableCreateToastTable(relOid, NULL, toast_options);
  					}
  					else if (IsA(stmt, CreateForeignTableStmt))
  					{
diff --git a/src/include/catalog/index 60387cc..3e90656 100644
*** a/src/include/catalog/index.h
--- b/src/include/catalog/index.h
***************
*** 85,95 **** extern double IndexBuildHeapScan(Relation heapRelation,
  
  extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
  
! extern void reindex_index(Oid indexId, bool skip_constraint_checks);
  
  #define REINDEX_CHECK_CONSTRAINTS	0x1
  #define REINDEX_SUPPRESS_INDEX_USE	0x2
! extern bool reindex_relation(Oid relid, bool toast_too, int flags);
  
  extern bool ReindexIsProcessingHeap(Oid heapOid);
  extern bool ReindexIsProcessingIndex(Oid indexOid);
--- 85,97 ----
  
  extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
  
! extern void reindex_index(Oid indexId, const char *toastFor,
! 			  bool skip_constraint_checks);
  
  #define REINDEX_CHECK_CONSTRAINTS	0x1
  #define REINDEX_SUPPRESS_INDEX_USE	0x2
! extern bool reindex_relation(Oid relid, const char *toastFor,
! 				 bool toast_too, int flags);
  
  extern bool ReindexIsProcessingHeap(Oid heapOid);
  extern bool ReindexIsProcessingIndex(Oid indexOid);
diff --git a/src/include/catalog/tindex de3623a..7bd2bdd 100644
*** a/src/include/catalog/toasting.h
--- b/src/include/catalog/toasting.h
***************
*** 17,23 ****
  /*
   * toasting.c prototypes
   */
! extern void AlterTableCreateToastTable(Oid relOid, Datum reloptions);
  extern void BootstrapToastTable(char *relName,
  					Oid toastOid, Oid toastIndexOid);
  
--- 17,24 ----
  /*
   * toasting.c prototypes
   */
! extern void AlterTableCreateToastTable(Oid relOid, const char *finalRelName,
! 						   Datum reloptions);
  extern void BootstrapToastTable(char *relName,
  					Oid toastOid, Oid toastIndexOid);
  
diff --git a/src/include/nodes/execnoindex 546b581..46d9d1a 100644
*** a/src/include/nodes/execnodes.h
--- b/src/include/nodes/execnodes.h
***************
*** 64,69 **** typedef struct IndexInfo
--- 64,70 ----
  	Oid		   *ii_ExclusionOps;	/* array with one entry per column */
  	Oid		   *ii_ExclusionProcs;		/* array with one entry per column */
  	uint16	   *ii_ExclusionStrats;		/* array with one entry per column */
+ 	const char *ii_ToastForRelName;		/* TOAST index only: name of main rel */
  	bool		ii_Unique;
  	bool		ii_ReadyForInserts;
  	bool		ii_Concurrent;
diff --git a/src/include/parser/parsindex ceaff2f..4303acf 100644
*** a/src/include/parser/parse_coerce.h
--- b/src/include/parser/parse_coerce.h
***************
*** 30,37 **** typedef enum CoercionPathType
--- 30,44 ----
  	COERCION_PATH_COERCEVIAIO	/* need a CoerceViaIO node */
  } CoercionPathType;
  
+ /* Bits in the return value of GetCoerceExemptions. */
+ typedef int CoerceExemptions;
+ 
+ #define COERCE_EXEMPT_NOCHANGE	0x1		/* expression never changes storage */
+ #define COERCE_EXEMPT_NOERROR	0x2		/* expression never throws an error */
  
  extern bool IsBinaryCoercible(Oid srctype, Oid targettype);
+ extern CoerceExemptions GetCoerceExemptions(Node *expr,
+ 					Index varno, AttrNumber varattno);
  extern bool IsPreferredType(TYPCATEGORY category, Oid type);
  extern TYPCATEGORY TypeCategory(Oid type);
  
diff --git a/src/test/regress/GNUmakefilindex 15b9ec4..c33ecb9 100644
diff --git a/src/test/regress/expecindex 3280065..7a5b96e 100644
*** a/src/test/regress/expected/alter_table.out
--- b/src/test/regress/expected/alter_table.out
***************
*** 1482,1487 **** create table tab2 (x int, y tab1);
--- 1482,1630 ----
  alter table tab1 alter column b type varchar; -- fails
  ERROR:  cannot alter table "tab1" because column "tab2"."y" uses its rowtype
  --
+ -- ALTER COLUMN ... SET DATA TYPE optimizations
+ --
+ SET client_min_messages = debug1;	-- Track rewrites.
+ -- Model a type change that throws the semantics of dependent expressions.
+ CREATE DOMAIN trickint AS int;
+ CREATE FUNCTION touchy_f(trickint)	RETURNS int4 LANGUAGE sql AS 'SELECT 100';
+ CREATE FUNCTION touchy_f(int4)		RETURNS int4 LANGUAGE sql AS 'SELECT $1';
+ CREATE DOMAIN lendom AS varchar(8);
+ CREATE DOMAIN checkdom AS text CHECK (VALUE LIKE '<%');
+ CREATE TABLE parent (keycol numeric PRIMARY KEY);
+ DEBUG:  building TOAST index for table "parent"
+ NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "parent_pkey" for table "parent"
+ DEBUG:  building index "parent_pkey" on table "parent"
+ INSERT INTO parent VALUES (0.12), (1.12);
+ CREATE TABLE t (
+ 	integral	int4					NOT NULL,
+ 	rational	numeric(9,4)	UNIQUE	NOT NULL	REFERENCES parent,
+ 	string		varchar(4)				NOT NULL,
+ 	strarr		varchar(2)[]			NOT NULL,
+ 	CHECK (touchy_f(integral) < 10),
+ 	EXCLUDE (integral WITH =)
+ );
+ DEBUG:  building TOAST index for table "t"
+ NOTICE:  CREATE TABLE / UNIQUE will create implicit index "t_rational_key" for table "t"
+ DEBUG:  building index "t_rational_key" on table "t"
+ NOTICE:  CREATE TABLE / EXCLUDE will create implicit index "t_integral_excl" for table "t"
+ DEBUG:  building index "t_integral_excl" on table "t"
+ CREATE INDEX ON t USING gin (strarr);
+ DEBUG:  building index "t_strarr_idx" on table "t"
+ INSERT INTO t VALUES (1, 0.12, '<a/>', '{ab,cd}'), (2, 1.12, '<b/>', '{ef,gh}');
+ -- Comments "rewrite", "verify" and "noop" signify whether ATRewriteTables
+ -- rewrites, scans or does nothing to the table proper.  An "-e" suffix denotes
+ -- an error outcome.
+ ALTER TABLE t ALTER integral TYPE trickint;							-- verify-e
+ DEBUG:  building index "t_integral_excl" on table "t"
+ DEBUG:  verifying table "t"
+ ERROR:  check constraint "t_integral_check" is violated by some row
+ ALTER TABLE t DROP CONSTRAINT t_integral_check;
+ ALTER TABLE t ALTER integral TYPE abstime USING integral::abstime;	-- noop
+ DEBUG:  building index "t_integral_excl" on table "t"
+ ALTER TABLE t ALTER integral TYPE oid USING integral::int4;			-- noop
+ DEBUG:  building index "t_integral_excl" on table "t"
+ ALTER TABLE t ALTER integral TYPE regtype;							-- noop
+ DEBUG:  building index "t_integral_excl" on table "t"
+ ALTER TABLE t ALTER rational TYPE numeric(7,4);						-- verify
+ DEBUG:  building TOAST index for table "t"
+ DEBUG:  rewriting table "t"
+ DEBUG:  building index "t_strarr_idx" on table "t"
+ DEBUG:  building index "t_integral_excl" on table "t"
+ DEBUG:  building index "t_rational_key" on table "t"
+ DEBUG:  validating foreign key constraint "t_rational_fkey"
+ ALTER TABLE t ALTER rational TYPE numeric(8,4);						-- noop
+ DEBUG:  building TOAST index for table "t"
+ DEBUG:  rewriting table "t"
+ DEBUG:  building index "t_strarr_idx" on table "t"
+ DEBUG:  building index "t_integral_excl" on table "t"
+ DEBUG:  building index "t_rational_key" on table "t"
+ DEBUG:  validating foreign key constraint "t_rational_fkey"
+ ALTER TABLE t ALTER rational TYPE numeric(8,1);						-- rewrite-e
+ DEBUG:  building TOAST index for table "t"
+ DEBUG:  rewriting table "t"
+ DEBUG:  building index "t_strarr_idx" on table "t"
+ DEBUG:  building index "t_integral_excl" on table "t"
+ DEBUG:  building index "t_rational_key" on table "t"
+ DEBUG:  validating foreign key constraint "t_rational_fkey"
+ ERROR:  insert or update on table "t" violates foreign key constraint "t_rational_fkey"
+ DETAIL:  Key (rational)=(0.1) is not present in table "parent".
+ ALTER TABLE t ALTER string TYPE varchar(6);							-- noop
+ DEBUG:  building TOAST index for table "t"
+ DEBUG:  rewriting table "t"
+ DEBUG:  building index "t_strarr_idx" on table "t"
+ DEBUG:  building index "t_integral_excl" on table "t"
+ DEBUG:  building index "t_rational_key" on table "t"
+ ALTER TABLE t ALTER string TYPE lendom;								-- noop
+ DEBUG:  building TOAST index for table "t"
+ DEBUG:  rewriting table "t"
+ DEBUG:  building index "t_strarr_idx" on table "t"
+ DEBUG:  building index "t_integral_excl" on table "t"
+ DEBUG:  building index "t_rational_key" on table "t"
+ ALTER TABLE t ALTER string TYPE xml USING string::xml;				-- verify
+ DEBUG:  building TOAST index for table "t"
+ DEBUG:  rewriting table "t"
+ DEBUG:  building index "t_strarr_idx" on table "t"
+ DEBUG:  building index "t_integral_excl" on table "t"
+ DEBUG:  building index "t_rational_key" on table "t"
+ ALTER TABLE t ALTER string TYPE checkdom;							-- verify
+ DEBUG:  verifying table "t"
+ ALTER TABLE t ALTER string TYPE text USING 'foo'::varchar;			-- rewrite
+ DEBUG:  building TOAST index for table "t"
+ DEBUG:  rewriting table "t"
+ DEBUG:  building index "t_strarr_idx" on table "t"
+ DEBUG:  building index "t_integral_excl" on table "t"
+ DEBUG:  building index "t_rational_key" on table "t"
+ ALTER TABLE t ALTER strarr TYPE varchar(4)[];						-- noop
+ DEBUG:  building TOAST index for table "t"
+ DEBUG:  rewriting table "t"
+ DEBUG:  building index "t_integral_excl" on table "t"
+ DEBUG:  building index "t_rational_key" on table "t"
+ DEBUG:  building index "t_strarr_idx" on table "t"
+ ALTER TABLE t ADD CONSTRAINT u0 UNIQUE (integral), -- build index exactly once
+ 			  ALTER integral TYPE int8;								-- rewrite
+ NOTICE:  ALTER TABLE / ADD UNIQUE will create implicit index "u0" for table "t"
+ DEBUG:  building TOAST index for table "t"
+ DEBUG:  rewriting table "t"
+ DEBUG:  building index "t_rational_key" on table "t"
+ DEBUG:  building index "t_strarr_idx" on table "t"
+ DEBUG:  building index "t_integral_excl" on table "t"
+ DEBUG:  building index "u0" on table "t"
+ -- Data and catalog end state.  We omit the columns that bear unstable OIDs.
+ SELECT * FROM t ORDER BY 1;
+  integral | rational | string | strarr  
+ ----------+----------+--------+---------
+         1 |   0.1200 | foo    | {ab,cd}
+         2 |   1.1200 | foo    | {ef,gh}
+ (2 rows)
+ 
+ SELECT relname, indclass FROM pg_index JOIN pg_class c ON c.oid = indexrelid
+ WHERE indrelid = 't'::regclass ORDER BY 1;
+      relname     | indclass 
+ -----------------+----------
+  t_integral_excl | 10029
+  t_rational_key  | 10037
+  t_strarr_idx    | 10103
+  u0              | 10029
+ (4 rows)
+ 
+ SELECT relname, attname, atttypid, atttypmod
+ FROM pg_attribute JOIN pg_class c ON c.oid = attrelid
+ WHERE attnum > 0 AND
+ 	attrelid IN (SELECT indexrelid FROM pg_index WHERE indrelid = 't'::regclass)
+ ORDER BY 1, 2;
+      relname     | attname  | atttypid | atttypmod 
+ -----------------+----------+----------+-----------
+  t_integral_excl | integral |       20 |        -1
+  t_rational_key  | rational |     1700 |    524296
+  t_strarr_idx    | strarr   |     1043 |        -1
+  u0              | integral |       20 |        -1
+ (4 rows)
+ 
+ -- Done.  Retain the table under a less-generic name.
+ ALTER TABLE t RENAME TO alter_type_test;
+ RESET client_min_messages;
+ --
  -- lock levels
  --
  drop type lockmodes;
diff --git a/src/test/regress/expected/big_alternew file mode 100644
index 0000000..1609c01
diff --git a/src/test/regress/sql/alter_table.sql b/index cfbfbb6..2525fe0 100644
*** a/src/test/regress/sql/alter_table.sql
--- b/src/test/regress/sql/alter_table.sql
***************
*** 1098,1103 **** create table tab2 (x int, y tab1);
--- 1098,1165 ----
  alter table tab1 alter column b type varchar; -- fails
  
  --
+ -- ALTER COLUMN ... SET DATA TYPE optimizations
+ --
+ SET client_min_messages = debug1;	-- Track rewrites.
+ 
+ -- Model a type change that throws the semantics of dependent expressions.
+ CREATE DOMAIN trickint AS int;
+ CREATE FUNCTION touchy_f(trickint)	RETURNS int4 LANGUAGE sql AS 'SELECT 100';
+ CREATE FUNCTION touchy_f(int4)		RETURNS int4 LANGUAGE sql AS 'SELECT $1';
+ CREATE DOMAIN lendom AS varchar(8);
+ CREATE DOMAIN checkdom AS text CHECK (VALUE LIKE '<%');
+ 
+ CREATE TABLE parent (keycol numeric PRIMARY KEY);
+ INSERT INTO parent VALUES (0.12), (1.12);
+ 
+ CREATE TABLE t (
+ 	integral	int4					NOT NULL,
+ 	rational	numeric(9,4)	UNIQUE	NOT NULL	REFERENCES parent,
+ 	string		varchar(4)				NOT NULL,
+ 	strarr		varchar(2)[]			NOT NULL,
+ 	CHECK (touchy_f(integral) < 10),
+ 	EXCLUDE (integral WITH =)
+ );
+ CREATE INDEX ON t USING gin (strarr);
+ INSERT INTO t VALUES (1, 0.12, '<a/>', '{ab,cd}'), (2, 1.12, '<b/>', '{ef,gh}');
+ 
+ -- Comments "rewrite", "verify" and "noop" signify whether ATRewriteTables
+ -- rewrites, scans or does nothing to the table proper.  An "-e" suffix denotes
+ -- an error outcome.
+ ALTER TABLE t ALTER integral TYPE trickint;							-- verify-e
+ ALTER TABLE t DROP CONSTRAINT t_integral_check;
+ ALTER TABLE t ALTER integral TYPE abstime USING integral::abstime;	-- noop
+ ALTER TABLE t ALTER integral TYPE oid USING integral::int4;			-- noop
+ ALTER TABLE t ALTER integral TYPE regtype;							-- noop
+ ALTER TABLE t ALTER rational TYPE numeric(7,4);						-- verify
+ ALTER TABLE t ALTER rational TYPE numeric(8,4);						-- noop
+ ALTER TABLE t ALTER rational TYPE numeric(8,1);						-- rewrite-e
+ ALTER TABLE t ALTER string TYPE varchar(6);							-- noop
+ ALTER TABLE t ALTER string TYPE lendom;								-- noop
+ ALTER TABLE t ALTER string TYPE xml USING string::xml;				-- verify
+ ALTER TABLE t ALTER string TYPE checkdom;							-- verify
+ ALTER TABLE t ALTER string TYPE text USING 'foo'::varchar;			-- rewrite
+ ALTER TABLE t ALTER strarr TYPE varchar(4)[];						-- noop
+ ALTER TABLE t ADD CONSTRAINT u0 UNIQUE (integral), -- build index exactly once
+ 			  ALTER integral TYPE int8;								-- rewrite
+ 
+ -- Data and catalog end state.  We omit the columns that bear unstable OIDs.
+ SELECT * FROM t ORDER BY 1;
+ 
+ SELECT relname, indclass FROM pg_index JOIN pg_class c ON c.oid = indexrelid
+ WHERE indrelid = 't'::regclass ORDER BY 1;
+ 
+ SELECT relname, attname, atttypid, atttypmod
+ FROM pg_attribute JOIN pg_class c ON c.oid = attrelid
+ WHERE attnum > 0 AND
+ 	attrelid IN (SELECT indexrelid FROM pg_index WHERE indrelid = 't'::regclass)
+ ORDER BY 1, 2;
+ 
+ -- Done.  Retain the table under a less-generic name.
+ ALTER TABLE t RENAME TO alter_type_test;
+ RESET client_min_messages;
+ 
+ --
  -- lock levels
  --
  drop type lockmodes;
diff --git a/src/test/regress/sql/big_alternew file mode 100644
index 0000000..3824d96
#12Robert Haas
robertmhaas@gmail.com
In reply to: Noah Misch (#11)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Thu, Jan 27, 2011 at 2:48 PM, Noah Misch <noah@leadboat.com> wrote:

Done as attached.  This preserves compatibility with our current handling of
composite type dependencies.  The rest you've seen before.

OK, so I took a look at this in more detail today. The current logic
for table rewrites looks like this:

1. If we're changing the data type of a column, or adding a column
with a default, or adding/dropping OIDs, rewrite the table. Stop.
2. Otherwise, if we're adding a constraint or NOT NULL, scan the table
and check constraints.
3. If we're changing tablespaces, copy the table block-by-block.

It seems to me that the revised logic needs to look like this:

1. If we're changing the data type of a column and the existing
contents are not binary coercible to the new contents, or if we're
adding a column with a default or adding/dropping OIDs, rewrite the
table. Stop.
2. Otherwise, if we're adding a constraint or NOT NULL, scan the table
and check constraints.
3. If we're changing the data type of a column in the table, reindex the table.
4. If we're changing tablespaces, copy the table block-by-block.

I might be missing something, but I don't see that the patch includes
step #3, which I think is necessary. For example, citext is binary
coercible to text, but you can't reuse the index because the
comparison function is different. Of course, if you're changing the
type of a column to its already-current type, you can skip #3, but if
that's the only case we can optimize, it's not much of an
accomplishment. I guess this gets back to the ALTER TYPE 7 patch,
which I haven't looked at in detail, but I have a feeling it may be
controversial.

Another problem here is that if you have to do both #2 and #3, you
might have been better off (or just as well off) doing #1, unless you
can somehow jigger things so that the same scan does both the
constraint checks and the index rebuild. That doesn't look simple.

Thoughts?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#13Noah Misch
noah@leadboat.com
In reply to: Robert Haas (#12)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

Robert,

Thanks for the obviously thought-out review.

On Sat, Feb 05, 2011 at 01:29:35AM -0500, Robert Haas wrote:

On Thu, Jan 27, 2011 at 2:48 PM, Noah Misch <noah@leadboat.com> wrote:

Done as attached. ?This preserves compatibility with our current handling of
composite type dependencies. ?The rest you've seen before.

OK, so I took a look at this in more detail today. The current logic
for table rewrites looks like this:

1. If we're changing the data type of a column, or adding a column
with a default, or adding/dropping OIDs, rewrite the table. Stop.
2. Otherwise, if we're adding a constraint or NOT NULL, scan the table
and check constraints.
3. If we're changing tablespaces, copy the table block-by-block.

Correct. It's perhaps obvious, but rewriting the table will always reindex it.

It seems to me that the revised logic needs to look like this:

1. If we're changing the data type of a column and the existing
contents are not binary coercible to the new contents, or if we're
adding a column with a default or adding/dropping OIDs, rewrite the
table. Stop.
2. Otherwise, if we're adding a constraint or NOT NULL, scan the table
and check constraints.

With this patch, step 2 changes changes to "Otherwise, if we're adding a
constraint or NOT NULL, or changing a column to a binary-compatible domain with
a domain CHECK constraint, scan the table and check constraints."

3. If we're changing the data type of a column in the table, reindex the table.

Rebuild indexes that depend on a changing column. Other indexes can stay.

4. If we're changing tablespaces, copy the table block-by-block.

I might be missing something, but I don't see that the patch includes
step #3, which I think is necessary. For example, citext is binary
coercible to text, but you can't reuse the index because the
comparison function is different. Of course, if you're changing the
type of a column to its already-current type, you can skip #3, but if
that's the only case we can optimize, it's not much of an
accomplishment. I guess this gets back to the ALTER TYPE 7 patch,
which I haven't looked at in detail, but I have a feeling it may be
controversial.

It's there, but it's happening rather implicitly. ATExecAlterColumnType builds
lists of indexes and constraints that depend on changing columns. Specifically,
it stashes their OIDs and the SQL to recreate them. ATPostAlterTypeCleanup
drops those objects by OID, then parses the SQL statements, now based on the
updated table definition. ATExecAddIndex and ATExecAddConstraint use those
parsed statements to recreate the objects. The key is the skip_build logic in
ATExecAddIndex: if ATRewriteTables will rewrite the table (and therefore *all*
indexes), we skip the build at that earlier stage to avoid building the same
index twice. The only thing I had to do was update the skip_build condition so
it continues to mirror the corresponding test in ATRewriteTables.

Originally I had this patch doing a full reindex, with an eye to having the next
patch reduce the scope to dependent indexes. However, all the infrastructure
was already there, and it actually made this patch smaller to skip directly to
what it does today.

ALTER TYPE 7 additionally skips builds of indexes that depend on a changing
column but can be proven compatible. So it's in the business of, for example
figuring out that text and varchar are compatible but text and citext are not.

Another problem here is that if you have to do both #2 and #3, you
might have been better off (or just as well off) doing #1, unless you
can somehow jigger things so that the same scan does both the
constraint checks and the index rebuild. That doesn't look simple.

We have no such optimization during #1, either, so #2+#3 is never worse. In
particular, #1 scans the table (# indexes total) + 1 times, while #2+#3 scans it
(# of indexes depending on changed columns) + 1 times.

There are some nice optimization opportunities here, to be sure. As a specific
first step, teach index_build to create multiple indexes with a single scan,
then have reindex_relation use that. Probably not simple. Combining that with
the ATRewriteTable scan would be less simple still.

nm

#14Robert Haas
robertmhaas@gmail.com
In reply to: Noah Misch (#13)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Sat, Feb 5, 2011 at 3:05 AM, Noah Misch <noah@leadboat.com> wrote:

Correct.  It's perhaps obvious, but rewriting the table will always reindex it.

Right.

3. If we're changing the data type of a column in the table, reindex the table.

Rebuild indexes that depend on a changing column.  Other indexes can stay.

Good point.

4. If we're changing tablespaces, copy the table block-by-block.

I might be missing something, but I don't see that the patch includes
step #3, which I think is necessary.  For example, citext is binary
coercible to text, but you can't reuse the index because the
comparison function is different.  Of course, if you're changing the
type of a column to its already-current type, you can skip #3, but if
that's the only case we can optimize, it's not much of an
accomplishment.  I guess this gets back to the ALTER TYPE 7 patch,
which I haven't looked at in detail, but I have a feeling it may be
controversial.

It's there, but it's happening rather implicitly.

I see now. So you're actually not really making any change to that
machinery. It's sufficient to just skip the rewrite of the heap when
it isn't needed, and without any particular code change the indexes
will sort themselves out.

We have no such optimization during #1, either, so #2+#3 is never worse.  In
particular, #1 scans the table (# indexes total) + 1 times, while #2+#3 scans it
(# of indexes depending on changed columns) + 1 times.

OK.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#15Robert Haas
robertmhaas@gmail.com
In reply to: Noah Misch (#11)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Thu, Jan 27, 2011 at 2:48 PM, Noah Misch <noah@leadboat.com> wrote:

Done as attached.

Looking at this still more, it appears that independent of any change
this patch may wish to make, there's a live bug here related to the
foreign table patch I committed back in December. Creating a foreign
table creates an eponymous rowtype, which can be used as a column in a
regular table. You can then change the data type of a column in the
foreign table, read from the regular table, and crash the server.

The simple fix for this is to just change the code in
ATPrepAlterColumnType to handle the foreign table case also:

if (tab->relkind == RELKIND_COMPOSITE_TYPE)
{
/*
* For composite types, do this check now. Tables will check
* it later when the table is being rewritten.
*/
find_composite_type_dependencies(rel->rd_rel->reltype,
NULL,
RelationGetRelationName(rel));
}

But this is a little unsatisfying, for two reasons. First, the error
message will be subtly wrong: we can make it complain about a table or
a type, but not a foreign table. At a quick look, it likes the right
fix might be to replace the second and third arguments to
find_composite_type_dependencies() with a Relation. Second, I wonder
if we shouldn't refactor things so that all the checks fire in
ATRewriteTables() rather than doing them in different places. Seems
like that might be cleaner.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#16Noah Misch
noah@leadboat.com
In reply to: Robert Haas (#15)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Sat, Feb 05, 2011 at 10:03:59AM -0500, Robert Haas wrote:

Looking at this still more, it appears that independent of any change
this patch may wish to make, there's a live bug here related to the
foreign table patch I committed back in December. Creating a foreign
table creates an eponymous rowtype, which can be used as a column in a
regular table. You can then change the data type of a column in the
foreign table, read from the regular table, and crash the server.

The simple fix for this is to just change the code in
ATPrepAlterColumnType to handle the foreign table case also:

if (tab->relkind == RELKIND_COMPOSITE_TYPE)
{
/*
* For composite types, do this check now. Tables will check
* it later when the table is being rewritten.
*/
find_composite_type_dependencies(rel->rd_rel->reltype,
NULL,
RelationGetRelationName(rel));
}

But this is a little unsatisfying, for two reasons. First, the error
message will be subtly wrong: we can make it complain about a table or
a type, but not a foreign table. At a quick look, it likes the right
fix might be to replace the second and third arguments to
find_composite_type_dependencies() with a Relation.

Seems like a clear improvement.

Second, I wonder
if we shouldn't refactor things so that all the checks fire in
ATRewriteTables() rather than doing them in different places. Seems
like that might be cleaner.

Offhand, this seems reasonable, too. I assumed there was some good reason it
couldn't be done there for non-tables, but nothing comes to mind.

#17Robert Haas
robertmhaas@gmail.com
In reply to: Noah Misch (#16)
1 attachment(s)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Sat, Feb 5, 2011 at 7:44 PM, Noah Misch <noah@leadboat.com> wrote:

But this is a little unsatisfying, for two reasons.  First, the error
message will be subtly wrong: we can make it complain about a table or
a type, but not a foreign table.  At a quick look, it likes the right
fix might be to replace the second and third arguments to
find_composite_type_dependencies() with a Relation.

Seems like a clear improvement.

That didn't quite work, because there's a caller in typecmds.c that
doesn't have the relation handy. So I made it take a relkind and a
name, which works fine.

Second, I wonder
if we shouldn't refactor things so that all the checks fire in
ATRewriteTables() rather than doing them in different places.  Seems
like that might be cleaner.

Offhand, this seems reasonable, too.  I assumed there was some good reason it
couldn't be done there for non-tables, but nothing comes to mind.

Actually, thinking about this more, I'm thinking if we're going to
change anything, it seems we ought to go the other way. If we ever
actually did support recursing into wherever the composite type
dependencies take us, we'd want to detect that before phase 3 and add
work-queue items for each table that we needed to futz with.

The attached patch takes this approach. It's very slightly more code,
but it reduces the amount of spooky action at a distance. The
existing coding is basically relying on the assumption that operations
which require finding composite type dependencies also require a table
rewrite. That was probably true at one point in time, but it's not
really quite right. It already requires compensating code foreign
tables and composite types (which can require this treatment even
though there's nothing to rewrite) and your proposed changes to avoid
table rewrites in cases where a type is changed to a compatible type
would break it in the opposite direction (the check would still be
needed even if the parent table doesn't need a rewrite, because the
rowtype could be indexed in some fashion that depends on the type of
the column being changed).

Comments?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Attachments:

find-composite-type-dependencies-refactor.patchtext/x-diff; charset=US-ASCII; name=find-composite-type-dependencies-refactor.patchDownload
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 6a17399..83f8be0 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -3378,23 +3378,6 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
 	}
 
 	/*
-	 * If we change column data types or add/remove OIDs, the operation has to
-	 * be propagated to tables that use this table's rowtype as a column type.
-	 * newrel will also be non-NULL in the case where we're adding a column
-	 * with a default.  We choose to forbid that case as well, since composite
-	 * types might eventually support defaults.
-	 *
-	 * (Eventually we'll probably need to check for composite type
-	 * dependencies even when we're just scanning the table without a rewrite,
-	 * but at the moment a composite type does not enforce any constraints,
-	 * so it's not necessary/appropriate to enforce them just during ALTER.)
-	 */
-	if (newrel)
-		find_composite_type_dependencies(oldrel->rd_rel->reltype,
-										 oldrel->rd_rel->relkind,
-										 RelationGetRelationName(oldrel));
-
-	/*
 	 * Generate the constraint and default execution states
 	 */
 
@@ -4055,7 +4038,7 @@ ATExecAddColumn(AlteredTableInfo *tab, Relation rel,
 	Oid			typeOid;
 	int32		typmod;
 	Form_pg_type tform;
-	Expr	   *defval;
+	Expr	   *defval = NULL;
 
 	attrdesc = heap_open(AttributeRelationId, RowExclusiveLock);
 
@@ -4304,6 +4287,20 @@ ATExecAddColumn(AlteredTableInfo *tab, Relation rel,
 		tab->new_changeoids = true;
 
 	/*
+	 * If we're adding an OID column, the operation has to be propagated to
+	 * tables that use this table's rowtype as a column type.  But we don't
+	 * currently have the infrastructure for that, so just throw an error.
+	 * We also forbid the case where we're adding a column with a default, since
+	 * composite types might eventually support defaults, and ALTER TABLE ..
+	 * ADD COLUMN .. DEFAULT would be expected to initialize the newly added
+	 * column to the default in each instantiation of the rowtype.
+	 */
+	if (isOid || defval)
+		find_composite_type_dependencies(rel->rd_rel->reltype,
+										 rel->rd_rel->relkind,
+										 RelationGetRelationName(rel));
+
+	/*
 	 * Add needed dependency entries for the new column.
 	 */
 	add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
@@ -4975,6 +4972,16 @@ ATExecDropColumn(List **wqueue, Relation rel, const char *colName,
 
 		/* Tell Phase 3 to physically remove the OID column */
 		tab->new_changeoids = true;
+
+		/*
+		 * The OID removal operation needs to be propagated to tables that use
+		 * this table's rowtype as a column type.  But we don't currently have
+		 * the infrastructure for that, so just throw an error if it would be
+		 * required.
+		 */
+		find_composite_type_dependencies(rel->rd_rel->reltype,
+										 rel->rd_rel->relkind,
+										 RelationGetRelationName(rel));
 	}
 }
 
@@ -6442,17 +6449,15 @@ ATPrepAlterColumnType(List **wqueue,
 					 errmsg("ALTER TYPE USING is not supported on foreign tables")));
 	}
 
-	if (tab->relkind == RELKIND_COMPOSITE_TYPE
-		|| tab->relkind == RELKIND_FOREIGN_TABLE)
-	{
-		/*
-		 * For composite types, do this check now.  Tables will check
-		 * it later when the table is being rewritten.
-		 */
-		find_composite_type_dependencies(rel->rd_rel->reltype,
-										 rel->rd_rel->relkind,
-										 RelationGetRelationName(rel));
-	}
+	/*
+	 * If we change column data types, the operation has to be propagated to
+	 * tables that use this table's rowtype as a column type.  But we don't
+	 * currently have the infrastructure for that, so just throw an error
+	 * if it would be required.
+	 */
+	find_composite_type_dependencies(rel->rd_rel->reltype,
+									 rel->rd_rel->relkind,
+									 RelationGetRelationName(rel));
 
 	ReleaseSysCache(tuple);
 
#18Noah Misch
noah@leadboat.com
In reply to: Robert Haas (#17)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Sun, Feb 06, 2011 at 02:15:47AM -0500, Robert Haas wrote:

On Sat, Feb 5, 2011 at 7:44 PM, Noah Misch <noah@leadboat.com> wrote:

But this is a little unsatisfying, for two reasons. ?First, the error
message will be subtly wrong: we can make it complain about a table or
a type, but not a foreign table. ?At a quick look, it likes the right
fix might be to replace the second and third arguments to
find_composite_type_dependencies() with a Relation.

Seems like a clear improvement.

That didn't quite work, because there's a caller in typecmds.c that
doesn't have the relation handy. So I made it take a relkind and a
name, which works fine.

Hmm, indeed. In get_rels_with_domain(), it's a scalar type.

Second, I wonder
if we shouldn't refactor things so that all the checks fire in
ATRewriteTables() rather than doing them in different places. ?Seems
like that might be cleaner.

Offhand, this seems reasonable, too. ?I assumed there was some good reason it
couldn't be done there for non-tables, but nothing comes to mind.

Actually, thinking about this more, I'm thinking if we're going to
change anything, it seems we ought to go the other way. If we ever
actually did support recursing into wherever the composite type
dependencies take us, we'd want to detect that before phase 3 and add
work-queue items for each table that we needed to futz with.

The attached patch takes this approach. It's very slightly more code,
but it reduces the amount of spooky action at a distance.

Comments?

Your patch improves the code. My standard for commending a refactor-only patch
is rather high, though, and this patch doesn't reach it. The ancestral code
placement wasn't obviously correct, but neither is this. So I'd vote -0.

nm

#19Robert Haas
robertmhaas@gmail.com
In reply to: Noah Misch (#18)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Sun, Feb 6, 2011 at 4:15 AM, Noah Misch <noah@leadboat.com> wrote:

That didn't quite work, because there's a caller in typecmds.c that
doesn't have the relation handy.  So I made it take a relkind and a
name, which works fine.

Hmm, indeed.  In get_rels_with_domain(), it's a scalar type.

Yeeeeeeah, that's actually a little ugly. It's actually a domain
over a composite type, not a composite type proper, IIUC. Better
ideas?

The attached patch takes this approach.  It's very slightly more code,
but it reduces the amount of spooky action at a distance.

Comments?

Your patch improves the code.  My standard for commending a refactor-only patch
is rather high, though, and this patch doesn't reach it.  The ancestral code
placement wasn't obviously correct, but neither is this.  So I'd vote -0.

Well, what's your suggestion, then? Your patch pops the test up from
ATRewriteTable() up to ATRewriteTables(), but that's not obviously
correct either, and it's an awkward place to do it because we don't
have the Relation object handy at the point where the check needs to
be done.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#20Noah Misch
noah@leadboat.com
In reply to: Robert Haas (#19)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Sun, Feb 06, 2011 at 07:54:52AM -0500, Robert Haas wrote:

On Sun, Feb 6, 2011 at 4:15 AM, Noah Misch <noah@leadboat.com> wrote:

That didn't quite work, because there's a caller in typecmds.c that
doesn't have the relation handy. ?So I made it take a relkind and a
name, which works fine.

Hmm, indeed. ?In get_rels_with_domain(), it's a scalar type.

Yeeeeeeah, that's actually a little ugly. It's actually a domain
over a composite type, not a composite type proper, IIUC. Better
ideas?

There are no domains over composite types. get_rels_with_domain() finds all
relations having columns of the (scalar) domain type. It then calls
find_composite_type_dependencies to identify uses of the composite types
discovered in the previous step.

Honestly, RELKIND_COMPOSITE_TYPE is a reasonable choice despite the technical
mismatch. One more-correct approach would be to have two arguments, a catalog
OID (pg_type or pg_class, currently) and a relkind, 0 when the catalog OID !=
pg_class. Might be an improvement, albeit a minor one.

The attached patch takes this approach. ?It's very slightly more code,
but it reduces the amount of spooky action at a distance.

Comments?

Your patch improves the code. ?My standard for commending a refactor-only patch
is rather high, though, and this patch doesn't reach it. ?The ancestral code
placement wasn't obviously correct, but neither is this. ?So I'd vote -0.

Well, what's your suggestion, then? Your patch pops the test up from
ATRewriteTable() up to ATRewriteTables(), but that's not obviously
correct either, and it's an awkward place to do it because we don't
have the Relation object handy at the point where the check needs to
be done.

Agreed. I couldn't think of any grand improvements, so my patch bore the
conceptually-smallest change I could come up with to handle that particular
issue. That is, I felt it was the change that could most easily be verified as
rejecting the same cases as the old test.

nm

#21Noah Misch
noah@leadboat.com
In reply to: Noah Misch (#20)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Sun, Feb 06, 2011 at 08:40:44AM -0500, Noah Misch wrote:

On Sun, Feb 06, 2011 at 07:54:52AM -0500, Robert Haas wrote:

Yeeeeeeah, that's actually a little ugly. It's actually a domain
over a composite type, not a composite type proper, IIUC. Better
ideas?

There are no domains over composite types. get_rels_with_domain() finds all
relations having columns of the (scalar) domain type. It then calls
find_composite_type_dependencies to identify uses of the composite types
discovered in the previous step.

Honestly, RELKIND_COMPOSITE_TYPE is a reasonable choice despite the technical
mismatch. One more-correct approach would be to have two arguments, a catalog
OID (pg_type or pg_class, currently) and a relkind, 0 when the catalog OID !=
pg_class. Might be an improvement, albeit a minor one.

Scratch that. How about classid and objid arguments, passing them to
getObjectionDescription() internally? We already do something very similar in
ATExecAlterColumnType for a related case.

#22Robert Haas
robertmhaas@gmail.com
In reply to: Noah Misch (#21)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Sun, Feb 6, 2011 at 8:58 AM, Noah Misch <noah@leadboat.com> wrote:

On Sun, Feb 06, 2011 at 08:40:44AM -0500, Noah Misch wrote:

On Sun, Feb 06, 2011 at 07:54:52AM -0500, Robert Haas wrote:

Yeeeeeeah, that's actually a little ugly.   It's actually a domain
over a composite type, not a composite type proper, IIUC. Better
ideas?

There are no domains over composite types.  get_rels_with_domain() finds all
relations having columns of the (scalar) domain type.  It then calls
find_composite_type_dependencies to identify uses of the composite types
discovered in the previous step.

Honestly, RELKIND_COMPOSITE_TYPE is a reasonable choice despite the technical
mismatch.  One more-correct approach would be to have two arguments, a catalog
OID (pg_type or pg_class, currently) and a relkind, 0 when the catalog OID !=
pg_class.  Might be an improvement, albeit a minor one.

Scratch that.  How about classid and objid arguments, passing them to
getObjectionDescription() internally?  We already do something very similar in
ATExecAlterColumnType for a related case.

That's not quite so good for translators, I think.

Another option is that we could just say "relation" (table, foreign
table, etc...) or "type". We use the word relation as a more generic
version of table in a few other places.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#23Robert Haas
robertmhaas@gmail.com
In reply to: Robert Haas (#22)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Sun, Feb 6, 2011 at 12:13 PM, Robert Haas <robertmhaas@gmail.com> wrote:

That's not quite so good for translators, I think.

Another option is that we could just say "relation" (table, foreign
table, etc...) or "type".  We use the word relation as a more generic
version of table in a few other places.

Or how about passing an ObjectType? Then we could specify
OBJECT_TABLE, OBJECT_FOREIGN_TABLE, or OBJECT_TYPE.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#24Noah Misch
noah@leadboat.com
In reply to: Robert Haas (#23)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Sun, Feb 06, 2011 at 12:54:19PM -0500, Robert Haas wrote:

On Sun, Feb 6, 2011 at 12:13 PM, Robert Haas <robertmhaas@gmail.com> wrote:

That's not quite so good for translators, I think.

Another option is that we could just say "relation" (table, foreign
table, etc...) or "type". ?We use the word relation as a more generic
version of table in a few other places.

Seems fine.

Or how about passing an ObjectType? Then we could specify
OBJECT_TABLE, OBJECT_FOREIGN_TABLE, or OBJECT_TYPE.

Could this be done without a several-line blob of code at each call site to
determine the answer? If and only if so, this sounds better.

#25Robert Haas
robertmhaas@gmail.com
In reply to: Noah Misch (#24)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Sun, Feb 6, 2011 at 8:18 PM, Noah Misch <noah@leadboat.com> wrote:

Or how about passing an ObjectType?  Then we could specify
OBJECT_TABLE, OBJECT_FOREIGN_TABLE, or OBJECT_TYPE.

Could this be done without a several-line blob of code at each call site to
determine the answer?  If and only if so, this sounds better.

Yeah, that's a problem. New thought: how about we go back more or
less to the original coding, except replacing the second argument
(only) with a Relation? In other words, callers will pass either a
Relation (which might be a table or foreign table) or a type name.
Not particularly elegant, but no worse than what we had before.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#26Noah Misch
noah@leadboat.com
In reply to: Robert Haas (#25)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Mon, Feb 07, 2011 at 12:04:02AM -0500, Robert Haas wrote:

On Sun, Feb 6, 2011 at 8:18 PM, Noah Misch <noah@leadboat.com> wrote:

Or how about passing an ObjectType? ?Then we could specify
OBJECT_TABLE, OBJECT_FOREIGN_TABLE, or OBJECT_TYPE.

Could this be done without a several-line blob of code at each call site to
determine the answer? ?If and only if so, this sounds better.

Yeah, that's a problem. New thought: how about we go back more or
less to the original coding, except replacing the second argument
(only) with a Relation? In other words, callers will pass either a
Relation (which might be a table or foreign table) or a type name.
Not particularly elegant, but no worse than what we had before.

Sounds good. Thanks.

#27Robert Haas
robertmhaas@gmail.com
In reply to: Noah Misch (#11)
1 attachment(s)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Thu, Jan 27, 2011 at 2:48 PM, Noah Misch <noah@leadboat.com> wrote:

On Wed, Jan 26, 2011 at 07:31:40AM -0500, Robert Haas wrote:

I'd also suggest that this big if-block you changed to a case
statement could just as well stay as an if-block.  There are only
three cases, and we want to avoid rearranging things more than
necessary.  It complicates both review and back-patching to no good
end.

Okay.  I've also left out the large reindent in ATRewriteTable for now.  Easy to
re-add it later if desired.

I think you should collect up what's left of ALTER TABLE 0 and the
stuff on this thread, rebase it, and submit it as a single patch on
this thread that applies directly against the master branch.  We may
decide to split it back up again in some other way, but I think the
current division isn't actually buying us much.

Done as attached.  This preserves compatibility with our current handling of
composite type dependencies.  The rest you've seen before.

OK, I finally got back to this. Sorry for the delay. I still feel
that it's possible to accomplish the same goal with less complexity.
See what you think of the attached.

Upon examination, it appeared to me that trying to make the
AlteredTableInfo contain an enum indicating whether we were doing a
scan, a rewrite, or nothing was requiring more code change than I
could really justify. What I ended up doing is replacing the 'bool
new_changedoids' member with a 'bool rewrite' member. I'm pretty sure
this is safe. The only reason we ever tested the new_changeoids
member was to see whether we needed to do a rewrite; it wasn't used
for anything else. The new member gets set either when we need to do
a rewrite, or when a column type changes in a fashion that requires a
rewrite. It's pretty easy to verify that this doesn't result in any
behavior change, except for one corner case: currently, if you add or
remove OIDs to/from a table and in the same ALTER TABLE command add a
constraint that involves creating an index, such as a primary key, the
new primary key index will get built, thrown away, and rebuilt a
second time. With this change, we just build it once, which seems
like an improvement, even though the case is vanishingly unlikely to
ever happen in practice.

I also have to say that after playing with a little bit, I find the
new debugging messages you added to be quite invaluable for
understanding what's really going on. The old output told you
basically nothing useful, even if you cranked it all the way up to
DEBUG3. This is much better.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Attachments:

alter-type-2-rmh.patchapplication/octet-stream; name=alter-type-2-rmh.patchDownload
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 9f02674..e3ceea8 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -766,9 +766,13 @@ ALTER TABLE <replaceable class="PARAMETER">name</replaceable>
    <para>
     Adding a column with a non-null default or changing the type of an
     existing column will require the entire table and indexes to be rewritten.
-    This might take a significant amount of time for a large table; and it will
-    temporarily require double the disk space.  Adding or removing a system
-    <literal>oid</> column likewise requires rewriting the entire table.
+    As an exception, if the old type and new types are binary compatible and
+    the <literal>USING</> clause does not change the column contents, the
+    table rewrite will be skipped, but the index rebuild is still required.
+    Adding or removing a system <literal>oid</> column likewise requires
+    rewriting the entire table.  Table and/or index rebuilds may take a
+    significant amount of time for a large table; and will temporarily require
+    as much as double the disk space.
    </para>
 
    <para>
@@ -797,9 +801,9 @@ ALTER TABLE <replaceable class="PARAMETER">name</replaceable>
    <para>
     To force an immediate rewrite of the table, you can use
     <link linkend="SQL-VACUUM">VACUUM FULL</>, <xref linkend="SQL-CLUSTER">
-    or one of the forms of ALTER TABLE that forces a rewrite, such as
-    SET DATA TYPE.  This results in no semantically-visible change in the
-    table, but gets rid of no-longer-useful data.
+    or one of the forms of ALTER TABLE that forces a rewrite.  This results in
+    no semantically-visible change in the table, but gets rid of
+    no-longer-useful data.
    </para>
 
    <para>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9e60121..452ced6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -1685,6 +1685,11 @@ index_build(Relation heapRelation,
 	procedure = indexRelation->rd_am->ambuild;
 	Assert(RegProcedureIsValid(procedure));
 
+	ereport(DEBUG1,
+			(errmsg("building index \"%s\" on table \"%s\"",
+					RelationGetRelationName(indexRelation),
+					RelationGetRelationName(heapRelation))));
+
 	/*
 	 * Switch to the table owner's userid, so that any index functions are run
 	 * as that user.  Also lock down security-restricted operations and
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index e4f352c..6b83647 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -71,6 +71,7 @@
 #include "storage/smgr.h"
 #include "utils/acl.h"
 #include "utils/builtins.h"
+#include "utils/datum.h"
 #include "utils/fmgroids.h"
 #include "utils/inval.h"
 #include "utils/lsyscache.h"
@@ -141,7 +142,7 @@ typedef struct AlteredTableInfo
 	List	   *constraints;	/* List of NewConstraint */
 	List	   *newvals;		/* List of NewColumnValue */
 	bool		new_notnull;	/* T if we added new NOT NULL constraints */
-	bool		new_changeoids; /* T if we added/dropped the OID column */
+	bool		rewrite;		/* T if we a rewrite is forced */
 	Oid			newTableSpace;	/* new tablespace; 0 means no change */
 	/* Objects to rebuild after completing ALTER TYPE operations */
 	List	   *changedConstraintOids;	/* OIDs of constraints to rebuild */
@@ -336,6 +337,7 @@ static void ATPrepAlterColumnType(List **wqueue,
 					  AlteredTableInfo *tab, Relation rel,
 					  bool recurse, bool recursing,
 					  AlterTableCmd *cmd, LOCKMODE lockmode);
+static bool ATColumnChangeRequiresRewrite(Node *expr, AttrNumber varattno);
 static void ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
 					  const char *colName, TypeName *typeName, LOCKMODE lockmode);
 static void ATPostAlterTypeCleanup(List **wqueue, AlteredTableInfo *tab, LOCKMODE lockmode);
@@ -3219,10 +3221,32 @@ ATRewriteTables(List **wqueue, LOCKMODE lockmode)
 			continue;
 
 		/*
+		 * If we change column data types or add/remove OIDs, the operation
+		 * has to be propagated to tables that use this table's rowtype as a
+		 * column type.  newrel will also be non-NULL in the case where we're
+		 * adding a column with a default.  We choose to forbid that case as
+		 * well, since composite types might eventually support defaults.
+		 *
+		 * (Eventually we'll probably need to check for composite type
+		 * dependencies even when we're just scanning the table without a
+		 * rewrite, but at the moment a composite type does not enforce any
+		 * constraints, so it's not necessary/appropriate to enforce them
+		 * just during ALTER.)
+		 */
+		if (tab->newvals != NIL || tab->rewrite)
+		{
+			Relation	rel;
+
+			rel = heap_open(tab->relid, NoLock);
+			find_composite_type_dependencies(rel->rd_rel->reltype, rel, NULL);
+			heap_close(rel, NoLock);
+		}
+
+		/*
 		 * We only need to rewrite the table if at least one column needs to
 		 * be recomputed, or we are adding/removing the OID column.
 		 */
-		if (tab->newvals != NIL || tab->new_changeoids)
+		if (tab->rewrite)
 		{
 			/* Build a temporary relation and copy data */
 			Relation	OldHeap;
@@ -3409,22 +3433,6 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
 	}
 
 	/*
-	 * If we change column data types or add/remove OIDs, the operation has to
-	 * be propagated to tables that use this table's rowtype as a column type.
-	 * newrel will also be non-NULL in the case where we're adding a column
-	 * with a default.  We choose to forbid that case as well, since composite
-	 * types might eventually support defaults.
-	 *
-	 * (Eventually we'll probably need to check for composite type
-	 * dependencies even when we're just scanning the table without a rewrite,
-	 * but at the moment a composite type does not enforce any constraints,
-	 * so it's not necessary/appropriate to enforce them just during ALTER.)
-	 */
-	if (newrel)
-		find_composite_type_dependencies(oldrel->rd_rel->reltype,
-										 oldrel, NULL);
-
-	/*
 	 * Generate the constraint and default execution states
 	 */
 
@@ -3490,6 +3498,15 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
 		List	   *dropped_attrs = NIL;
 		ListCell   *lc;
 
+		if (newrel)
+			ereport(DEBUG1,
+					(errmsg("rewriting table \"%s\"",
+							RelationGetRelationName(oldrel))));
+		else
+			ereport(DEBUG1,
+					(errmsg("verifying table \"%s\"",
+							RelationGetRelationName(oldrel))));
+
 		econtext = GetPerTupleExprContext(estate);
 
 		/*
@@ -3532,7 +3549,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
 
 		while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
 		{
-			if (newrel)
+			if (tab->rewrite)
 			{
 				Oid			tupOid = InvalidOid;
 
@@ -4330,6 +4347,7 @@ ATExecAddColumn(AlteredTableInfo *tab, Relation rel,
 			newval->expr = defval;
 
 			tab->newvals = lappend(tab->newvals, newval);
+			tab->rewrite = true;
 		}
 
 		/*
@@ -4346,7 +4364,7 @@ ATExecAddColumn(AlteredTableInfo *tab, Relation rel,
 	 * table to fix that.
 	 */
 	if (isOid)
-		tab->new_changeoids = true;
+		tab->rewrite = true;
 
 	/*
 	 * Add needed dependency entries for the new column.
@@ -5019,7 +5037,7 @@ ATExecDropColumn(List **wqueue, Relation rel, const char *colName,
 		tab = ATGetQueueEntry(wqueue, rel);
 
 		/* Tell Phase 3 to physically remove the OID column */
-		tab->new_changeoids = true;
+		tab->rewrite = true;
 	}
 }
 
@@ -5043,7 +5061,7 @@ ATExecAddIndex(AlteredTableInfo *tab, Relation rel,
 	/* suppress schema rights check when rebuilding existing index */
 	check_rights = !is_rebuild;
 	/* skip index build if phase 3 will have to rewrite table anyway */
-	skip_build = (tab->newvals != NIL);
+	skip_build = tab->rewrite;
 	/* suppress notices when rebuilding existing index */
 	quiet = is_rebuild;
 
@@ -6002,6 +6020,9 @@ validateForeignKeyConstraint(char *conname,
 	HeapTuple	tuple;
 	Trigger		trig;
 
+	ereport(DEBUG1,
+			(errmsg("validating foreign key constraint \"%s\"", conname)));
+
 	/*
 	 * Build a trigger call structure; we'll need it either way.
 	 */
@@ -6560,6 +6581,8 @@ ATPrepAlterColumnType(List **wqueue,
 		newval->expr = (Expr *) transform;
 
 		tab->newvals = lappend(tab->newvals, newval);
+		if (ATColumnChangeRequiresRewrite(transform, attnum))
+			tab->rewrite = true;
 	}
 	else if (tab->relkind == RELKIND_FOREIGN_TABLE)
 	{
@@ -6599,6 +6622,29 @@ ATPrepAlterColumnType(List **wqueue,
 		ATTypedTableRecursion(wqueue, rel, cmd, lockmode);
 }
 
+/*
+ * When the data type of a column is changed, a rewrite might not be require
+ * if the data type is being changed to its current type, or more interestingly
+ * to a type to which it is binary coercible.  But we must check carefully that
+ * the USING clause isn't trying to insert some other value.
+ */
+static bool
+ATColumnChangeRequiresRewrite(Node *expr, AttrNumber varattno)
+{
+	Assert(expr != NULL);
+
+	for (;;)
+	{
+		/* only one varno, so no need to check that */
+		if (IsA(expr, Var) && ((Var *) expr)->varattno == varattno)
+			return false;
+		else if (IsA(expr, RelabelType))
+			expr = (Node *) ((RelabelType *) expr)->arg;
+		else
+			return true;
+	}
+}
+
 static void
 ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
 					  const char *colName, TypeName *typeName, LOCKMODE lockmode)
#28Noah Misch
noah@leadboat.com
In reply to: Robert Haas (#27)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

Hi Robert,

On Fri, Feb 11, 2011 at 10:27:11AM -0500, Robert Haas wrote:

On Thu, Jan 27, 2011 at 2:48 PM, Noah Misch <noah@leadboat.com> wrote:

Done as attached. ?This preserves compatibility with our current handling of
composite type dependencies. ?The rest you've seen before.

OK, I finally got back to this. Sorry for the delay. I still feel
that it's possible to accomplish the same goal with less complexity.
See what you think of the attached.

It's a nice, clean patch. However, it achieves that by targeting a smaller
goal. Specifically, it omits:

1) Test cases and DEBUG messages sufficient to facilitate them
2) Skip rewrite for T -> constraint-free domain over T
3) Downgrade rewrite to scan for T -> constrained domain over T

Upon examination, it appeared to me that trying to make the
AlteredTableInfo contain an enum indicating whether we were doing a
scan, a rewrite, or nothing was requiring more code change than I
could really justify.

Offhand, I count 12 changed lines to introduce the enum. There may be other
good reasons not to do it, but surely that wasn't it?

What I ended up doing is replacing the 'bool
new_changedoids' member with a 'bool rewrite' member. I'm pretty sure
this is safe. The only reason we ever tested the new_changeoids
member was to see whether we needed to do a rewrite; it wasn't used
for anything else. The new member gets set either when we need to do
a rewrite, or when a column type changes in a fashion that requires a
rewrite. It's pretty easy to verify that this doesn't result in any
behavior change, except for one corner case: currently, if you add or
remove OIDs to/from a table and in the same ALTER TABLE command add a
constraint that involves creating an index, such as a primary key, the
new primary key index will get built, thrown away, and rebuilt a
second time. With this change, we just build it once, which seems
like an improvement, even though the case is vanishingly unlikely to
ever happen in practice.

This is fairly similar to the design in my first patch. The name was different
(new_bits), and that patch had an extra bool for scan-only cases. I won't
object to moving back to this, but I did find that your enum suggestion worked
out significantly better.

Even supposing we push off all scan-only cases to another patch, it would be
good to have the tablecmds.c-internal representation of that in mind. No sense
in simplifying a 12-line change to an 8-line change, only to redo it next patch.

I also have to say that after playing with a little bit, I find the
new debugging messages you added to be quite invaluable for
understanding what's really going on. The old output told you
basically nothing useful, even if you cranked it all the way up to
DEBUG3. This is much better.

Thanks. I added them solely to make automated testing possible, but they did
turn out to have user value. I'll certainly make use of them when rewriting an
inheritance tree of tables or a large table with many indexes.

Hunk-specific comments follow, largely concerning documentation. I really,
really don't want to get mired in a discussion of exact documentation text. In
fact I think I'd rather hunt crocodiles barefoot, armed with nothing but ALTER
TABLE ... DISABLE TRIGGER ALL than have such a discussion. But I'll articulate
the rationale behind my original constructions, in case they include facts you
did not consider when rewriting them.

diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 9f02674..e3ceea8 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -766,9 +766,13 @@ ALTER TABLE <replaceable class="PARAMETER">name</replaceable>
<para>
Adding a column with a non-null default or changing the type of an
existing column will require the entire table and indexes to be rewritten.
-    This might take a significant amount of time for a large table; and it will
-    temporarily require double the disk space.  Adding or removing a system
-    <literal>oid</> column likewise requires rewriting the entire table.
+    As an exception, if the old type and new types are binary compatible and

In the documentation for CREATE CAST, we define "binary compatible" using "Two
types that are binary coercible both ways are also referred to as binary
compatible." This feature does not require binary compatibility, merely a
one-way binary coercion. Generally, though, I like where you're going. My
version was accurate but obtuse.

+ the <literal>USING</> clause does not change the column contents, the

This is probably fine, but note that "USING col || ''" does not change the
column contents, yet we won't optimize it.

+ table rewrite will be skipped, but the index rebuild is still required.

This wrongly suggests that the rebuild mentioned earlier in the paragraph,
affecting all indexes, will take place. It's a more-limited rebuild.

+    Adding or removing a system <literal>oid</> column likewise requires
+    rewriting the entire table.  Table and/or index rebuilds may take a
+    significant amount of time for a large table; and will temporarily require
+    as much as double the disk space.
</para>
<para>
@@ -797,9 +801,9 @@ ALTER TABLE <replaceable class="PARAMETER">name</replaceable>
<para>
To force an immediate rewrite of the table, you can use
<link linkend="SQL-VACUUM">VACUUM FULL</>, <xref linkend="SQL-CLUSTER">
-    or one of the forms of ALTER TABLE that forces a rewrite, such as
-    SET DATA TYPE.  This results in no semantically-visible change in the
-    table, but gets rid of no-longer-useful data.
+    or one of the forms of ALTER TABLE that forces a rewrite.  This results in
+    no semantically-visible change in the table, but gets rid of
+    no-longer-useful data.

This presents three options without any indication of how to choose between
them. A user wanting just a rewrite should look no further than VACUUM FULL.
We should document that ALTER TABLE can rewrite, both for general user planning
and to dispel thoughts of doing both a VACUUM FULL and an ALTER TABLE in short
succession. However, it doesn't help to advertise ALTER TABLE or CLUSTER as
rewrite tools per se.

Before 9.0, the situation was entirely different. ALTER TABLE was far and away
*the best* rewrite tool. That's why I specifically documented the fact that
pre-9.0 users should transition to VACUUM FULL.

</para>

<para>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9e60121..452ced6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -1685,6 +1685,11 @@ index_build(Relation heapRelation,
procedure = indexRelation->rd_am->ambuild;
Assert(RegProcedureIsValid(procedure));
+	ereport(DEBUG1,
+			(errmsg("building index \"%s\" on table \"%s\"",
+					RelationGetRelationName(indexRelation),
+					RelationGetRelationName(heapRelation))));
+
/*
* Switch to the table owner's userid, so that any index functions are run
* as that user.  Also lock down security-restricted operations and
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index e4f352c..6b83647 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -71,6 +71,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"

This header is not needed in your version, is it?

#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -141,7 +142,7 @@ typedef struct AlteredTableInfo
List	   *constraints;	/* List of NewConstraint */
List	   *newvals;		/* List of NewColumnValue */
bool		new_notnull;	/* T if we added new NOT NULL constraints */
-	bool		new_changeoids; /* T if we added/dropped the OID column */
+	bool		rewrite;		/* T if we a rewrite is forced */
Oid			newTableSpace;	/* new tablespace; 0 means no change */
/* Objects to rebuild after completing ALTER TYPE operations */
List	   *changedConstraintOids;	/* OIDs of constraints to rebuild */
@@ -336,6 +337,7 @@ static void ATPrepAlterColumnType(List **wqueue,
AlteredTableInfo *tab, Relation rel,
bool recurse, bool recursing,
AlterTableCmd *cmd, LOCKMODE lockmode);
+static bool ATColumnChangeRequiresRewrite(Node *expr, AttrNumber varattno);

Moving this from parse_coerce.c to tablecmds.c does make sense.

Thanks,
nm

#29Robert Haas
robertmhaas@gmail.com
In reply to: Noah Misch (#28)
1 attachment(s)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Fri, Feb 11, 2011 at 1:08 PM, Noah Misch <noah@leadboat.com> wrote:

It's a nice, clean patch.  However, it achieves that by targeting a smaller
goal.  Specifically, it omits:

1) Test cases and DEBUG messages sufficient to facilitate them

That was an intentional omission.

2) Skip rewrite for T -> constraint-free domain over T
3) Downgrade rewrite to scan for T -> constrained domain over T

These were not.

Upon examination, it appeared to me that trying to make the
AlteredTableInfo contain an enum indicating whether we were doing a
scan, a rewrite, or nothing was requiring more code change than I
could really justify.

Offhand, I count 12 changed lines to introduce the enum.  There may be other
good reasons not to do it, but surely that wasn't it?

I was unable to see what were getting out of that logic, and I
couldn't readily verify that it was correct.

@@ -766,9 +766,13 @@ ALTER TABLE <replaceable class="PARAMETER">name</replaceable>
    <para>
     Adding a column with a non-null default or changing the type of an
     existing column will require the entire table and indexes to be rewritten.
-    This might take a significant amount of time for a large table; and it will
-    temporarily require double the disk space.  Adding or removing a system
-    <literal>oid</> column likewise requires rewriting the entire table.
+    As an exception, if the old type and new types are binary compatible and

In the documentation for CREATE CAST, we define "binary compatible" using "Two
types that are binary coercible both ways are also referred to as binary
compatible."  This feature does not require binary compatibility, merely a
one-way binary coercion.  Generally, though, I like where you're going.  My
version was accurate but obtuse.

OK, I have touched up that language in the attached version.

+    the <literal>USING</> clause does not change the column contents, the

This is probably fine, but note that "USING col || ''" does not change the
column contents, yet we won't optimize it.

True; but I think we can let the fine user figure that out for themselves. :-)

+    table rewrite will be skipped, but the index rebuild is still required.

This wrongly suggests that the rebuild mentioned earlier in the paragraph,
affecting all indexes, will take place.  It's a more-limited rebuild.

Ah, OK. I've changed the wording there.

This presents three options without any indication of how to choose between
them.  A user wanting just a rewrite should look no further than VACUUM FULL.
We should document that ALTER TABLE can rewrite, both for general user planning
and to dispel thoughts of doing both a VACUUM FULL and an ALTER TABLE in short
succession.  However, it doesn't help to advertise ALTER TABLE or CLUSTER as
rewrite tools per se.

I was wondering if we should try to be more clear about that, but I
think it's a separate issue from this patch.

--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -71,6 +71,7 @@
 #include "storage/smgr.h"
 #include "utils/acl.h"
 #include "utils/builtins.h"
+#include "utils/datum.h"

This header is not needed in your version, is it?

Looks like it isn't, thanks.

Moving this from parse_coerce.c to tablecmds.c does make sense.

OK, glad you agree. It seemed sensible to me..

I think what I'd like to do if there are no major objections is commit
this as-is, and then if you could perhaps provide a patch containing
the set of changes that are necessary to recapture the cases I
inadvertently failed to handle, namely:

2) Skip rewrite for T -> constraint-free domain over T
3) Downgrade rewrite to scan for T -> constrained domain over T

Then I'll review that separately. I think this change stands on its
own, and committing it in steps will be simple for me than doing the
whole thing in one go.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Attachments:

alter-type-2-rmh-v2.patchapplication/octet-stream; name=alter-type-2-rmh-v2.patchDownload
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 9f02674..7e6e72f 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -766,9 +766,13 @@ ALTER TABLE <replaceable class="PARAMETER">name</replaceable>
    <para>
     Adding a column with a non-null default or changing the type of an
     existing column will require the entire table and indexes to be rewritten.
-    This might take a significant amount of time for a large table; and it will
-    temporarily require double the disk space.  Adding or removing a system
-    <literal>oid</> column likewise requires rewriting the entire table.
+    As an exception, if the old type type is binary coercible to the new
+    type and the <literal>USING</> clause does not change the column contents,
+    a table rewrite is not needed, but any indexes on the affected columns 
+    must still be rebuilt.  Adding or removing a system <literal>oid</> column
+    also requires rewriting the entire table.  Table and/or index rebuilds may
+    take a significant amount of time for a large table; and will temporarily
+    require as much as double the disk space.
    </para>
 
    <para>
@@ -797,9 +801,9 @@ ALTER TABLE <replaceable class="PARAMETER">name</replaceable>
    <para>
     To force an immediate rewrite of the table, you can use
     <link linkend="SQL-VACUUM">VACUUM FULL</>, <xref linkend="SQL-CLUSTER">
-    or one of the forms of ALTER TABLE that forces a rewrite, such as
-    SET DATA TYPE.  This results in no semantically-visible change in the
-    table, but gets rid of no-longer-useful data.
+    or one of the forms of ALTER TABLE that forces a rewrite.  This results in
+    no semantically-visible change in the table, but gets rid of
+    no-longer-useful data.
    </para>
 
    <para>
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9e60121..452ced6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -1685,6 +1685,11 @@ index_build(Relation heapRelation,
 	procedure = indexRelation->rd_am->ambuild;
 	Assert(RegProcedureIsValid(procedure));
 
+	ereport(DEBUG1,
+			(errmsg("building index \"%s\" on table \"%s\"",
+					RelationGetRelationName(indexRelation),
+					RelationGetRelationName(heapRelation))));
+
 	/*
 	 * Switch to the table owner's userid, so that any index functions are run
 	 * as that user.  Also lock down security-restricted operations and
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index e4f352c..d849018 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -141,7 +141,7 @@ typedef struct AlteredTableInfo
 	List	   *constraints;	/* List of NewConstraint */
 	List	   *newvals;		/* List of NewColumnValue */
 	bool		new_notnull;	/* T if we added new NOT NULL constraints */
-	bool		new_changeoids; /* T if we added/dropped the OID column */
+	bool		rewrite;		/* T if we a rewrite is forced */
 	Oid			newTableSpace;	/* new tablespace; 0 means no change */
 	/* Objects to rebuild after completing ALTER TYPE operations */
 	List	   *changedConstraintOids;	/* OIDs of constraints to rebuild */
@@ -336,6 +336,7 @@ static void ATPrepAlterColumnType(List **wqueue,
 					  AlteredTableInfo *tab, Relation rel,
 					  bool recurse, bool recursing,
 					  AlterTableCmd *cmd, LOCKMODE lockmode);
+static bool ATColumnChangeRequiresRewrite(Node *expr, AttrNumber varattno);
 static void ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
 					  const char *colName, TypeName *typeName, LOCKMODE lockmode);
 static void ATPostAlterTypeCleanup(List **wqueue, AlteredTableInfo *tab, LOCKMODE lockmode);
@@ -3219,10 +3220,32 @@ ATRewriteTables(List **wqueue, LOCKMODE lockmode)
 			continue;
 
 		/*
+		 * If we change column data types or add/remove OIDs, the operation
+		 * has to be propagated to tables that use this table's rowtype as a
+		 * column type.  newrel will also be non-NULL in the case where we're
+		 * adding a column with a default.  We choose to forbid that case as
+		 * well, since composite types might eventually support defaults.
+		 *
+		 * (Eventually we'll probably need to check for composite type
+		 * dependencies even when we're just scanning the table without a
+		 * rewrite, but at the moment a composite type does not enforce any
+		 * constraints, so it's not necessary/appropriate to enforce them
+		 * just during ALTER.)
+		 */
+		if (tab->newvals != NIL || tab->rewrite)
+		{
+			Relation	rel;
+
+			rel = heap_open(tab->relid, NoLock);
+			find_composite_type_dependencies(rel->rd_rel->reltype, rel, NULL);
+			heap_close(rel, NoLock);
+		}
+
+		/*
 		 * We only need to rewrite the table if at least one column needs to
 		 * be recomputed, or we are adding/removing the OID column.
 		 */
-		if (tab->newvals != NIL || tab->new_changeoids)
+		if (tab->rewrite)
 		{
 			/* Build a temporary relation and copy data */
 			Relation	OldHeap;
@@ -3409,22 +3432,6 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
 	}
 
 	/*
-	 * If we change column data types or add/remove OIDs, the operation has to
-	 * be propagated to tables that use this table's rowtype as a column type.
-	 * newrel will also be non-NULL in the case where we're adding a column
-	 * with a default.  We choose to forbid that case as well, since composite
-	 * types might eventually support defaults.
-	 *
-	 * (Eventually we'll probably need to check for composite type
-	 * dependencies even when we're just scanning the table without a rewrite,
-	 * but at the moment a composite type does not enforce any constraints,
-	 * so it's not necessary/appropriate to enforce them just during ALTER.)
-	 */
-	if (newrel)
-		find_composite_type_dependencies(oldrel->rd_rel->reltype,
-										 oldrel, NULL);
-
-	/*
 	 * Generate the constraint and default execution states
 	 */
 
@@ -3490,6 +3497,15 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
 		List	   *dropped_attrs = NIL;
 		ListCell   *lc;
 
+		if (newrel)
+			ereport(DEBUG1,
+					(errmsg("rewriting table \"%s\"",
+							RelationGetRelationName(oldrel))));
+		else
+			ereport(DEBUG1,
+					(errmsg("verifying table \"%s\"",
+							RelationGetRelationName(oldrel))));
+
 		econtext = GetPerTupleExprContext(estate);
 
 		/*
@@ -3532,7 +3548,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
 
 		while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
 		{
-			if (newrel)
+			if (tab->rewrite)
 			{
 				Oid			tupOid = InvalidOid;
 
@@ -4330,6 +4346,7 @@ ATExecAddColumn(AlteredTableInfo *tab, Relation rel,
 			newval->expr = defval;
 
 			tab->newvals = lappend(tab->newvals, newval);
+			tab->rewrite = true;
 		}
 
 		/*
@@ -4346,7 +4363,7 @@ ATExecAddColumn(AlteredTableInfo *tab, Relation rel,
 	 * table to fix that.
 	 */
 	if (isOid)
-		tab->new_changeoids = true;
+		tab->rewrite = true;
 
 	/*
 	 * Add needed dependency entries for the new column.
@@ -5019,7 +5036,7 @@ ATExecDropColumn(List **wqueue, Relation rel, const char *colName,
 		tab = ATGetQueueEntry(wqueue, rel);
 
 		/* Tell Phase 3 to physically remove the OID column */
-		tab->new_changeoids = true;
+		tab->rewrite = true;
 	}
 }
 
@@ -5043,7 +5060,7 @@ ATExecAddIndex(AlteredTableInfo *tab, Relation rel,
 	/* suppress schema rights check when rebuilding existing index */
 	check_rights = !is_rebuild;
 	/* skip index build if phase 3 will have to rewrite table anyway */
-	skip_build = (tab->newvals != NIL);
+	skip_build = tab->rewrite;
 	/* suppress notices when rebuilding existing index */
 	quiet = is_rebuild;
 
@@ -6002,6 +6019,9 @@ validateForeignKeyConstraint(char *conname,
 	HeapTuple	tuple;
 	Trigger		trig;
 
+	ereport(DEBUG1,
+			(errmsg("validating foreign key constraint \"%s\"", conname)));
+
 	/*
 	 * Build a trigger call structure; we'll need it either way.
 	 */
@@ -6560,6 +6580,8 @@ ATPrepAlterColumnType(List **wqueue,
 		newval->expr = (Expr *) transform;
 
 		tab->newvals = lappend(tab->newvals, newval);
+		if (ATColumnChangeRequiresRewrite(transform, attnum))
+			tab->rewrite = true;
 	}
 	else if (tab->relkind == RELKIND_FOREIGN_TABLE)
 	{
@@ -6599,6 +6621,29 @@ ATPrepAlterColumnType(List **wqueue,
 		ATTypedTableRecursion(wqueue, rel, cmd, lockmode);
 }
 
+/*
+ * When the data type of a column is changed, a rewrite might not be require
+ * if the data type is being changed to its current type, or more interestingly
+ * to a type to which it is binary coercible.  But we must check carefully that
+ * the USING clause isn't trying to insert some other value.
+ */
+static bool
+ATColumnChangeRequiresRewrite(Node *expr, AttrNumber varattno)
+{
+	Assert(expr != NULL);
+
+	for (;;)
+	{
+		/* only one varno, so no need to check that */
+		if (IsA(expr, Var) && ((Var *) expr)->varattno == varattno)
+			return false;
+		else if (IsA(expr, RelabelType))
+			expr = (Node *) ((RelabelType *) expr)->arg;
+		else
+			return true;
+	}
+}
+
 static void
 ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
 					  const char *colName, TypeName *typeName, LOCKMODE lockmode)
#30Robert Haas
robertmhaas@gmail.com
In reply to: Noah Misch (#28)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Fri, Feb 11, 2011 at 1:08 PM, Noah Misch <noah@leadboat.com> wrote:

Even supposing we push off all scan-only cases to another patch, it would be
good to have the tablecmds.c-internal representation of that in mind.  No sense
in simplifying a 12-line change to an 8-line change, only to redo it next patch.

Incidentally, I don't really agree with this, as a philosophical
point. There can be a lot of point to simplifying things, even if it
means redoing a little work, if it makes them easier to understand,
both for the people reviewing at the time and for the benefit of
people reading the commit log in the future.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#31Noah Misch
noah@leadboat.com
In reply to: Robert Haas (#29)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Fri, Feb 11, 2011 at 01:55:45PM -0500, Robert Haas wrote:

On Fri, Feb 11, 2011 at 1:08 PM, Noah Misch <noah@leadboat.com> wrote:

Even supposing we push off all scan-only cases to another patch, it would be
good to have the tablecmds.c-internal representation of that in mind. ?No sense
in simplifying a 12-line change to an 8-line change, only to redo it next patch.

Incidentally, I don't really agree with this, as a philosophical
point. There can be a lot of point to simplifying things, even if it
means redoing a little work, if it makes them easier to understand,
both for the people reviewing at the time and for the benefit of
people reading the commit log in the future.

Good to know. I can envision that perspective, and I share it when the savings
is rather more substantial, say >10% of the patch or >100 lines. Below that
threshold, the energy I expend grasping two interface changes in one patch
series exceeds my annoyance at the premature generality.

On Fri, Feb 11, 2011 at 01:34:14PM -0500, Robert Haas wrote:

On Fri, Feb 11, 2011 at 1:08 PM, Noah Misch <noah@leadboat.com> wrote:

Upon examination, it appeared to me that trying to make the
AlteredTableInfo contain an enum indicating whether we were doing a
scan, a rewrite, or nothing was requiring more code change than I
could really justify.

Offhand, I count 12 changed lines to introduce the enum. ?There may be other
good reasons not to do it, but surely that wasn't it?

I was unable to see what were getting out of that logic, and I
couldn't readily verify that it was correct.

The value probably gets clearer when you need to use all three states.

I think what I'd like to do if there are no major objections is commit
this as-is, and then if you could perhaps provide a patch containing
the set of changes that are necessary to recapture the cases I
inadvertently failed to handle, namely:

2) Skip rewrite for T -> constraint-free domain over T
3) Downgrade rewrite to scan for T -> constrained domain over T

Then I'll review that separately. I think this change stands on its
own, and committing it in steps will be simple for me than doing the
whole thing in one go.

That works for me. Know that, barring other suggestions, the followup patch
will replace AlteredTableInfo.rewrite with the enum field. Just want to make
sure that's not a surprise. ... And now that I've read your second reply, I
probably didn't even have to mention it.

Thanks again,
nm

#32Robert Haas
robertmhaas@gmail.com
In reply to: Noah Misch (#31)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Fri, Feb 11, 2011 at 2:17 PM, Noah Misch <noah@leadboat.com> wrote:

Good to know.  I can envision that perspective, and I share it when the savings
is rather more substantial, say >10% of the patch or >100 lines.  Below that
threshold, the energy I expend grasping two interface changes in one patch
series exceeds my annoyance at the premature generality.

It's a fair point.

One other thing that may be worth mentioning is that you're diving
into the community at a fairly high level. It's obvious that you have
a pretty solid understanding of the code, better than mine in some
areas, and I don't want to be the dumb guy who won't commit your patch
because I'm too, like, dumb. OTOH, as you've probably realized, our
community dynamic is that the committer is the one who gets yelled at
when something is broken.

Then I'll review that separately.  I think this change stands on its
own, and committing it in steps will be simple for me than doing the
whole thing in one go.

That works for me.  Know that, barring other suggestions, the followup patch
will replace AlteredTableInfo.rewrite with the enum field.  Just want to make
sure that's not a surprise. ... And now that I've read your second reply, I
probably didn't even have to mention it.

It's all good.

You might want to consider a second boolean in lieu of a three way
enum. I'm not sure if that's cleaner but if it lets you write:

if (blah)
at->verify = true;

instead of:

if (blah)
at->worklevel = Min(at->worklevel, WORK_VERIFY);

...then I think that might be cleaner.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#33Noah Misch
noah@leadboat.com
In reply to: Robert Haas (#32)
1 attachment(s)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Fri, Feb 11, 2011 at 02:49:27PM -0500, Robert Haas wrote:

You might want to consider a second boolean in lieu of a three way
enum. I'm not sure if that's cleaner but if it lets you write:

if (blah)
at->verify = true;

instead of:

if (blah)
at->worklevel = Min(at->worklevel, WORK_VERIFY);

...then I think that might be cleaner.

Good point; the Max() calls did not make much sense all by themselves. The
point was to make sure nothing decreased the worklevel. Wrapping them in a
macro, say, ATRequireWork, probably would have helped.

That said, I've tried both constructions, and I marginally prefer the end result
with AlteredTableInfo.verify. I've inlined ATColumnChangeRequiresRewrite into
ATPrepAlterColumnType; it would need to either pass back two bools or take an
AlteredTableInfo arg to mutate, so this seemed cleaner. I've omitted the
assertion that my previous version added to ATRewriteTable; it was helpful for
other scan-only type changes, but it's excessive for domains alone. Otherwise,
the differences are cosmetic.

The large block in ATRewriteTable is now superfluous. For easier review, I
haven't removed it.

I missed a typo in the last patch: "T if we a rewrite is forced". Not changed
in this patch as I assume you'll want to commit it separately.

nm

Attachments:

at2v5-domains.patchtext/plain; charset=us-asciiDownload
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 452ced6..466be25 100644
diff --git a/src/backend/commands/index 1db42d0..2e10661 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 142,147 **** typedef struct AlteredTableInfo
--- 142,148 ----
  	List	   *newvals;		/* List of NewColumnValue */
  	bool		new_notnull;	/* T if we added new NOT NULL constraints */
  	bool		rewrite;		/* T if we a rewrite is forced */
+ 	bool		verify;			/* T if we shall verify constraints */
  	Oid			newTableSpace;	/* new tablespace; 0 means no change */
  	/* Objects to rebuild after completing ALTER TYPE operations */
  	List	   *changedConstraintOids;	/* OIDs of constraints to rebuild */
***************
*** 336,342 **** static void ATPrepAlterColumnType(List **wqueue,
  					  AlteredTableInfo *tab, Relation rel,
  					  bool recurse, bool recursing,
  					  AlterTableCmd *cmd, LOCKMODE lockmode);
- static bool ATColumnChangeRequiresRewrite(Node *expr, AttrNumber varattno);
  static void ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
  					  const char *colName, TypeName *typeName, LOCKMODE lockmode);
  static void ATPostAlterTypeCleanup(List **wqueue, AlteredTableInfo *tab, LOCKMODE lockmode);
--- 337,342 ----
***************
*** 3242,3247 **** ATRewriteTables(List **wqueue, LOCKMODE lockmode)
--- 3242,3251 ----
  			heap_close(rel, NoLock);
  		}
  
+ 		/* New NOT NULL constraints always require a scan. */
+ 		if (tab->new_notnull)
+ 			tab->verify = true;
+ 
  		/*
  		 * We only need to rewrite the table if at least one column needs to
  		 * be recomputed, or we are adding/removing the OID column.
***************
*** 3313,3319 **** ATRewriteTables(List **wqueue, LOCKMODE lockmode)
  			 * Test the current data within the table against new constraints
  			 * generated by ALTER TABLE commands, but don't rebuild data.
  			 */
! 			if (tab->constraints != NIL || tab->new_notnull)
  				ATRewriteTable(tab, InvalidOid, lockmode);
  
  			/*
--- 3317,3323 ----
  			 * Test the current data within the table against new constraints
  			 * generated by ALTER TABLE commands, but don't rebuild data.
  			 */
! 			if (tab->verify)
  				ATRewriteTable(tab, InvalidOid, lockmode);
  
  			/*
***************
*** 3387,3393 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  	Relation	newrel;
  	TupleDesc	oldTupDesc;
  	TupleDesc	newTupDesc;
- 	bool		needscan = false;
  	List	   *notnull_attrs;
  	int			i;
  	ListCell   *l;
--- 3391,3396 ----
***************
*** 3446,3452 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  		switch (con->contype)
  		{
  			case CONSTR_CHECK:
- 				needscan = true;
  				con->qualstate = (List *)
  					ExecPrepareExpr((Expr *) con->qual, estate);
  				break;
--- 3449,3454 ----
***************
*** 3481,3491 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  				!newTupDesc->attrs[i]->attisdropped)
  				notnull_attrs = lappend_int(notnull_attrs, i);
  		}
- 		if (notnull_attrs)
- 			needscan = true;
  	}
  
- 	if (newrel || needscan)
  	{
  		ExprContext *econtext;
  		Datum	   *values;
--- 3483,3490 ----
***************
*** 3549,3558 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  
  		while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
  		{
! 			if (tab->rewrite)
! 			{
! 				Oid			tupOid = InvalidOid;
  
  				/* Extract data from old tuple */
  				heap_deform_tuple(tuple, oldTupDesc, values, isnull);
  				if (oldTupDesc->tdhasoid)
--- 3548,3557 ----
  
  		while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
  		{
! 			Oid			tupOid = InvalidOid;
  
+ 			if (newrel)
+ 			{
  				/* Extract data from old tuple */
  				heap_deform_tuple(tuple, oldTupDesc, values, isnull);
  				if (oldTupDesc->tdhasoid)
***************
*** 3561,3570 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  				/* Set dropped attributes to null in new tuple */
  				foreach(lc, dropped_attrs)
  					isnull[lfirst_int(lc)] = true;
  
  				/*
  				 * Process supplied expressions to replace selected columns.
! 				 * Expression inputs come from the old tuple.
  				 */
  				ExecStoreTuple(tuple, oldslot, InvalidBuffer, false);
  				econtext->ecxt_scantuple = oldslot;
--- 3560,3574 ----
  				/* Set dropped attributes to null in new tuple */
  				foreach(lc, dropped_attrs)
  					isnull[lfirst_int(lc)] = true;
+ 			}
  
+ 			if (tab->newvals != NIL)
+ 			{
  				/*
  				 * Process supplied expressions to replace selected columns.
! 				 * Expression inputs come from the old tuple.  If !newrel, we've
! 				 * proven that no expression will change a tuple value, but they
! 				 * may throw errors.
  				 */
  				ExecStoreTuple(tuple, oldslot, InvalidBuffer, false);
  				econtext->ecxt_scantuple = oldslot;
***************
*** 3578,3584 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
--- 3582,3591 ----
  													 &isnull[ex->attnum - 1],
  														  NULL);
  				}
+ 			}
  
+ 			if (newrel)
+ 			{
  				/*
  				 * Form the new tuple. Note that we don't explicitly pfree it,
  				 * since the per-tuple memory context will be reset shortly.
***************
*** 5270,5275 **** ATAddCheckConstraint(List **wqueue, AlteredTableInfo *tab, Relation rel,
--- 5277,5283 ----
  		newcon->qual = (Node *) make_ands_implicit((Expr *) ccon->expr);
  
  		tab->constraints = lappend(tab->constraints, newcon);
+ 		tab->verify = true;
  
  		/* Save the actually assigned name if it was defaulted */
  		if (constr->conname == NULL)
***************
*** 5618,5623 **** ATAddForeignKeyConstraint(AlteredTableInfo *tab, Relation rel,
--- 5626,5635 ----
  		newcon->qual = (Node *) fkconstraint;
  
  		tab->constraints = lappend(tab->constraints, newcon);
+ 		/*
+ 		 * No need to set tab->verify; foreign key validation is a distinct
+ 		 * aspect of Phase 3.
+ 		 */
  	}
  
  	/*
***************
*** 6581,6588 **** ATPrepAlterColumnType(List **wqueue,
  		newval->expr = (Expr *) transform;
  
  		tab->newvals = lappend(tab->newvals, newval);
! 		if (ATColumnChangeRequiresRewrite(transform, attnum))
! 			tab->rewrite = true;
  	}
  	else if (tab->relkind == RELKIND_FOREIGN_TABLE)
  	{
--- 6593,6625 ----
  		newval->expr = (Expr *) transform;
  
  		tab->newvals = lappend(tab->newvals, newval);
! 
! 		/*
! 		 * Decide how much phase-3 per-tuple work this change entails.  None
! 		 * needed when the present type changes to itself, to a constraint-free
! 		 * domain thereon, or to a type to which it is binary coercible.  Any
! 		 * constrained domain requires a verification scan.  Anything else
! 		 * requires a rewrite.  Beware of a USING clause trying to substitute
! 		 * some other value.
! 		 */
! 		while (!tab->rewrite)
! 		{
! 			/* only one varno, so no need to check that */
! 			if (IsA(transform, Var) && ((Var *) transform)->varattno == attnum)
! 				break;
! 			else if (IsA(transform, RelabelType))
! 				transform = (Node *) ((RelabelType *) transform)->arg;
! 			else if (IsA(transform, CoerceToDomain))
! 			{
! 				CoerceToDomain *d = (CoerceToDomain *) transform;
! 
! 				if (GetDomainConstraints(d->resulttype) != NIL)
! 					tab->verify = true;
! 				transform = (Node *) d->arg;
! 			}
! 			else
! 				tab->rewrite = true;
! 		}
  	}
  	else if (tab->relkind == RELKIND_FOREIGN_TABLE)
  	{
***************
*** 6622,6650 **** ATPrepAlterColumnType(List **wqueue,
  		ATTypedTableRecursion(wqueue, rel, cmd, lockmode);
  }
  
- /*
-  * When the data type of a column is changed, a rewrite might not be require
-  * if the data type is being changed to its current type, or more interestingly
-  * to a type to which it is binary coercible.  But we must check carefully that
-  * the USING clause isn't trying to insert some other value.
-  */
- static bool
- ATColumnChangeRequiresRewrite(Node *expr, AttrNumber varattno)
- {
- 	Assert(expr != NULL);
- 
- 	for (;;)
- 	{
- 		/* only one varno, so no need to check that */
- 		if (IsA(expr, Var) && ((Var *) expr)->varattno == varattno)
- 			return false;
- 		else if (IsA(expr, RelabelType))
- 			expr = (Node *) ((RelabelType *) expr)->arg;
- 		else
- 			return true;
- 	}
- }
- 
  static void
  ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
  					  const char *colName, TypeName *typeName, LOCKMODE lockmode)
--- 6659,6664 ----
diff --git a/src/test/regress/GNUmakefiindex 4c8af0f..eeb9487 100644
diff --git a/src/test/regress/expecnew file mode 100644
index 0000000..248fb39
diff --git a/src/test/regress/parallel_schedule b/srindex 3b99e86..0420922 100644
diff --git a/src/test/regress/serial_scheindex b348f0e..9954cf7 100644
diff --git a/src/test/regress/sql/big_anew file mode 100644
index 0000000..8302d87
#34Robert Haas
robertmhaas@gmail.com
In reply to: Noah Misch (#33)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Sat, Feb 12, 2011 at 10:45 AM, Noah Misch <noah@leadboat.com> wrote:

That said, I've tried both constructions, and I marginally prefer the end result
with AlteredTableInfo.verify.  I've inlined ATColumnChangeRequiresRewrite into
ATPrepAlterColumnType; it would need to either pass back two bools or take an
AlteredTableInfo arg to mutate, so this seemed cleaner.

I think I like the idea of passing it the AlteredTableInfo.

I've omitted the
assertion that my previous version added to ATRewriteTable; it was helpful for
other scan-only type changes, but it's excessive for domains alone.  Otherwise,
the differences are cosmetic.

So in the case of a constrained domain, it looks like we're going to
evaluate the changed columns, but if no error is thrown, we're going
to assume they match the original ones and throw out the data? Yikes.
I didn't like that Assert much, but maybe we need it, because this is
scary.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#35Noah Misch
noah@leadboat.com
In reply to: Robert Haas (#34)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Sun, Feb 13, 2011 at 12:04:20AM -0500, Robert Haas wrote:

On Sat, Feb 12, 2011 at 10:45 AM, Noah Misch <noah@leadboat.com> wrote:

That said, I've tried both constructions, and I marginally prefer the end result
with AlteredTableInfo.verify. ?I've inlined ATColumnChangeRequiresRewrite into
ATPrepAlterColumnType; it would need to either pass back two bools or take an
AlteredTableInfo arg to mutate, so this seemed cleaner.

I think I like the idea of passing it the AlteredTableInfo.

Okay.

I've omitted the
assertion that my previous version added to ATRewriteTable; it was helpful for
other scan-only type changes, but it's excessive for domains alone. ?Otherwise,
the differences are cosmetic.

So in the case of a constrained domain, it looks like we're going to
evaluate the changed columns, but if no error is thrown, we're going
to assume they match the original ones and throw out the data?

Correct. We can see that a RelabelType changes no values by inspecting
ExecEvalRelabelType. Likewise, by inspecting ExecEvalCoerceToDomain, we can
know that a CoerceToDomain node may introduce errors but never modified values.

Yikes.
I didn't like that Assert much, but maybe we need it, because this is
scary.

Can you elaborate on the fear-inducing aspect? I don't mind re-adding the
Assert, but it seems that some positive understanding of the assumption's
validity is in order.

nm

#36Robert Haas
robertmhaas@gmail.com
In reply to: Noah Misch (#35)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Sun, Feb 13, 2011 at 6:50 AM, Noah Misch <noah@leadboat.com> wrote:

Yikes.
 I didn't like that Assert much, but maybe we need it, because this is
scary.

Can you elaborate on the fear-inducing aspect?  I don't mind re-adding the
Assert, but it seems that some positive understanding of the assumption's
validity is in order.

Well, it's pretty obvious that if somehow we were to go down that code
path and not get back a value that was identical to the one we had
before, we'd have a corrupted table. It seems that just about any
refactoring of tablecmds.c might create such a possibility.
Assertions are a good idea when the reasons why something is true are
far-removed (in the code) from the places where they need to be true.

I'm somewhat uncomfortable also with the dependency of this code on
very subtle semantic differences between if (newrel) and if
(tab->newvals != NIL). I think this would blow up and die if for any
reason newrel were non-NULL but tab->newvals were NIL. Actually,
doesn't that happen if we're adding or dropping an OID column?

I think it might be cleaner to have tab->verify_constraints rather
than tab->verify. Then ATRewriteTables() could test if
(tab->verify_constraints || tab->new_notnull), and you wouldn't need
the bit that sets at->verify if at->new_notnull is already set. That
part makes the semantics of tab->verify depend on which point in
processing we're at, which I have found to be a recipe for confusion
in other parts of the source base (planner, I'm looking at you).

Correct me if I'm wrong here, but we could handle the
domains-without-contraints part here with about three additional lines
of code, right? It's only the domains-with-constraints part that
requires the rest of this. I'm half-tempted to put that part off to
9.2, in the hopes of getting a more substantial solution that can also
handle things like text -> xml which we don't have time to re-engineer
right now.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#37Noah Misch
noah@leadboat.com
In reply to: Robert Haas (#36)
1 attachment(s)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Sun, Feb 13, 2011 at 02:53:07PM -0500, Robert Haas wrote:

On Sun, Feb 13, 2011 at 6:50 AM, Noah Misch <noah@leadboat.com> wrote:

Yikes.
?I didn't like that Assert much, but maybe we need it, because this is
scary.

Can you elaborate on the fear-inducing aspect? ?I don't mind re-adding the
Assert, but it seems that some positive understanding of the assumption's
validity is in order.

Well, it's pretty obvious that if somehow we were to go down that code
path and not get back a value that was identical to the one we had
before, we'd have a corrupted table.

Certainly. Understand that the first increment of this patch already made a
similar assumption about RelabelType, and CoerceToDomain is no less stable in
this respect. We're exposed to this level of responsibility already. We had
better get the code right, but what else is new?

It seems that just about any
refactoring of tablecmds.c might create such a possibility.

Uhhh. Example?

Assertions are a good idea when the reasons why something is true are
far-removed (in the code) from the places where they need to be true.

Yes. Anyway, since we both clearly like the assertion, I've re-added it.

I'm somewhat uncomfortable also with the dependency of this code on
very subtle semantic differences between if (newrel) and if
(tab->newvals != NIL). I think this would blow up and die if for any
reason newrel were non-NULL but tab->newvals were NIL. Actually,
doesn't that happen if we're adding or dropping an OID column?

Adding or dropping OIDs as the sole operation of the ALTER TABLE does result in
newrel != NULL and tab->newvals == NIL, but the code handles that fine. The
loop just re-forms the tuple from its original values/nulls lists.

I think it might be cleaner to have tab->verify_constraints rather
than tab->verify. Then ATRewriteTables() could test if
(tab->verify_constraints || tab->new_notnull), and you wouldn't need
the bit that sets at->verify if at->new_notnull is already set.

I wouldn't care for that change. Despite the name, NOT NULL and FOREIGN KEY
constraints wouldn't be represented. Casting to a domain to check the domain
constraints is a stretch of the term, and it will be fully obsolete when we
optimize xml -> text and such.

However, I've moved the setting of tab->verify to the points where we set
tab->new_notnull, rather than doing it later in that one place.

Correct me if I'm wrong here, but we could handle the
domains-without-contraints part here with about three additional lines
of code, right? It's only the domains-with-constraints part that
requires the rest of this.

Correct.

I'm half-tempted to put that part off to
9.2, in the hopes of getting a more substantial solution that can also
handle things like text -> xml which we don't have time to re-engineer
right now.

I see.

Anyway, I've attached an updated version with these changes:
- Restored the transform expression walk to its own function
- Assert re-added
- tab->verify set alongside tab->new_notnull, not later

nm

Attachments:

at2v6-domains.patchtext/plain; charset=us-asciiDownload
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 452ced6..466be25 100644
diff --git a/src/backend/commands/index 1db42d0..06942c3 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 71,76 ****
--- 71,77 ----
  #include "storage/smgr.h"
  #include "utils/acl.h"
  #include "utils/builtins.h"
+ #include "utils/datum.h"
  #include "utils/fmgroids.h"
  #include "utils/inval.h"
  #include "utils/lsyscache.h"
***************
*** 142,147 **** typedef struct AlteredTableInfo
--- 143,149 ----
  	List	   *newvals;		/* List of NewColumnValue */
  	bool		new_notnull;	/* T if we added new NOT NULL constraints */
  	bool		rewrite;		/* T if we a rewrite is forced */
+ 	bool		verify;			/* T if we shall verify constraints */
  	Oid			newTableSpace;	/* new tablespace; 0 means no change */
  	/* Objects to rebuild after completing ALTER TYPE operations */
  	List	   *changedConstraintOids;	/* OIDs of constraints to rebuild */
***************
*** 336,342 **** static void ATPrepAlterColumnType(List **wqueue,
  					  AlteredTableInfo *tab, Relation rel,
  					  bool recurse, bool recursing,
  					  AlterTableCmd *cmd, LOCKMODE lockmode);
! static bool ATColumnChangeRequiresRewrite(Node *expr, AttrNumber varattno);
  static void ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
  					  const char *colName, TypeName *typeName, LOCKMODE lockmode);
  static void ATPostAlterTypeCleanup(List **wqueue, AlteredTableInfo *tab, LOCKMODE lockmode);
--- 338,345 ----
  					  AlteredTableInfo *tab, Relation rel,
  					  bool recurse, bool recursing,
  					  AlterTableCmd *cmd, LOCKMODE lockmode);
! static void ATColumnChangeSetWorkLevel(AlteredTableInfo *tab,
! 						   Node *expr, AttrNumber varattno);
  static void ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
  					  const char *colName, TypeName *typeName, LOCKMODE lockmode);
  static void ATPostAlterTypeCleanup(List **wqueue, AlteredTableInfo *tab, LOCKMODE lockmode);
***************
*** 3313,3319 **** ATRewriteTables(List **wqueue, LOCKMODE lockmode)
  			 * Test the current data within the table against new constraints
  			 * generated by ALTER TABLE commands, but don't rebuild data.
  			 */
! 			if (tab->constraints != NIL || tab->new_notnull)
  				ATRewriteTable(tab, InvalidOid, lockmode);
  
  			/*
--- 3316,3322 ----
  			 * Test the current data within the table against new constraints
  			 * generated by ALTER TABLE commands, but don't rebuild data.
  			 */
! 			if (tab->verify)
  				ATRewriteTable(tab, InvalidOid, lockmode);
  
  			/*
***************
*** 3387,3393 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  	Relation	newrel;
  	TupleDesc	oldTupDesc;
  	TupleDesc	newTupDesc;
- 	bool		needscan = false;
  	List	   *notnull_attrs;
  	int			i;
  	ListCell   *l;
--- 3390,3395 ----
***************
*** 3446,3452 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  		switch (con->contype)
  		{
  			case CONSTR_CHECK:
- 				needscan = true;
  				con->qualstate = (List *)
  					ExecPrepareExpr((Expr *) con->qual, estate);
  				break;
--- 3448,3453 ----
***************
*** 3481,3491 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  				!newTupDesc->attrs[i]->attisdropped)
  				notnull_attrs = lappend_int(notnull_attrs, i);
  		}
- 		if (notnull_attrs)
- 			needscan = true;
  	}
  
- 	if (newrel || needscan)
  	{
  		ExprContext *econtext;
  		Datum	   *values;
--- 3482,3489 ----
***************
*** 3549,3558 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  
  		while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
  		{
! 			if (tab->rewrite)
! 			{
! 				Oid			tupOid = InvalidOid;
  
  				/* Extract data from old tuple */
  				heap_deform_tuple(tuple, oldTupDesc, values, isnull);
  				if (oldTupDesc->tdhasoid)
--- 3547,3560 ----
  
  		while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
  		{
! 			Oid			tupOid = InvalidOid;
  
+ 			if (newrel
+ #ifdef USE_ASSERT_CHECKING
+ 				|| assert_enabled
+ #endif
+ 				)
+ 			{
  				/* Extract data from old tuple */
  				heap_deform_tuple(tuple, oldTupDesc, values, isnull);
  				if (oldTupDesc->tdhasoid)
***************
*** 3561,3570 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  				/* Set dropped attributes to null in new tuple */
  				foreach(lc, dropped_attrs)
  					isnull[lfirst_int(lc)] = true;
  
  				/*
  				 * Process supplied expressions to replace selected columns.
! 				 * Expression inputs come from the old tuple.
  				 */
  				ExecStoreTuple(tuple, oldslot, InvalidBuffer, false);
  				econtext->ecxt_scantuple = oldslot;
--- 3563,3577 ----
  				/* Set dropped attributes to null in new tuple */
  				foreach(lc, dropped_attrs)
  					isnull[lfirst_int(lc)] = true;
+ 			}
  
+ 			if (tab->newvals != NIL)
+ 			{
  				/*
  				 * Process supplied expressions to replace selected columns.
! 				 * Expression inputs come from the old tuple.  If !newrel, we've
! 				 * proven that no expression will change a tuple value, but they
! 				 * may throw errors.
  				 */
  				ExecStoreTuple(tuple, oldslot, InvalidBuffer, false);
  				econtext->ecxt_scantuple = oldslot;
***************
*** 3577,3584 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
--- 3584,3618 ----
  														  econtext,
  													 &isnull[ex->attnum - 1],
  														  NULL);
+ 
+ #ifdef USE_ASSERT_CHECKING
+ 					if (assert_enabled)
+ 					{
+ 						Datum		oldval = values[ex->attnum - 1];
+ 						bool		oldisnull = isnull[ex->attnum - 1];
+ 						Form_pg_attribute f = newTupDesc->attrs[ex->attnum - 1];
+ 
+ 						if (f->attbyval && f->attlen == -1)
+ 							oldval = PointerGetDatum(PG_DETOAST_DATUM(oldval));
+ 
+ 						/*
+ 						 * We don't detect the gross error of !newrel when the
+ 						 * typlen actually changed.  attbyval could differ in
+ 						 * theory, but we assume it does not.
+ 						 */
+ 						Assert(newrel ||
+ 							   (isnull[ex->attnum - 1] == oldisnull
+ 								&& (oldisnull ||
+ 									datumIsEqual(oldval,
+ 												 values[ex->attnum - 1],
+ 												 f->attbyval, f->attlen))));
+ 					}
+ #endif
  				}
+ 			}
  
+ 			if (newrel)
+ 			{
  				/*
  				 * Form the new tuple. Note that we don't explicitly pfree it,
  				 * since the per-tuple memory context will be reset shortly.
***************
*** 4357,4362 **** ATExecAddColumn(AlteredTableInfo *tab, Relation rel,
--- 4391,4397 ----
  		 * anything.)
  		 */
  		tab->new_notnull |= colDef->is_not_null;
+ 		tab->verify |= tab->new_notnull;
  	}
  
  	/*
***************
*** 4553,4558 **** ATExecSetNotNull(AlteredTableInfo *tab, Relation rel,
--- 4588,4594 ----
  
  		/* Tell Phase 3 it needs to test the constraint */
  		tab->new_notnull = true;
+ 		tab->verify = true;
  	}
  
  	heap_close(attr_rel, RowExclusiveLock);
***************
*** 5270,5275 **** ATAddCheckConstraint(List **wqueue, AlteredTableInfo *tab, Relation rel,
--- 5306,5312 ----
  		newcon->qual = (Node *) make_ands_implicit((Expr *) ccon->expr);
  
  		tab->constraints = lappend(tab->constraints, newcon);
+ 		tab->verify = true;
  
  		/* Save the actually assigned name if it was defaulted */
  		if (constr->conname == NULL)
***************
*** 5618,5623 **** ATAddForeignKeyConstraint(AlteredTableInfo *tab, Relation rel,
--- 5655,5664 ----
  		newcon->qual = (Node *) fkconstraint;
  
  		tab->constraints = lappend(tab->constraints, newcon);
+ 		/*
+ 		 * No need to set tab->verify; foreign key validation is a distinct
+ 		 * aspect of Phase 3.
+ 		 */
  	}
  
  	/*
***************
*** 6581,6588 **** ATPrepAlterColumnType(List **wqueue,
  		newval->expr = (Expr *) transform;
  
  		tab->newvals = lappend(tab->newvals, newval);
! 		if (ATColumnChangeRequiresRewrite(transform, attnum))
! 			tab->rewrite = true;
  	}
  	else if (tab->relkind == RELKIND_FOREIGN_TABLE)
  	{
--- 6622,6628 ----
  		newval->expr = (Expr *) transform;
  
  		tab->newvals = lappend(tab->newvals, newval);
! 		ATColumnChangeSetWorkLevel(tab, transform, attnum);
  	}
  	else if (tab->relkind == RELKIND_FOREIGN_TABLE)
  	{
***************
*** 6623,6647 **** ATPrepAlterColumnType(List **wqueue,
  }
  
  /*
!  * When the data type of a column is changed, a rewrite might not be require
!  * if the data type is being changed to its current type, or more interestingly
!  * to a type to which it is binary coercible.  But we must check carefully that
!  * the USING clause isn't trying to insert some other value.
   */
! static bool
! ATColumnChangeRequiresRewrite(Node *expr, AttrNumber varattno)
  {
  	Assert(expr != NULL);
  
! 	for (;;)
  	{
  		/* only one varno, so no need to check that */
  		if (IsA(expr, Var) && ((Var *) expr)->varattno == varattno)
! 			return false;
  		else if (IsA(expr, RelabelType))
  			expr = (Node *) ((RelabelType *) expr)->arg;
  		else
! 			return true;
  	}
  }
  
--- 6663,6697 ----
  }
  
  /*
!  * Decide how much phase-3 per-tuple work this change entails.  None needed when
!  * the present type changes to itself, to a constraint-free domain thereon, or
!  * to a type to which it is binary coercible.  Any constrained domain requires a
!  * verification scan.  Anything else requires a rewrite.  Beware of a USING
!  * clause trying to substitute some other value.
   */
! static void
! ATColumnChangeSetWorkLevel(AlteredTableInfo *tab,
! 						   Node *expr, AttrNumber varattno)
  {
  	Assert(expr != NULL);
  
! 	while (!tab->rewrite)
  	{
  		/* only one varno, so no need to check that */
  		if (IsA(expr, Var) && ((Var *) expr)->varattno == varattno)
! 			break;
  		else if (IsA(expr, RelabelType))
  			expr = (Node *) ((RelabelType *) expr)->arg;
+ 		else if (IsA(expr, CoerceToDomain))
+ 		{
+ 			CoerceToDomain *d = (CoerceToDomain *) expr;
+ 
+ 			if (GetDomainConstraints(d->resulttype) != NIL)
+ 				tab->verify = true;
+ 			expr = (Node *) d->arg;
+ 		}
  		else
! 			tab->rewrite = true;
  	}
  }
  
diff --git a/src/test/regress/GNUmakefiindex 4c8af0f..eeb9487 100644
diff --git a/src/test/regress/expecnew file mode 100644
index 0000000..248fb39
diff --git a/src/test/regress/parallel_schedule b/srindex 3b99e86..0420922 100644
diff --git a/src/test/regress/serial_scheindex b348f0e..9954cf7 100644
diff --git a/src/test/regress/sql/big_anew file mode 100644
index 0000000..8302d87
#38Stephen Frost
sfrost@snowman.net
In reply to: Noah Misch (#37)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

Noah,

I'm even less familiar w/ this code than Robert, but figured I'd give a
shot at reviewing this anyway. I definitely like avoiding table
rewrites if I can get away with it. :)

First question is- why do you use #ifdef USE_ASSERT_CHECKING ..?
assert_enabled exists and will work the way you expect regardless, no?
Strikes me as unlikely that the checks would be a real performance
problem..

In ATColumnChangeSetWorkLevel(), I'm really not a huge fan of using a
passed-in argument to move through a list with.. I'd suggest using a
local variable that is set from what's passed in. I do see that's
inheirited, but still, you've pretty much redefined that entire
function anyway..

Also, I feel like that while(!tab->rewrite) really deserves more
explanation of what's happening than the function-level comment above
gives it. I'd prefer to see a comment above the while() explaining
that we're moving through a list to see if there's any level at which
expr is something complicated or is referring to a domain which has
constraints on it (presuming that I've followed what's going on
correctly, that is..). It also seems like it'd make more sense to me to
be a while() controlled by (IsA(expr, Var) && ((Var *) expr)->varattno
== varattno), since that's really the normal "stopping point".

These are all more stylistic issues than anything else. Last, but not
least, I do worry about how this may impact contrib modules, external
projects, or user-added data types, such as PostGIS, hstore, and ip4r.
Could we/should we limit this to only PG data types that we 'know' are
binary compatible? Is there any way or reason such external modules
could be fouled up by this?

Thanks!

Stephen

#39Noah Misch
noah@leadboat.com
In reply to: Stephen Frost (#38)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

Hi Stephen,

Thanks for jumping in on this one.

On Mon, Feb 14, 2011 at 01:12:21PM -0500, Stephen Frost wrote:

First question is- why do you use #ifdef USE_ASSERT_CHECKING ..?

The other six code sites checking assert_enabled directly do the same.

assert_enabled exists and will work the way you expect regardless, no?

Yes.

Strikes me as unlikely that the checks would be a real performance
problem..

Agreed.

In ATColumnChangeSetWorkLevel(), I'm really not a huge fan of using a
passed-in argument to move through a list with.. I'd suggest using a
local variable that is set from what's passed in. I do see that's
inheirited, but still, you've pretty much redefined that entire
function anyway..

Could do that. However, the function would reference the original argument just
once, to make that copy. I'm not seeing a win.

Also, I feel like that while(!tab->rewrite) really deserves more
explanation of what's happening than the function-level comment above
gives it. I'd prefer to see a comment above the while() explaining
that we're moving through a list to see if there's any level at which
expr is something complicated or is referring to a domain which has
constraints on it (presuming that I've followed what's going on
correctly, that is..).

The way I like to think about it is that we're recursively walking an expression
tree to determine which of three categories it falls into: always produces the
same value without error; always produces the same value on success, but may
throw an error; neither of the above. We have a whitelist of node types that
are acceptable, and anything else makes us assume the worst. (The nodes we
accept are simple enough that the recursion degenerates to iteration.) Here's
the comment explaining the algorithm, from an earlier version of the patch:

+ /* GetCoerceExemptions()
+  *		Assess invariants of a coercion expression.
+  *
+  * Various common expressions arising from type coercion are subject to
+  * optimizations.  For example, a simple varchar -> text cast will never change
+  * the underlying data (COERCE_EXEMPT_NOCHANGE) and never yield an error
+  * (COERCE_EXEMPT_NOERROR).  A varchar(8) -> varchar(4) will never change the
+  * data, but it may yield an error.  Given a varno and varattno denoting "the"
+  * source datum, determine which invariants hold for an expression by walking it
+  * per these rules:
+  *
+  *	1. A Var with the varno/varattno in question has both invariants.
+  *	2. A RelabelType node inherits the invariants of its sole argument.
+  *	3. A CoerceToDomain node inherits any COERCE_EXEMPT_NOCHANGE invariant from
+  *		its sole argument.  When GetDomainConstraints() == NIL, it also inherits
+  *		COERCE_EXEMPT_NOERROR.  Otherwise, COERCE_EXEMPT_NOERROR becomes false.
+  *	4. All other nodes have neither invariant.
+  *
+  * Returns a bit string that may contain the following bits:
+  *	COERCE_EXEMPT_NOCHANGE: expression result will always have the same binary
+  *				representation as a Var expression having the given varno and
+  *				varattno
+  *	COERCE_EXEMPT_NOERROR: expression will never throw an error
+  */

The return value changed, but the rest remains accurate. Here's a similar
explanation from the design thread; it covers a more general case:

http://archives.postgresql.org/message-id/20101231013534.GA7521@tornado.leadboat.com

Should I restore some of that? Any other particular text that would have helped?

It also seems like it'd make more sense to me to
be a while() controlled by (IsA(expr, Var) && ((Var *) expr)->varattno
== varattno), since that's really the normal "stopping point".

If we can optimize to some extent, that is the stopping point. If not,
tab->rewrite is the stopping point. I picked the no-optimization case as
"normal" and used that as the loop condition, but one could go either way.

These are all more stylistic issues than anything else. Last, but not
least, I do worry about how this may impact contrib modules, external
projects, or user-added data types, such as PostGIS, hstore, and ip4r.
Could we/should we limit this to only PG data types that we 'know' are
binary compatible? Is there any way or reason such external modules
could be fouled up by this?

External modules are safe. If a binary coercion cast (CREATE CAST ... WITHOUT
FUNCTION) implements the type conversion for an ALTER TABLE ... SET DATA TYPE,
coerce_to_target_type() will inject a RelabelType node, and this code will pick
up on that and avoid the rewrite. If such a cast were defined erroneously, the
user is no less in trouble. He'd get a table rewrite, but the rewrite would
just transfer the bits unchanged. The validity of this optimization does not
rely on any user-settable property of a data type, but it does lean heavily on
the execution semantics of specific nodes.

Thanks,
nm

#40Stephen Frost
sfrost@snowman.net
In reply to: Noah Misch (#39)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

* Noah Misch (noah@leadboat.com) wrote:

On Mon, Feb 14, 2011 at 01:12:21PM -0500, Stephen Frost wrote:

First question is- why do you use #ifdef USE_ASSERT_CHECKING ..?

The other six code sites checking assert_enabled directly do the same.

Wow, I could have sworn that I looked at what others were doing and
didn't see that. Sorry for the noise.

In ATColumnChangeSetWorkLevel(), I'm really not a huge fan of using a
passed-in argument to move through a list with.. I'd suggest using a
local variable that is set from what's passed in. I do see that's
inheirited, but still, you've pretty much redefined that entire
function anyway..

Could do that. However, the function would reference the original argument just
once, to make that copy. I'm not seeing a win.

Perhaps I'm just deficient in this area, but I think of arguments,
unless specifically intended otherwise, to be 'read-only' and based on
that assumption, seeing it come up as an lvalue hits me as wrong.

I'm happy enough to let someone else decide if they agree with me or you
though. :)

The way I like to think about it is that we're recursively walking an expression
tree to determine which of three categories it falls into: always produces the
same value without error; always produces the same value on success, but may
throw an error; neither of the above. We have a whitelist of node types that
are acceptable, and anything else makes us assume the worst. (The nodes we
accept are simple enough that the recursion degenerates to iteration.)

It's that degeneration that definitely hits me as making the whole thing
look a bit 'funny'. When I first looked at the loop, I was looking for
a tree structure and trying to figure out how it could work with just a
simple while().

http://archives.postgresql.org/message-id/20101231013534.GA7521@tornado.leadboat.com

Should I restore some of that? Any other particular text that would have helped?

I definitely think the examples given, enumerating the types of nodes
that matter for this (and why they're the ones we look for), and the
rules that are followed would help a great deal. Anyone else who comes
across this code may be wondering "do I need to do something here for
this new node type that I just added".

It also seems like it'd make more sense to me to
be a while() controlled by (IsA(expr, Var) && ((Var *) expr)->varattno
== varattno), since that's really the normal "stopping point".

If we can optimize to some extent, that is the stopping point. If not,
tab->rewrite is the stopping point. I picked the no-optimization case as
"normal" and used that as the loop condition, but one could go either way.

I would think you could still short-circuit the loop when you've
discovered a case where we have to rewrite the table anyway. Having the
comments updated to reflect what's going on and why stopping on
tab->rewrite, in particular, makes sense is more important though.

The validity of this optimization does not
rely on any user-settable property of a data type, but it does lean heavily on
the execution semantics of specific nodes.

After thinking through this and diving into coerce_to_target_type() a
bit, I'm finally coming to understand how this is working. The gist of
it, if I follow correctly, is that if the planner doesn't think we have
to do anything but copy this value to make it the new data type, then
we're good to go. That makes sense, when you think about it, but boy
does it go the long way around to get there.

Essentially, coerce_to_target_type() is returning an expression tree and
ATColumnChangeSetWorkLevel() is checking to see if that expression tree
is "copy the value". Maybe this is a bit much, but if
coerce_to_target_type() knows the expression given to it is a
straight-up copy, perhaps it could pass that information along in an
easier to use format than an expression tree? That would obviate the
need for ATColumnChangeSetWorkLevel() entirely, if I understand
correctly.

Of course, coerce_to_target_type() is used by lots of other places,
almost all of which probably have to have an expression tree to stuff
into a plan, so maybe a simpler function could be defined which operates
at the level that ATColumnChangeSetWorkLevel() needs? Or perhaps other
places would benefit from knowing that a given conversion is an actual
no-op rather than copying the value?

Just my 2c, I don't believe the patch could cause problems now that I'm
understanding it better, but it sure does seem excessive to use an
expression tree to figure out when something is a no-op.

Thanks,

Stephen

#41Noah Misch
noah@leadboat.com
In reply to: Stephen Frost (#40)
1 attachment(s)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Mon, Feb 14, 2011 at 04:06:59PM -0500, Stephen Frost wrote:

* Noah Misch (noah@leadboat.com) wrote:

On Mon, Feb 14, 2011 at 01:12:21PM -0500, Stephen Frost wrote:

In ATColumnChangeSetWorkLevel(), I'm really not a huge fan of using a
passed-in argument to move through a list with.. I'd suggest using a
local variable that is set from what's passed in. I do see that's
inheirited, but still, you've pretty much redefined that entire
function anyway..

Could do that. However, the function would reference the original argument just
once, to make that copy. I'm not seeing a win.

Perhaps I'm just deficient in this area, but I think of arguments,
unless specifically intended otherwise, to be 'read-only' and based on
that assumption, seeing it come up as an lvalue hits me as wrong.

I'm happy enough to let someone else decide if they agree with me or you
though. :)

Same here. If there's a general project tendency either way, I'll comply. I
haven't noticed one, but I haven't been looking.

I've attached a new version of the patch that attempts to flesh out the comments
based on your feedback. Does it improve things?

The validity of this optimization does not
rely on any user-settable property of a data type, but it does lean heavily on
the execution semantics of specific nodes.

After thinking through this and diving into coerce_to_target_type() a
bit, I'm finally coming to understand how this is working. The gist of
it, if I follow correctly, is that if the planner doesn't think we have
to do anything but copy this value to make it the new data type, then
we're good to go. That makes sense, when you think about it, but boy
does it go the long way around to get there.

Essentially. The planner is not yet involved. We're looking at an expression
tree after parse analysis, before planning.

Essentially, coerce_to_target_type() is returning an expression tree and
ATColumnChangeSetWorkLevel() is checking to see if that expression tree
is "copy the value". Maybe this is a bit much, but if
coerce_to_target_type() knows the expression given to it is a
straight-up copy, perhaps it could pass that information along in an
easier to use format than an expression tree? That would obviate the
need for ATColumnChangeSetWorkLevel() entirely, if I understand
correctly.

PostgreSQL has a strong tradition of passing around expression trees and walking
them to (re-)discover facts. See the various clauses.h functions. Also, when
you have a USING clause, coerce_to_target_type() doesn't know the whole picture.
We could teach it to discover the whole picture, but that would amount to a very
similar tree walk in a different place.

Of course, coerce_to_target_type() is used by lots of other places,
almost all of which probably have to have an expression tree to stuff
into a plan, so maybe a simpler function could be defined which operates
at the level that ATColumnChangeSetWorkLevel() needs?

Offhand, I can't think of any concrete implementation along those lines that
would simplify the overall task. Did you have something more specific in mind?

Or perhaps other
places would benefit from knowing that a given conversion is an actual
no-op rather than copying the value?

RelabelType itself does not cause a copy; the existing datum passes through. In
the case of ALTER TABLE, we do eventually copy the datum via heap_form_tuple.

There may be other places that would benefit from similar analysis. For that
reason, I originally had ATColumnChangeSetWorkLevel() in parse_coerce.c with the
name GetCoerceExemptions(). I'm not aware of any specific applications, though.
For now it seemed like premature abstraction.

Thanks again,
nm

Attachments:

at2v7-domains.patchtext/plain; charset=us-asciiDownload
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 452ced6..466be25 100644
diff --git a/src/backend/commands/index 1db42d0..ab0bcda 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
***************
*** 71,76 ****
--- 71,77 ----
  #include "storage/smgr.h"
  #include "utils/acl.h"
  #include "utils/builtins.h"
+ #include "utils/datum.h"
  #include "utils/fmgroids.h"
  #include "utils/inval.h"
  #include "utils/lsyscache.h"
***************
*** 142,147 **** typedef struct AlteredTableInfo
--- 143,149 ----
  	List	   *newvals;		/* List of NewColumnValue */
  	bool		new_notnull;	/* T if we added new NOT NULL constraints */
  	bool		rewrite;		/* T if we a rewrite is forced */
+ 	bool		verify;			/* T if we shall verify constraints */
  	Oid			newTableSpace;	/* new tablespace; 0 means no change */
  	/* Objects to rebuild after completing ALTER TYPE operations */
  	List	   *changedConstraintOids;	/* OIDs of constraints to rebuild */
***************
*** 336,342 **** static void ATPrepAlterColumnType(List **wqueue,
  					  AlteredTableInfo *tab, Relation rel,
  					  bool recurse, bool recursing,
  					  AlterTableCmd *cmd, LOCKMODE lockmode);
! static bool ATColumnChangeRequiresRewrite(Node *expr, AttrNumber varattno);
  static void ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
  					  const char *colName, TypeName *typeName, LOCKMODE lockmode);
  static void ATPostAlterTypeCleanup(List **wqueue, AlteredTableInfo *tab, LOCKMODE lockmode);
--- 338,345 ----
  					  AlteredTableInfo *tab, Relation rel,
  					  bool recurse, bool recursing,
  					  AlterTableCmd *cmd, LOCKMODE lockmode);
! static void ATColumnChangeSetWorkLevel(AlteredTableInfo *tab,
! 						   Node *expr, AttrNumber varattno);
  static void ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
  					  const char *colName, TypeName *typeName, LOCKMODE lockmode);
  static void ATPostAlterTypeCleanup(List **wqueue, AlteredTableInfo *tab, LOCKMODE lockmode);
***************
*** 3313,3319 **** ATRewriteTables(List **wqueue, LOCKMODE lockmode)
  			 * Test the current data within the table against new constraints
  			 * generated by ALTER TABLE commands, but don't rebuild data.
  			 */
! 			if (tab->constraints != NIL || tab->new_notnull)
  				ATRewriteTable(tab, InvalidOid, lockmode);
  
  			/*
--- 3316,3322 ----
  			 * Test the current data within the table against new constraints
  			 * generated by ALTER TABLE commands, but don't rebuild data.
  			 */
! 			if (tab->verify)
  				ATRewriteTable(tab, InvalidOid, lockmode);
  
  			/*
***************
*** 3387,3393 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  	Relation	newrel;
  	TupleDesc	oldTupDesc;
  	TupleDesc	newTupDesc;
- 	bool		needscan = false;
  	List	   *notnull_attrs;
  	int			i;
  	ListCell   *l;
--- 3390,3395 ----
***************
*** 3446,3452 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  		switch (con->contype)
  		{
  			case CONSTR_CHECK:
- 				needscan = true;
  				con->qualstate = (List *)
  					ExecPrepareExpr((Expr *) con->qual, estate);
  				break;
--- 3448,3453 ----
***************
*** 3481,3491 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  				!newTupDesc->attrs[i]->attisdropped)
  				notnull_attrs = lappend_int(notnull_attrs, i);
  		}
- 		if (notnull_attrs)
- 			needscan = true;
  	}
  
- 	if (newrel || needscan)
  	{
  		ExprContext *econtext;
  		Datum	   *values;
--- 3482,3489 ----
***************
*** 3549,3558 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  
  		while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
  		{
! 			if (tab->rewrite)
! 			{
! 				Oid			tupOid = InvalidOid;
  
  				/* Extract data from old tuple */
  				heap_deform_tuple(tuple, oldTupDesc, values, isnull);
  				if (oldTupDesc->tdhasoid)
--- 3547,3560 ----
  
  		while ((tuple = heap_getnext(scan, ForwardScanDirection)) != NULL)
  		{
! 			Oid			tupOid = InvalidOid;
  
+ 			if (newrel
+ #ifdef USE_ASSERT_CHECKING
+ 				|| assert_enabled
+ #endif
+ 				)
+ 			{
  				/* Extract data from old tuple */
  				heap_deform_tuple(tuple, oldTupDesc, values, isnull);
  				if (oldTupDesc->tdhasoid)
***************
*** 3561,3570 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
  				/* Set dropped attributes to null in new tuple */
  				foreach(lc, dropped_attrs)
  					isnull[lfirst_int(lc)] = true;
  
  				/*
  				 * Process supplied expressions to replace selected columns.
! 				 * Expression inputs come from the old tuple.
  				 */
  				ExecStoreTuple(tuple, oldslot, InvalidBuffer, false);
  				econtext->ecxt_scantuple = oldslot;
--- 3563,3577 ----
  				/* Set dropped attributes to null in new tuple */
  				foreach(lc, dropped_attrs)
  					isnull[lfirst_int(lc)] = true;
+ 			}
  
+ 			if (tab->newvals != NIL)
+ 			{
  				/*
  				 * Process supplied expressions to replace selected columns.
! 				 * Expression inputs come from the old tuple.  If !newrel, we've
! 				 * proven that no expression will change a tuple value, but they
! 				 * may throw errors.
  				 */
  				ExecStoreTuple(tuple, oldslot, InvalidBuffer, false);
  				econtext->ecxt_scantuple = oldslot;
***************
*** 3577,3584 **** ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
--- 3584,3618 ----
  														  econtext,
  													 &isnull[ex->attnum - 1],
  														  NULL);
+ 
+ #ifdef USE_ASSERT_CHECKING
+ 					if (assert_enabled)
+ 					{
+ 						Datum		oldval = values[ex->attnum - 1];
+ 						bool		oldisnull = isnull[ex->attnum - 1];
+ 						Form_pg_attribute f = newTupDesc->attrs[ex->attnum - 1];
+ 
+ 						if (f->attbyval && f->attlen == -1)
+ 							oldval = PointerGetDatum(PG_DETOAST_DATUM(oldval));
+ 
+ 						/*
+ 						 * We don't detect the gross error of !newrel when the
+ 						 * typlen actually changed.  attbyval could differ in
+ 						 * theory, but we assume it does not.
+ 						 */
+ 						Assert(newrel ||
+ 							   (isnull[ex->attnum - 1] == oldisnull
+ 								&& (oldisnull ||
+ 									datumIsEqual(oldval,
+ 												 values[ex->attnum - 1],
+ 												 f->attbyval, f->attlen))));
+ 					}
+ #endif
  				}
+ 			}
  
+ 			if (newrel)
+ 			{
  				/*
  				 * Form the new tuple. Note that we don't explicitly pfree it,
  				 * since the per-tuple memory context will be reset shortly.
***************
*** 4357,4362 **** ATExecAddColumn(AlteredTableInfo *tab, Relation rel,
--- 4391,4397 ----
  		 * anything.)
  		 */
  		tab->new_notnull |= colDef->is_not_null;
+ 		tab->verify |= tab->new_notnull;
  	}
  
  	/*
***************
*** 4553,4558 **** ATExecSetNotNull(AlteredTableInfo *tab, Relation rel,
--- 4588,4594 ----
  
  		/* Tell Phase 3 it needs to test the constraint */
  		tab->new_notnull = true;
+ 		tab->verify = true;
  	}
  
  	heap_close(attr_rel, RowExclusiveLock);
***************
*** 5270,5275 **** ATAddCheckConstraint(List **wqueue, AlteredTableInfo *tab, Relation rel,
--- 5306,5312 ----
  		newcon->qual = (Node *) make_ands_implicit((Expr *) ccon->expr);
  
  		tab->constraints = lappend(tab->constraints, newcon);
+ 		tab->verify = true;
  
  		/* Save the actually assigned name if it was defaulted */
  		if (constr->conname == NULL)
***************
*** 5618,5623 **** ATAddForeignKeyConstraint(AlteredTableInfo *tab, Relation rel,
--- 5655,5664 ----
  		newcon->qual = (Node *) fkconstraint;
  
  		tab->constraints = lappend(tab->constraints, newcon);
+ 		/*
+ 		 * No need to set tab->verify; foreign key validation is a distinct
+ 		 * aspect of Phase 3.
+ 		 */
  	}
  
  	/*
***************
*** 6581,6588 **** ATPrepAlterColumnType(List **wqueue,
  		newval->expr = (Expr *) transform;
  
  		tab->newvals = lappend(tab->newvals, newval);
! 		if (ATColumnChangeRequiresRewrite(transform, attnum))
! 			tab->rewrite = true;
  	}
  	else if (tab->relkind == RELKIND_FOREIGN_TABLE)
  	{
--- 6622,6628 ----
  		newval->expr = (Expr *) transform;
  
  		tab->newvals = lappend(tab->newvals, newval);
! 		ATColumnChangeSetWorkLevel(tab, transform, attnum);
  	}
  	else if (tab->relkind == RELKIND_FOREIGN_TABLE)
  	{
***************
*** 6623,6647 **** ATPrepAlterColumnType(List **wqueue,
  }
  
  /*
!  * When the data type of a column is changed, a rewrite might not be require
!  * if the data type is being changed to its current type, or more interestingly
!  * to a type to which it is binary coercible.  But we must check carefully that
!  * the USING clause isn't trying to insert some other value.
   */
! static bool
! ATColumnChangeRequiresRewrite(Node *expr, AttrNumber varattno)
  {
  	Assert(expr != NULL);
  
! 	for (;;)
  	{
! 		/* only one varno, so no need to check that */
  		if (IsA(expr, Var) && ((Var *) expr)->varattno == varattno)
! 			return false;
  		else if (IsA(expr, RelabelType))
  			expr = (Node *) ((RelabelType *) expr)->arg;
  		else
! 			return true;
  	}
  }
  
--- 6663,6722 ----
  }
  
  /*
!  * Decide how much phase-3 per-tuple work this column type change entails, based
!  * on its transformation expression tree.  There are three possibilities:
!  *
!  *  1. Rewrite; always sufficient.  Example: int2 -> int4.
!  *  2. Verification scan.  Example: text -> constrained domain over text.
!  *  3. No per-tuple work.  Example: varchar(N) -> text.
!  *
!  * To certify #2 or #3 as sufficient, the transformation expression tree must
!  * consist solely of nodes from this list:
!  *
!  *	A. Var with the varattno of the column under alteration.  Allows #2 or #3.
!  *	B. RelabelType.  Allows #2 or #3.
!  *	C. CoerceToDomain.  Allows #2; allows #3 when GetDomainConstraints() == NIL.
!  *
!  * (A) primordially conveys the current column data.  (B) and (C) are acceptable
!  * because they always pass a datum through unchanged; any other node having
!  * that property could potentially join this list.
   */
! static void
! ATColumnChangeSetWorkLevel(AlteredTableInfo *tab,
! 						   Node *expr, AttrNumber varattno)
  {
  	Assert(expr != NULL);
  
! 	/*
! 	 * Walking an expression tree is normally a recursive operation.  Since all
! 	 * node types on our current whitelist have no more than one descendent node
! 	 * requiring assessment, the recursion degenerates to iteration.
! 	 *
! 	 * When tab->rewrite becomes true, no optimizations are possible, and we're
! 	 * done.  It may be set already from analysis of an earlier subcommand in
! 	 * the same ALTER TABLE operation, and that's fine.
! 	 */
! 	while (!tab->rewrite)
  	{
! 		/*
! 		 * Non-matching varattno arises when an explicit USING clause references
! 		 * a different column.  Only one varno, so no need to check that.  When
! 		 * we reach this node, we're done, and some optimizations do apply.
! 		 */
  		if (IsA(expr, Var) && ((Var *) expr)->varattno == varattno)
! 			break;
  		else if (IsA(expr, RelabelType))
  			expr = (Node *) ((RelabelType *) expr)->arg;
+ 		else if (IsA(expr, CoerceToDomain))
+ 		{
+ 			CoerceToDomain *d = (CoerceToDomain *) expr;
+ 
+ 			if (GetDomainConstraints(d->resulttype) != NIL)
+ 				tab->verify = true;
+ 			expr = (Node *) d->arg;
+ 		}
  		else
! 			tab->rewrite = true;
  	}
  }
  
diff --git a/src/test/regress/GNUmakefiindex 4c8af0f..eeb9487 100644
diff --git a/src/test/regress/expecnew file mode 100644
index 0000000..248fb39
diff --git a/src/test/regress/parallel_schedule b/srindex 3b99e86..0420922 100644
diff --git a/src/test/regress/serial_scheindex b348f0e..9954cf7 100644
diff --git a/src/test/regress/sql/big_anew file mode 100644
index 0000000..8302d87
#42Robert Haas
robertmhaas@gmail.com
In reply to: Noah Misch (#37)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

On Mon, Feb 14, 2011 at 7:52 AM, Noah Misch <noah@leadboat.com> wrote:

I'm half-tempted to put that part off to
9.2, in the hopes of getting a more substantial solution that can also
handle things like text -> xml which we don't have time to re-engineer
right now.

I see.

After sleeping on it, I think this route makes most sense. The
ability to downgrade a rewrite to a scan is really a separate feature,
and I'd like to see us implement that in a more complete way when/if
we're going to do it; and I'd rather commit it at the beginning of a
development cycle when we have more time to find any lurking bugs. So
I've committed a change that just handles the unconstrained domain
case. I think for 9.2 we should revisit the following areas:

1. Downgrading rewrites to scans (vs. skipping them altogether). One
idea is that we might modify CREATE CAST so that you can do this:

CREATE CAST (source_type AS target_type) WITH [ CHECK ] FUNCTION
function_name (argument_type [, ...])
[ AS ASSIGNMENT | AS IMPLICIT ];

The inclusion of the keyword "check" there would inform the system
that the binary representation can't change, but (as distinguished
from WITHOUT FUNCTION) an error might be thrown. Of course, I'm not
quite sure how to get this information over to the alter table
machinery cleanly.

2. Detecting binary-coercible cases that involve typemods, rather than
just type OIDs.

3. Avoiding index rebuilds.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#43Stephen Frost
sfrost@snowman.net
In reply to: Noah Misch (#41)
Re: ALTER TYPE 2: skip already-provable no-work rewrites

Noah,

* Noah Misch (noah@leadboat.com) wrote:

On Mon, Feb 14, 2011 at 04:06:59PM -0500, Stephen Frost wrote:
I've attached a new version of the patch that attempts to flesh out the comments
based on your feedback. Does it improve things?

Yes, much better, thanks!

Offhand, I can't think of any concrete implementation along those lines that
would simplify the overall task. Did you have something more specific in mind?

Sadly, no. :)

For now it seemed like premature abstraction.

Fair enough.

All in all, this patch looks good to me. Looks like that might be moot,
however, based on Robert's comments. Still, thanks for it, I certainly
look forward to having it eventually. :)

Thanks again,

Stephen